Commit graph

7 commits

Author SHA1 Message Date
Marco Beretta
6f4e36665e
🗄️ perf: Cut Serial and Redundant Database Reads (#16711)
* perf: Cut Serial and Redundant Database Reads

* perf: Pick the Earliest Balance per User in MongoDB
2026-10-03 10:41:47 -04:00
Danny Avila
696bcb6c1e
🏦 fix: Reserve Credits for In-Flight Requests (#15879)
* 💳 fix: Reserve Balance for In-Flight Requests

Admit each balance-checked request against the credits that other
in-flight requests have not already reserved, and hold its prompt cost
on the balance record until the turn settles. Apply a due auto-refill in
the same fenced write so one refill window credits exactly once.

- data-schemas: reserveBalance / releaseBalanceReservation replace
  createAutoRefillTransaction; the write is a compare-and-swap on
  tokenCredits, reservations and lastRefill (no pipeline updates)
- balance.reservations is select:false so existing reads never see it
- checkBalance returns a reservation handle; BaseClient and the
  assistants controllers release it once the turn settles
- balance.reservationTtlMs bounds how long an unreleased reservation
  counts (default 30 minutes)

* 🧮 refactor: Admit Reservations Without Contention and Record Refills Durably

- admission is a conditional $push/$inc against a reservedCredits total,
  so concurrent admissions of a funded balance commit on their first write
- expired reservations are pruned per element with their credits
- auto-refill is fenced on every setting its decision reads and leaves a
  pendingRefill ledger marker in the same write; any later read records the
  transaction under the fixed id and clears the marker
- a missing record is created with $setOnInsert inside reserveBalance
- reservation lifecycle moves to packages/api (createBalanceReservations,
  withBalanceReservations); BaseClient releases once usage is recorded

* 🪪 fix: Resolve Each User to One Balance Record

- balance records a creator inserts are keyed by the user id, so creators
  racing on a missing record converge on one document; every balance read
  and write resolves a user to their oldest record, so existing duplicates
  stay inert
- upsertBalanceFields takes insert-only fields; the balance-config
  middleware, schedules and lazy initialization pass the starting credit
  through them so it never overwrites a record another writer created
- the auto-refill write is also fenced on reservedCredits
- the scheduled-chat balance pre-skip counts reserved credits as unavailable

* ⏳ fix: Count Only Unexpired Reservations in Scheduled Balance Checks

findBalanceByUser with includeReservedCredits now totals the reservations
that have not expired, so a reservation left by a crashed request stops
pre-skipping scheduled chats at its expiry instead of when a later
reservation write prunes it.

* 🔁 fix: Renew Balance Reservations While Their Request Runs

A reservation is renewed every half TTL until it is released, so only a
reservation whose process stopped expires. Pruning removes a reservation
only while it is still expired at the write, so one renewed after the
pruning read survives.

* 🛟 fix: Keep Reservations Alive Through Renewal Failures and Lost Acks

- renewal reschedules itself: every half TTL, capped to the timer range,
  and a failed renewal retries within a tenth of that window
- an admission write that errors removes its possibly committed
  reservation before rethrowing

* 🛑 fix: Hold a Stopped Turn's Reservation Until Its Charge Lands

- an aborted turn lets its reservations lapse after a minute (or the TTL)
  instead of releasing them, so the Stop request's charge lands before the
  credits are admitted again
- balance.reservationTtlMs has a 10 second minimum, enforced in the schema
  and at runtime, so renewal stays bounded

* ↩️ revert: Release a Stopped Turn's Reservation When the Turn Settles

An aborted turn releases its reservations when it settles, the same as
every other exit, instead of holding them for a fixed lapse window that
refused funded requests after every Stop. The minimum reservation TTL
stays.

* ⏸️ fix: Hold Assistants Reservations Through Background Run Retries

An in-progress Assistants run continues in the background after the
handler responds; the turn's reservations now stay held until that run
settles (createBalanceReservations().holdUntil).

* 🔁 fix: Apply an Auto-Refill Once per Admission

A refill interval that is due again the moment it is applied, such as a zero
interval, let one admission refill repeatedly until the retry bound. Each
admission now applies at most one refill, as the previous check did.

* 🧹 fix: Prune Expired Reservations in One Write

An admission that found expired reservations issued one write per hold, so a
backlog left by a stopped process fanned out into as many concurrent writes.
The prune is now a single write fenced on every hold it removes; a hold renewed
after the read makes the write miss and the admission re-read.

* ⏱️ fix: Renew Reservations From Their Stored Expiry

The next renewal was scheduled half a TTL after the previous write settled,
and the first one half a TTL after the admission returned, so a slow write
could let a live hold expire before its renewal. Renewals are now scheduled
from the expiry the last write stored.

Expired holds are pruned as one bulk write of per-hold fenced updates, so a
hold that changed after the read is skipped while the rest are removed, and
the admission no longer spends a retry on the whole prune.
2026-09-13 12:53:19 -04:00
Helge Wiethoff
d6c173c94b
🐍 fix: Use Standard Mongoose Module Resolution in Config Scripts (#9143) 2025-08-19 11:15:09 -04:00
Danny Avila
a2fc7d312a
🏗️ refactor: Extract DB layers to data-schemas for shared use (#7650)
* refactor: move model definitions and database-related methods to packages/data-schemas

* ci: update tests due to new DB structure

fix: disable mocking `librechat-data-provider`

feat: Add schema exports to data-schemas package

- Introduced a new schema module that exports various schemas including action, agent, and user schemas.
- Updated index.ts to include the new schema exports for better modularity and organization.

ci: fix appleStrategy tests

fix: Agent.spec.js

ci: refactor handleTools tests to use MongoMemoryServer for in-memory database

fix: getLogStores imports

ci: update banViolation tests to use MongoMemoryServer and improve session mocking

test: refactor samlStrategy tests to improve mock configurations and user handling

ci: fix crypto mock in handleText tests for improved accuracy

ci: refactor spendTokens tests to improve model imports and setup

ci: refactor Message model tests to use MongoMemoryServer and improve database interactions

* refactor: streamline IMessage interface and move feedback properties to types/message.ts

* refactor: use exported initializeRoles from `data-schemas`, remove api workspace version (this serves as an example of future migrations that still need to happen)

* refactor: update model imports to use destructuring from `~/db/models` for consistency and clarity

* refactor: remove unused mongoose imports from model files for cleaner code

* refactor: remove unused mongoose imports from Share, Prompt, and Transaction model files for cleaner code

* refactor: remove unused import in Transaction model for cleaner code

* ci: update deploy workflow to reference new Docker Dev Branch Images Build and add new workflow for building Docker images on dev branch

* chore: cleanup imports
2025-05-30 22:18:13 -04:00
Danny Avila
6fc664e4a3
⚙️ feat: includedTools and script changes (#2690)
* chore: add email to npm scripts (user-stats and list-balances)

* feat: included tools

* chore: update console terminal links

* chore: add back typing
2024-05-13 10:07:10 -04:00
Danny Avila
050a92b318
🔄 refactor(config): Move connectWithTimeout Outside of Helpers Module (#1513) 2024-01-07 14:43:27 -05:00
Linus Gasser
1a95bef677
📃 feat: add list-balances, remove-user, and improve User scripts (#1418)
* Refactoring opening of DB to config/helpers.js

* Adding two user scripts:

- 'delete-user' to remove a user definitely
- 'list-balances' to show the balances of all the users
2023-12-30 13:25:12 -05:00