mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 14:08:21 +00:00
8 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4e0d9145e0
|
⚛️ feat: Atomic Admin Config Changes Endpoint (set, unset, tombstone, entry upsert/remove/rename) (#17328)
* 🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes Admin config writes went straight to MongoDB with dotted-path `$set`, `$unset`, `$pull` and `$addToSet`, and nothing checked `configVersion`. That caused six problems. * ⚛️ feat: Atomic Admin Config Changes Endpoint (set, unset, tombstone, entry upsert/remove/rename) Admin clients can change a principal's config overrides only one kind of change at a time: - PATCH `/fields` for sets, at most 100 entries; - POST `/fields/tombstone` for one tombstone; - DELETE `/fields` for one reset. * 🔁 fix: Reload librechat.yaml on Admin Config Writes Base-only sections such as `filters` and `messageFilter` live only in librechat.yaml, and an admin config write was how a running server picked up an edit to them. The coherent-cache change limited admin-write invalidation to the override and MCP caches, so the YAML base was never re-read after startup: the content-filter e2e suite, which activates filters by editing the YAML and writing a reload sentinel, saw `filters: undefined` from GET /base and every activation timed out. Admin config writes clear the YAML base config again, alongside the override and MCP caches, and the clear is broadcast with the others, so every instance reloads its YAML. The reload keeps the last good config on failure and republishes the tools derived from it, so the global tool cache is left for it to overwrite rather than emptied first. * 🧭 fix: Cap Dotted Field Path Depth and Hold Revision Preconditions on No-Op Writes A dotted fieldPath was split without the 32-segment cap that segments already had, so a deep path reached the recursive path writer and failed with a 500. Both spellings now share MAX_CONFIG_PATH_DEPTH and answer 400 CONFIG_PATH_INVALID / too_deep on PATCH, tombstone and DELETE /fields. Writes reduced to nothing (protected interface permissions, base-only or dedicated sections) answered 200 without checking expectedRevision. They now compare it with the stored document and answer 409 CONFIG_VERSION_CONFLICT with currentRevision on a mismatch, as the write itself would. * 📡 fix: Retry the Config Invalidation Subscription When Redis Comes Back A SUBSCRIBE that failed at startup (Redis unreachable, the client's retries spent) was only logged, and ioredis resubscribes only to channels it has joined, so the instance missed every other instance's config invalidation until it restarted. The subscription is now sent again each time the subscriber becomes ready until one succeeds; after that the client's own resubscribe takes over, and closing stops the retries. * 🧷 fix: Tighten Config Change Batches Around Array Entries, Cancelled Changes and MCP Tombstones - A tombstone with an index segment below a field configSchema does not define (an unknown key, or what an intersection or any holds) is refused as inside_array: the merge removes inherited values from objects only, so it was stored and reported applied while the element stayed visible. - unset joins set and rename in the merged-by-key position check: unset endpoints.custom.0.headers answers 400 indexed_merge_key_write, since the position can name another endpoint by the time the batch applies. upsertEntry and removeEntry address entries by key. - A batch whose changes cancel out (set interface.customWelcome, then unset interface) writes nothing: no empty document, revision bump, audit entry or invalidation. Its changes are reported as unchanged. - A whole-section mcpServers tombstone reports each librechat.yaml stdio server it leaves in place as ignored (process_mcp_server). - removeEntry no longer attributes the whole collection to itself, and existing issues of merged-by-key entries are matched by key rather than index, so removing an entry beside a legacy-invalid one (or shifting it) is no longer refused for that entry's issue. * 🪪 fix: Keep a Requested Priority on New Principals and Report Ignored MCP Renames POST …/changes discarded a broad manager's priority for a principal without a document when no change altered it: the batch answered written: false instead of creating the priority-only document PUT creates. An explicit priority on an absent principal now counts as a change. A rename under an MCP server librechat.yaml runs as a process persisted an override the runtime merge drops, but was not reported in ignored. It now is, like set and tombstone. An unset there stays unreported: it asks for the YAML value, which is what runs. * 🔁 fix: Drop YAML Reloads and Merges Overtaken by a Config Write Admin config writes clear the YAML base config on every instance, so a write is how a running server picks up an edit to librechat.yaml. Two races could undo that: - A base reload that read the old YAML before a write's clear cached it after the clear, and the base has no TTL, so the old base was served until the next write. - A request that read the old base before the clear, and was still resolving principals, cached its merge under the new override generation for the override TTL. The config service now keeps a base-config generation that `clearAppConfigCache` bumps. A reload publishes the base, the tools and the last good config only if its generation is still current, a read after a clear never joins a reload started before it, and the newer reload starts once the stale one settles so the loader's process-wide side effects end with the newest YAML. Merges capture the generation, base included, before they read the base config. Tests: - service.spec: a clear during a reload (stale success, stale failure, a read waiting on the newer reload) and a merge built on the pre-clear base. - invalidation.redis.spec: a write on one instance makes another reload librechat.yaml. - admin-config-reload.spec (api): every admin config write route, through the real router, handlers, database and config service, reloads librechat.yaml; a request with nothing to write does not. - invalidateConfigCaches.spec no longer asserts an order the parallel clears do not have. * 🧪 test: Reload librechat.yaml After a Committed Change Batch and Not After an Unwritten One * 🧮 fix: Validate Change Batches on Their End State and Report Base-Only Fields a Tombstone Keeps A batch's validation scope was built from every intermediate write, located in the state each change produced. A change a later one undid (set `balance.startBalance`, then restore `balance`) still widened the scope, so an issue the stored document already had there rejected an unrelated change, and an entry a later change moved (upsert one entry, then remove one before it) pointed its written index at a neighbour with a legacy issue. The written paths are now located in the batch's end state and kept only where that state differs from the stored document, with entries of merged-by-key arrays compared by key. A tombstone above a base-only field (`endpoints.agents`, `endpoints.agents.pullRequests`) is stored and hides the rest of the subtree, while the runtime merge puts back the provider, host, OAuth app and shared cache slots librechat.yaml sets. The response now lists those fields in `ignored` with reason `base_only_path`, as it already does for the stdio MCP servers a whole-section tombstone leaves in place. * 🧬 fix: Encode Config Keys in Objects Built in Another Realm The key encoder treated an object as plain only when its prototype was this realm's Object.prototype, and returned any other object unchanged, keys and all. Under Jest, structuredClone (which the secret pipeline uses) returns objects from the host realm, so the items of an array the pipeline had cloned, such as Azure groups or custom endpoints, reached the engine with their dotted keys raw. Node in production has one realm, but the encoding must not depend on prototype identity: a skipped object stores its dotted keys raw without an error. A plain object is now one whose prototype is null or a root Object.prototype from any realm. The encoding spec covers an object built in a vm context. * 🧪 test: Check Dotted Record Keys Reach the Store's Whole-Document Write dev's admin config route test asserted the keys on patchConfigFields, which the handlers no longer call; it now asserts them on the draft writeConfigDocument receives. * 🚩 feat: Publish the expectedRevision, prunesEmptyParents and awaitedInvalidation Config Flags GET /api/admin/config/meta now reports the three write-coherence behaviours this branch implements, so an admin client can drop the workarounds each one replaces: - expectedRevision: responses carry `revision`; PUT, PATCH /fields, POST /fields/tombstone and PATCH /active take `expectedRevision` in the body and both DELETE routes in the query. A mismatch answers 409 CONFIG_VERSION_CONFLICT with currentRevision, a malformed value 400 INVALID_EXPECTED_REVISION; unpinned writes re-read and re-apply up to 3 times. - prunesEmptyParents: a field delete or tombstone removes the objects it empties, stopping at arrays; shells-only documents count as empty for assign-only lifecycle calls. - awaitedInvalidation: writes answer after the config caches are cleared (broadcast with Redis), GET /base bypasses the merged-config cache, and a merge read before a clear is never cached. meta.spec probes each one against the behaviour: a stale and a malformed precondition on a PATCH, the store operations' unset and tombstone on nested objects and an array element, and a held invalidation, the GET /base read options and a merge raced by a clear. * 🔀 fix: Keep Config Keys Raw Where the Engine Stores Them, So Older Servers Read Them During Upgrades Servers before the key encoding (v0.8.9-rc1 among them) read every override key literally. The Mongoose binding encoded every write, so during a rolling upgrade the first write from an upgraded server turned a stored `gpt-4.1` or Bedrock inference profile into `gpt-4%2E1` for every server not yet upgraded, and an older server's restore of a deleted group or role config dropped the `keyEncoding` marker its schema lacks, after which upgraded servers read the encoded keys literally. The binding now sends keys raw first and encodes them only when the engine refuses a raw field name (DocumentDB, Cosmos, FerretDB or MongoDB 4.4 for `$` names): the refused write changed nothing, so it runs again encoded under the same revision precondition, and that process encodes later writes holding such keys directly. MongoDB 4.4+ stores dotted names (5.0+ also `$` names inside a value), so there an older server reads exactly what it wrote. A document whose raw keys spell escape sequences while none holds a raw `.`, `$` or NUL is still written encoded, since a raw read would decode it. A raw write clears the marker. Reads decode a marked document (raw keys an older server added beside encoded ones, or wrote under a marker it left, carry no escape sequence and read as stored) and an unmarked one unless some key holds a raw `.`, `$` or NUL, which no encoded document does; that reads an encoded document whose marker an older server dropped. Encoding stays in the binding for every engine that needs it (decision #15); the residual cases are in documentdb-compat.md. * ⏱️ fix: Remove Config Cache Writes a Clear Overtook in Flight With an asynchronous APP_CONFIG store (Redis or a custom Keyv store), a clear can bump the generation and finish its delete after a reload or merge checked the generation but before its `set` landed. The older value then repopulated `_BASE_` until the next write, or a merged config for the full override TTL. The base reload, the last-good restoration and the merge cache now check the generation again once their write lands and delete what they wrote if a clear overtook it. A deleted entry only costs the next read a reload or a merge; a newer base load cannot be removed this way, since it starts only after the stale one settles. * 🔐 fix: Require the Stored Web Search Key Again When a Change Batch Moves Its URL #17323 refuses a PUT, PATCH, tombstone or field delete that changes or removes a web search URL beside a stored API key unless the key is entered again, so a stored key cannot be sent to a destination it was not entered for. POST /changes skipped that check: a batch could set, reset or tombstone `webSearch.searxngInstanceUrl` (or replace `webSearch` without the key, which keeps the stored key) and point the stored key at another host. A batch is now judged on the document it would store, like PATCH: a URL that ends up beside a stored key it did not travel with answers 400 CONFIG_SECRET_REENTRY_REQUIRED and writes nothing, dry runs included. Only keys the request sent count as entered again; a secret a set kept from the stored document does not. * 🚩 feat: Publish the changesBatch Flag and the maxChangeOps Limit in Config Meta GET /api/admin/config/meta now reports `features.changesBatch: true` and `limits.maxChangeOps` (1000, from MAX_CONFIG_CHANGE_OPS), so the admin panel can gate its one-request save on the endpoint and size its batches from the server. The meta.spec probe checks the behaviour itself: an oversized batch answers TOO_MANY_OPS, a batch with one invalid op writes nothing, and a valid set plus tombstone lands as one write. |
||
|
|
0b4dc238b3
|
🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes (#17326)
* 🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes Admin config writes went straight to MongoDB with dotted-path `$set`, `$unset`, `$pull` and `$addToSet`, and nothing checked `configVersion`. That caused six problems. * 🔁 fix: Reload librechat.yaml on Admin Config Writes Base-only sections such as `filters` and `messageFilter` live only in librechat.yaml, and an admin config write was how a running server picked up an edit to them. The coherent-cache change limited admin-write invalidation to the override and MCP caches, so the YAML base was never re-read after startup: the content-filter e2e suite, which activates filters by editing the YAML and writing a reload sentinel, saw `filters: undefined` from GET /base and every activation timed out. Admin config writes clear the YAML base config again, alongside the override and MCP caches, and the clear is broadcast with the others, so every instance reloads its YAML. The reload keeps the last good config on failure and republishes the tools derived from it, so the global tool cache is left for it to overwrite rather than emptied first. * 🧭 fix: Cap Dotted Field Path Depth and Hold Revision Preconditions on No-Op Writes A dotted fieldPath was split without the 32-segment cap that segments already had, so a deep path reached the recursive path writer and failed with a 500. Both spellings now share MAX_CONFIG_PATH_DEPTH and answer 400 CONFIG_PATH_INVALID / too_deep on PATCH, tombstone and DELETE /fields. Writes reduced to nothing (protected interface permissions, base-only or dedicated sections) answered 200 without checking expectedRevision. They now compare it with the stored document and answer 409 CONFIG_VERSION_CONFLICT with currentRevision on a mismatch, as the write itself would. * 📡 fix: Retry the Config Invalidation Subscription When Redis Comes Back A SUBSCRIBE that failed at startup (Redis unreachable, the client's retries spent) was only logged, and ioredis resubscribes only to channels it has joined, so the instance missed every other instance's config invalidation until it restarted. The subscription is now sent again each time the subscriber becomes ready until one succeeds; after that the client's own resubscribe takes over, and closing stops the retries. * 🔁 fix: Drop YAML Reloads and Merges Overtaken by a Config Write Admin config writes clear the YAML base config on every instance, so a write is how a running server picks up an edit to librechat.yaml. Two races could undo that: - A base reload that read the old YAML before a write's clear cached it after the clear, and the base has no TTL, so the old base was served until the next write. - A request that read the old base before the clear, and was still resolving principals, cached its merge under the new override generation for the override TTL. The config service now keeps a base-config generation that `clearAppConfigCache` bumps. A reload publishes the base, the tools and the last good config only if its generation is still current, a read after a clear never joins a reload started before it, and the newer reload starts once the stale one settles so the loader's process-wide side effects end with the newest YAML. Merges capture the generation, base included, before they read the base config. Tests: - service.spec: a clear during a reload (stale success, stale failure, a read waiting on the newer reload) and a merge built on the pre-clear base. - invalidation.redis.spec: a write on one instance makes another reload librechat.yaml. - admin-config-reload.spec (api): every admin config write route, through the real router, handlers, database and config service, reloads librechat.yaml; a request with nothing to write does not. - invalidateConfigCaches.spec no longer asserts an order the parallel clears do not have. * 🧬 fix: Encode Config Keys in Objects Built in Another Realm The key encoder treated an object as plain only when its prototype was this realm's Object.prototype, and returned any other object unchanged, keys and all. Under Jest, structuredClone (which the secret pipeline uses) returns objects from the host realm, so the items of an array the pipeline had cloned, such as Azure groups or custom endpoints, reached the engine with their dotted keys raw. Node in production has one realm, but the encoding must not depend on prototype identity: a skipped object stores its dotted keys raw without an error. A plain object is now one whose prototype is null or a root Object.prototype from any realm. The encoding spec covers an object built in a vm context. * 🧪 test: Check Dotted Record Keys Reach the Store's Whole-Document Write dev's admin config route test asserted the keys on patchConfigFields, which the handlers no longer call; it now asserts them on the draft writeConfigDocument receives. * 🚩 feat: Publish the expectedRevision, prunesEmptyParents and awaitedInvalidation Config Flags GET /api/admin/config/meta now reports the three write-coherence behaviours this branch implements, so an admin client can drop the workarounds each one replaces: - expectedRevision: responses carry `revision`; PUT, PATCH /fields, POST /fields/tombstone and PATCH /active take `expectedRevision` in the body and both DELETE routes in the query. A mismatch answers 409 CONFIG_VERSION_CONFLICT with currentRevision, a malformed value 400 INVALID_EXPECTED_REVISION; unpinned writes re-read and re-apply up to 3 times. - prunesEmptyParents: a field delete or tombstone removes the objects it empties, stopping at arrays; shells-only documents count as empty for assign-only lifecycle calls. - awaitedInvalidation: writes answer after the config caches are cleared (broadcast with Redis), GET /base bypasses the merged-config cache, and a merge read before a clear is never cached. meta.spec probes each one against the behaviour: a stale and a malformed precondition on a PATCH, the store operations' unset and tombstone on nested objects and an array element, and a held invalidation, the GET /base read options and a merge raced by a clear. * 🔀 fix: Keep Config Keys Raw Where the Engine Stores Them, So Older Servers Read Them During Upgrades Servers before the key encoding (v0.8.9-rc1 among them) read every override key literally. The Mongoose binding encoded every write, so during a rolling upgrade the first write from an upgraded server turned a stored `gpt-4.1` or Bedrock inference profile into `gpt-4%2E1` for every server not yet upgraded, and an older server's restore of a deleted group or role config dropped the `keyEncoding` marker its schema lacks, after which upgraded servers read the encoded keys literally. The binding now sends keys raw first and encodes them only when the engine refuses a raw field name (DocumentDB, Cosmos, FerretDB or MongoDB 4.4 for `$` names): the refused write changed nothing, so it runs again encoded under the same revision precondition, and that process encodes later writes holding such keys directly. MongoDB 4.4+ stores dotted names (5.0+ also `$` names inside a value), so there an older server reads exactly what it wrote. A document whose raw keys spell escape sequences while none holds a raw `.`, `$` or NUL is still written encoded, since a raw read would decode it. A raw write clears the marker. Reads decode a marked document (raw keys an older server added beside encoded ones, or wrote under a marker it left, carry no escape sequence and read as stored) and an unmarked one unless some key holds a raw `.`, `$` or NUL, which no encoded document does; that reads an encoded document whose marker an older server dropped. Encoding stays in the binding for every engine that needs it (decision #15); the residual cases are in documentdb-compat.md. * ⏱️ fix: Remove Config Cache Writes a Clear Overtook in Flight With an asynchronous APP_CONFIG store (Redis or a custom Keyv store), a clear can bump the generation and finish its delete after a reload or merge checked the generation but before its `set` landed. The older value then repopulated `_BASE_` until the next write, or a merged config for the full override TTL. The base reload, the last-good restoration and the merge cache now check the generation again once their write lands and delete what they wrote if a clear overtook it. A deleted entry only costs the next read a reload or a merge; a newer base load cannot be removed this way, since it starts only after the stale one settles. |
||
|
|
2591a15c10
|
🔑 feat: Keep Dotted Record Keys in Admin Config Writes and Publish the Config Contract (#17322)
* 🔑 feat: Keep Dotted Record Keys in Admin Config Writes and Publish the Config Contract Keep dotted record keys in admin config write bodies behind a path-scoped sanitizer, reject $-prefixed and prototype keys after auth, report dotted_key issues with segments, and publish GET /api/admin/config/meta (features, rules, limits, versions) so clients can gate on backend capabilities. * 🧪 test: Expect Issue Segments in the Config Override e2e Scenario Override validation issues now carry `segments`; assert the full issue shape. |
||
|
|
84851d8655
|
🍃 fix: Serialize Index Builds and the Agent Edge Cleanup for DocumentDB (#15670)
* 🍃 fix: Serialize Index Builds and the Agent Edge Cleanup for DocumentDB Amazon DocumentDB admits one index build per collection at a time and rejects a second with code 40333. MongoDBSaver.setup() starts the compound and TTL builds of each checkpoint collection concurrently and returns the rejections, which buildMongoSaver logged and ignored, so the TTL index was the likely loser on every boot and checkpoints never expired. setupCheckpointIndexes re-runs the idempotent setup() while any rejection is a 40333. auditLog and the two index migrations bypassed createIndexesWithRetry; buildIndexWithRetry is the raw-collection form and they use it. removeAgentIdsFromEdges pruned deleted agent ids with an aggregation-pipeline update, which DocumentDB rejects; the guard missed it because the pipeline reached updateMany as a function return value. It is now a read, a prune in code, and a compare-and-set bulk write, keeping the per-graph atomicity the pipeline had. The static guard now carries the full 5.0 unsupported-operator matrix, rejects $set/$unset at stage position and the $count accumulator, follows calls to functions that return an array, and holds every index build to the retrying helpers. * 🍃 fix: Keep Companion Setup Errors Past the Deadline and Trace Appended Stages Codex round 1. setupCheckpointIndexes now carries the non-conflict errors of the last pass and returns them beside the conflict when the peer-build deadline passes, so a permanent failure on one collection is still logged when the other stays busy. The alias-stage detector also inspects the arguments of push, unshift and concat and both branches of a conditional element, since a pipeline assembled by push was invisible to it. Both tests fail without their fix. * 🍃 fix: Page the Edge Cleanup, Count the Build Deadline From the First Conflict, and Widen the Guard Codex round 2 and Copilot. buildWhenCollectionFree now starts its deadline at the first conflict rather than the first attempt, so a saver that admits one long build before reporting a companion 40333 still gets its rerun instead of caching with an index missing. removeAgentIdsFromEdges reads matching graphs a page at a time (sort by _id, limit) and uses the filter as its cursor, bounding application memory the way the server-side update did; the loop is bounded by passes without progress rather than a fixed attempt count. The guard follows array-returning class and object methods, resolves stages held in variables, and exempts the Meilisearch createIndex by its primaryKey option instead of by file. retryWithBackoff rejects NaN options instead of skipping every attempt. Every new test fails without its change. * 🍃 fix: Walk the Edge Cleanup Behind an _id Cursor and Propagate Setup Rejections Codex round 3. removeAgentIdsFromEdges pages behind an _id cursor so every pass resumes where the last one stopped instead of rescanning the cleaned prefix, with one sweep from the top once the cursor is exhausted to pick up compare-and-set misses; a test lands a concurrent edit on a first-page graph and checks the miss is cleaned. setupCheckpointIndexes rethrows a rejection of setup() itself so buildMongoSaver keeps its in-process fallback, converting only a build conflict that outlasts the deadline. The guard resolves a returned identifier against array-bound variables, so a pipeline built in a local and returned is still a pipeline update. * 🍃 fix: Bound Cleanup Sweeps and the Audit Index Wait, Reuse the Removed-Id Set Codex round 4. removeAgentIdsFromEdges gives up after a bounded number of sweeps from the top when references keep being added, since progress otherwise resets the stall bound forever (reachable in deleteUserAgents, which deletes the agents after cleanup); the removed-id set is built once per cleanup rather than per graph. The audit append index build on the request path now has a deadline, so a peer holding the collection makes the append fail open instead of holding the request; createAuditLogMethods takes optional index-build overrides so a test can shorten it. The guard treats a variable annotated with an array type as array-valued whatever its initializer, and states that dataflow is followed within a file only. |
||
|
|
c13990a398
|
🍃 test: Restore Sweep Coverage of Typed-Criteria Methods (#15609)
Some checks failed
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
* 🍃 test: Restore Sweep Coverage of Typed-Criteria Methods The typed-criteria refactor (#15580) validates query criteria key by key and fails closed on anything unrecognized — correctly, since a silently accepted bad criterion would widen a filter into an unscoped `find` or `deleteMany`. The sweep synthesizes arguments from parameter names, so a parameter named `query` received a string, which `buildFilter` read as a criteria object and rejected at index '0'. Six methods therefore went from adjudicated to un-driven with the suite still green: `getActions`, `getAssistants`, `deleteAction(s)`, and `deleteAssistant(s)`. Criteria-shaped parameters are now synthesized as an empty object, which builds an empty filter and still reaches the engine. Recovered 13 methods in total — the six regressions plus seven that had been silently un-driven for the same reason (prompts, groups, conversation tags), including through the runs recorded as authoritative. Baseline coverage rises from 372 to 385 of 518 methods. The compatibility assessment now flags this count as the thing to watch across releases: a coverage regression hides behind a passing suite, and only a matrix diff surfaces it. * 🍃 test: Preserve searchMessages' String Query in the Sweep Criteria-shaped synthesis is name-based, so it also caught `searchMessages(query: string, ...)`, whose parameter is a genuine search string. On a Meili-enabled deployment that object is forwarded verbatim into `meiliSearch`, so the recorded outcome would reflect a harness-generated request rather than the method's behavior. `searchMessages` gets an `ARG_OVERRIDES` entry supplying a real query string; its failure is now the honest 'MeiliSearch plugin not registered' rather than an invalid request. It is the only method in `src/methods` whose `query`/`criteria`/`filter` parameter is string-typed, which the synthesis comment records along with the symptom a future one would show. An error-driven variant (repair only on 'Unknown query criterion') was tried first and rejected: it protects string parameters automatically but loses seven methods whose criteria parameter makes them early-return on a string rather than throw a recognizable error, taking coverage from 385 back to 378. Coverage holds at 385 of 518 driven, zero engine rejections. |
||
|
|
81338cef95
|
🍃 test: Sweep Every DB Schema Method Against a Live Engine (#15392)
* 🍃 test: Sweep Every data-schemas Method Against a Live Engine Adds `misc/documentdb/sweep.documentdb.spec.ts`: drives all 510 exported data-schemas methods against a real engine and records, per method, how many driver queries it issued and whether the engine rejected any of them. Every DocumentDB incompatibility found so far — pipeline updates, `$$REMOVE`, `$facet`, the transaction-probe false negative, the Mongoose-compiled mixed projection — was invisible until someone thought to look for its class. The sweep inverts that: the engine adjudicates whatever each method emits, known class or not. Query shapes are parsed before they are matched, so an empty database adjudicates most entry paths. Mechanics: arguments are synthesized from each function's parameter names, then repaired adaptively from the validation errors they provoke (ObjectId and number casts, required schema paths, object-parameter shapes) until the method reaches the database; the driver prototype is instrumented to count calls and capture server errors — including on cursor consumption, where parse errors actually surface — and duplicate-key violations are classified as benign. A method that never issues a query is reported un-adjudicated instead of silently green: coverage is visible, not assumed. Two modes with identical mechanics so matrices diff cleanly: in-memory MongoDB (`SWEEP_BASELINE=true`, free, CI-able) and live DocumentDB (`DOCUMENTDB_URI`). `SWEEP_REPORT_PATH` writes JSON for machine diffing; `DOCUMENTDB_STRICT=true` turns rejections into failures. First full run against live DocumentDB 5.0.0: 363 of 510 methods issued at least one query, zero engine rejections, zero divergences from the MongoDB baseline. The jest config gains the main config's transform-ignore patterns so the full method bundle (which pulls ESM markdown tooling) loads. * 🍃 fix: Adjudicate Index DDL, Real ACL Cascades, and a Run-Scoped Sweep Database Review round 1, all four findings real: - The driver instrumentation now wraps `createIndex`, `createIndexes`, `dropIndex`, `dropIndexes`, and `listIndexes` (cursor-wrapped), so an engine rejecting index DDL — which `recordAuditEntry` issues before any listed operation and then swallows — is adjudicated instead of surfacing as a false `no-queries` under STRICT. - The `removeAllPermissions` no-op override is gone: `createMethods` supplies a real implementation delegating to `deleteAclEntries`, so driven deletion cascades exercise their ACL operations (not-driven drops 147 → 142). - Each race timer is cleared when its attempt settles, instead of hundreds of referenced timers living until the shared deadline and the last one holding the process open after the report. - The live run connects to a run-scoped database (`librechat_sweep_<runId>`), so its `dropDatabase` teardown can never erase fixtures out from under the sibling live suites sharing the URI, nor any retained state in the URI's own database. * 🍃 fix: Attribute Straggler Queries by Async Context and Adjudicate Transaction Commits Review round 2, all three findings real: - Per-method attribution now rides AsyncLocalStorage instead of a module global: a method that outlives its timeout keeps recording queries and engine errors under ITS OWN row rather than corrupting whichever row the sweep advanced to, and a final reclassification pass upgrades any row whose engine error arrived after its outcome was assigned. - The session prototype's `commitTransaction`/`abortTransaction`/ `withTransaction` are instrumented: transaction-lifecycle rejections surface from the session, not a collection operation, and several methods convert them into domain errors — previously a real commit rejection classified as a mere `threw` and STRICT passed. - The compatibility doc's published numbers now come from the corrected-harness baseline (368 driven / 142 un-driven) and state explicitly that the recorded live-run matrix predates the instrumentation widening, so the next live run is authoritative for the newly adjudicated surfaces. * 🍃 fix: Settle Stragglers Before STRICT and Commit a Real Transaction on a Replica-Set Baseline Review round 3, all four findings real: - STRICT is now enforced against the FINALIZED rows: `afterAll` waits (bounded) for every timed-out invocation to settle, reclassifies rows whose engine errors arrived late, and asserts zero rejections — previously a straggler's rejection could arrive after its per-test assertion had already passed against the `timeout` outcome. - The baseline runs on a single-node replica set with the live branch's exact connection options (`autoIndex`/`autoCreate` off, run-scoped `dbName`): a standalone baseline would reject driven transactions for topology reasons, and asymmetric model auto-creation would attribute DDL in one mode and not the other — both manufacturing false matrix divergences. - `resolveMCPAuthorityProof` gets a seeded case (fixture ported from the compat suite, executed in tenant context via the new `within` wrapper on overrides): generic synthesis died on target validation before a session ever started, so the session instrumentation adjudicated nothing. It now drives 22 operations through the production path, transaction commit included, on both engines. * 🍃 docs: Sync the Published Baseline Counts With the Seeded Harness * 🍃 fix: Materialize Transaction-Read Collections and Settle Stragglers Before Teardown Review round 4, three findings real (the fourth — stale doc totals — was already fixed on the pushed head): - Straggler settlement moved BEFORE engine teardown: dropping the database or closing the client would abort outstanding work with closed-client errors that are not engine verdicts, leaving a timed-out row unadjudicated while STRICT passes. - The authority fixtures (sweep AND compat) materialize PluginAuth and Token before the transaction: `loadCurrentProof` reads both inside it, and DocumentDB rejects any in-transaction statement touching a non-existent collection — the rejection surfaces as `proof_unavailable` before the commit is reached. This is the mechanism behind the compat authority test failing on every live cluster run while passing against MongoDB; with the collections materialized, the next live run adjudicates the transaction itself instead of the missing namespace. - Engine errors are recognized by `instanceof MongoServerError` instead of a `name` equality, so server-originated subclasses (MongoWriteConcernError, MongoBulkWriteError) can no longer slip through the session wrapper as domain throws. * 🍃 fix: Materialize Group and Config for the Authority Transaction and Exclude Constructor Exports Review round 5, both findings real. The proof transaction also reads Group and Config; the sweep fixture materializes both (the compat fixture already creates a Config record and now materializes Group). SessionError — an error class exported alongside the session methods — is excluded from the method map, correcting the totals to 509 methods / 369 driven / 140 un-driven; the published baseline is synced in the same commit. * 🍃 fix: Build a Fresh Method Bundle per Case and Emit a Normalized Matrix Review round 6, both findings real: - Every row now invokes a freshly constructed method bundle. Factory-local caches leaked across alphabetically ordered rows — the session methods cache `sessionIndexesPromise`, so `ensureSessionIndexes` returned that settled promise and was misreported as `no-queries`. It now issues its 3 index operations, making the matrix order-independent (370 driven / 139 un-driven). - The report separates run metadata from a canonical `rows` payload with run-specific values normalized out (run id, generated ObjectIds and UUIDs), so the two engines' matrices diff cleanly instead of differing on every row that embeds a synthesized value in its error text. * 🍃 fix: Materialize Authority Snapshot Namespaces Before Its Transaction Amazon DocumentDB rejects any statement inside a transaction that touches a collection which does not exist, and `asMCPError` converts that server rejection into `proof_unavailable` — so on a deployment where no PluginAuth or Token row has ever been written, every MCP authority proof fails with an error naming nothing about the real cause. This is the mechanism behind the live-harness authority probe failing on every real cluster run since July while passing against MongoDB. `loadAuthoritativeSnapshot` now materializes the nine namespaces its snapshot reads before opening the transaction, once per process. MongoDB tolerates the in-transaction read either way, so nothing changes there; a create failure is logged at debug and left to surface from the transaction itself. The regression test drops the credential namespaces to reproduce the state and asserts the resolve recreates them and succeeds; it fails without the fix. * 🍃 docs: Record the Authoritative Hardened Live Run The hardened harness ran against a live DocumentDB 5.0.0 cluster: 370 of 509 methods drove at least one query, zero engine rejections, and the matrix is byte-identical to the MongoDB baseline — every outcome and query count matches. This run adjudicated index DDL, transaction commits and aborts, ACL cascades, and the authority snapshot transaction, none of which the first pre-hardening run could see. The same run turned the compatibility suite's MCP authority probe green for the first time on any real cluster, confirming the namespace-materialization diagnosis for the `proof_unavailable` failures seen since July. * 🍃 fix: Retry Snapshot Namespace Creation After a Transient Failure The catch suppressed every `createCollection` error, so a transient failure settled the once-per-process memo as successful: creation was skipped for the rest of the process, the transaction kept failing on the missing collection, and every later proof returned `proof_unavailable` even after the condition cleared. The existing reset-on-reject path could never fire because the inner promise never rejected. Only `NamespaceExists` (including a concurrent creator winning the race) is suppressed now; every other failure propagates, clears the memo, and lets the next request retry. The regression test rejects the first creation with a transient error, asserts the proof fails, then asserts the following request recreates the namespace and succeeds — it fails against the swallow-all form. * 🍃 fix: Throttle Snapshot Namespace Preflight Retries With a Bounded Cooldown Clearing the memo on every rejection made a persistent failure — an authorization error, say — turn each authority request into another serial collection preflight: a DDL retry storm precisely while the database is unhealthy, since the single-flight promise only coalesces requests while an attempt is pending. A failure is now recorded with its timestamp and replayed without touching the database until a bounded cooldown elapses (30s, overridable through the existing hooks for tests). Transient failures stay retryable, just not on every request. Both sides of the boundary are covered: one test asserts a persistent failure issues no further creations inside the cooldown, and the existing retry test runs with the cooldown disabled to assert recovery still works. The throttle test fails with the cooldown gate removed. * 🍃 fix: Instrument Db-Level Namespace DDL in the Sweep The authority path's namespace preflight calls `Model.createCollection()`, which delegates to `Db.createCollection` — not to any `Collection` operation the sweep wrapped. A DocumentDB rejection of that DDL would therefore be laundered by `asMCPError` into `proof_unavailable`, leaving the row classified as a mere `threw` while `DOCUMENTDB_STRICT=true` passed: the exact false-green class this suite exists to close, reintroduced by the preflight itself. `Db.createCollection`/`dropCollection`/`renameCollection` are now instrumented, and `NamespaceExists` (48) joins the benign server codes since an idempotent preflight against an existing collection is the expected outcome, not an incompatibility. The authority row's adjudicated query count rises from 22 to 31, which is those nine preflight operations becoming visible. |
||
|
|
9684b2478f
|
🍃 fix: Restore Amazon DocumentDB Compatibility Across Trigger and Subagent Writes (#15375)
* 🍃 fix: Restore Amazon DocumentDB Compatibility Across Trigger and Subagent Writes The README advertises Amazon DocumentDB 5.0+ support, but six sites added between 2026-07-29 and 2026-08-30 reintroduced constructs the engine rejects, and the transaction probe silently reported a false negative. All seven are adjudicated against a live DocumentDB 5.0.0 cluster, not against AWS's omission-based documentation. Rejected by every engine version, now rewritten with supported operators: - `claimNextAgentTriggerDelivery` used a pipeline-form update. It reads the globally oldest candidate under the same sort, then claims that exact `_id` and lifecycle state with a compare-and-swap. Ownership is still granted only by the fenced write. An empty queue costs the one read it costs today; a successful claim costs one additional round trip. - `renewAgentTriggerDeliveryProducerLease` used a `$cond` to keep the later deadline. `$max` is the exact classic-operator form and evaluates no expression. - `claimBackgroundToolResults` and `releaseBackgroundToolResultClaims` rewrote the entire `content` array through `$map`. The filtered positional operator addresses the matching parts directly, so each write touches only those parts and still needs no read-modify-write. - The bounded subagent thread-view projections used `$$REMOVE` to omit keys. They now emit `null` and are pruned back to absent inside the pass that already materializes each record, preserving the absent-vs-present contract. - The selected-task read used `$facet`. It is now two concurrent reads, which also removes the 16 MiB single-document risk the surrounding code already avoids for the same reason. `supportsTransactions` probed a canary collection that does not exist, and DocumentDB rejects a transaction touching a non-existent collection. The probe returned `false` on an engine that fully supports transactions, so every caller silently took the non-transactional path. It now materializes the canary first, once per process. Adds `documentdb.spec.ts`, a static guard over the whole class, because no runtime test can catch it: every suite runs `mongodb-memory-server`, which is real MongoDB and accepts all of these constructs. Adds `misc/documentdb/audit.documentdb.spec.ts` to re-adjudicate against a real cluster, and records the live verdicts and connection requirements in the compatibility assessment. * 🍃 fix: Address Review — CAS Exhaustion, AST Guard, Single-Snapshot Selected Read Addresses all three review findings, and the strengthened guard immediately found a seventh incompatible site the original character-level scan missed. - Exhausting the claim's compare-and-swap budget now throws instead of returning null. Every lost round read a claimable row, so the queue is not empty; null is the engine's confirmed-empty signal and advances its idle backoff, which would leave queued work waiting. A throw lands on the engine's retryable claim-failure path, which holds the polling cadence. Covered by a new test asserting the bounded budget and that the loser owns nothing. - The static guard now walks the TypeScript AST: it catches pipelines bound to a variable first, cast with `as`, chosen by a conditional, or carried in a `bulkWrite` operation's `update` property (including shorthand). Every shape is proven against fixtures in the suite itself, so the detectors can never silently go inert. The stronger scan flagged `updateToolCallResult`, whose pipeline lived in a `stages` variable: the backgrounded tool-call settle path was still broken on DocumentDB. Rewritten with the filtered positional operator — subfield sets preserve an existing `resultClaim` without per-part branching, an explicit `$unset` keeps the old whole-object replacement's wakeup-disarm semantics, and the attachments merge becomes `$pull` plus `$push` (two atomic writes that converge under the retry the claim fencing already provides). New tests lock the disarm and supplied-claim semantics, and the live audit suite now drives this method too. - The selected-task read is a single aggregation again: each selected row yields both its bounded message fields and its source payload from the same document version, restoring the `$facet` snapshot guarantee the concurrent split lost, and returning the endpoint to three reads. * 🍃 fix: Close Settle Atomicity Windows and Spread Claim Contention Self-review pass (eight independent review angles over the branch diff). Correctness: - The attachments merge is a single fenced write again: read the array, merge with the exact semantics the old `$filter`/`$concatArrays` had, and write it back with the settle patch in one update guarded on the array being unchanged — the same compare-and-swap this file already uses for control receipts. The `$pull`-then-`$push` split could strip attachments on a failure between the writes (the throw path retires the delivery, so nothing healed it) and duplicated them under concurrent re-application. - A supplied `resultClaim` is stamped BEFORE the settle write instead of after it. The claim-last order left a terminal, claimable, unclaimed part visible to the ~1s wakeup poll, which could acquire and deliver a result a manual poll had already delivered; claim-first leaves only inert intermediate states (a claim on a part that is not yet terminal admits no consumer). The stamp's guard is `resultClaim: null`, restoring the old `$ifNull` treatment of a stored null. - `releaseBackgroundToolResultClaims` filters on `content: { $type: 'array' }`: the filtered positional operator errors on a row without the array where the old pipeline no-op'd, and that throw surfaced as retryable release failures that left claims stamped forever. - The claim loop reads a BATCH of candidates per round, so the engine's parallel claimers spread across the batch instead of convoying on one head-of-queue row; exhaustion now throws the typed `AgentTriggerClaimContentionError`. The dead-capability adoption fence also pins `availableAt`, which the write copies. - The transaction probe probes first and creates the canary only after a failure — plain MongoDB never pays the DDL or gains the collection — logs a failed create instead of swallowing it, and concurrent first probes share one in-flight promise. Structure and guard: - One `partScope(prefix)` builder yields the `$elemMatch` document filter and the arrayFilters element filter from a single predicate, deleting the aggregation `$expr` twin, and the agent-ownership rule shared by settling and claiming lives in one `agentOwnershipFilter` helper. - The static guard scans all three backend workspaces (`packages/data-schemas`, `packages/api`, `api`), parses each file once, and shares the tree between both detectors. - The audit suite asserts each probe's returned verdict, the compat suite's run recipe carries the connection parameters the audit header documents, and the `$facet` pipeline assertion covers every captured aggregation in one loop. * 🍃 fix: Keep Attachment Contention Retryable and the Selected Snapshot Authoritative Second codex round, both findings real: - Exhausting the attachment-merge fence now returns an unmatched result with a warn log instead of throwing. The settle retry loop treats unmatched as retry-then-heal-later and degrades to false; a throw escaped it onto ambiguous-failure handling that can retire the completion, losing a healthy completed tool result to mere contention. Locked by a test asserting the bounded rounds resolve to `{ matched: false }` rather than a rejection. - When the selected task's rows also appear in the normal page window, the dedup pass kept the page copy — read by an independent concurrent aggregate — and discarded the single-snapshot version, letting a page row claim a transcript that `selectedSources` does not carry. Duplicates are now replaced in place with the snapshot version (page order kept), so the selected message/source pair stays per-document coherent end to end. * 🍃 fix: Treat a Stored-Null Claim Stamp as Unclaimed in the Claim Fence Third codex round. The subfield-preserving settle write keeps a persisted `resultClaim: null` that the old whole-object rewrite dropped, and the claim path's outer `$elemMatch` fence still required the field to be absent via `$exists: false`. The in-memory claimable scan, the claim arrayFilters, and the settle stamp all already treat a null stamp as unclaimed, so a part carrying one became terminal but permanently unclaimable: every manual poll and wakeup read `ready` in memory, missed the fence, and returned `not_ready` forever. The fence now matches missing-or-null like every other layer. The regression test persists an actual null stamp — asserted present before claiming, so the test cannot pass vacuously — and claims it. |
||
|
|
23d1ad473d
|
🍃 fix: Amazon DocumentDB Compatibility for Pipeline-Form Updates (#14495)
* 🍃 fix: Amazon DocumentDB Compatibility for Pipeline-Form Updates - Rewrite acceptTerms without aggregation-pipeline update + $$NOW (null-guarded first-acceptance claim preserves the original timestamp under concurrent and repeat requests) - Rewrite decrementTagCounts clamp-at-zero decrement as ordered two-op bulkWrite (clamp before guarded $inc) - Rewrite extendFilesTTL TTL hold as projected read + per-doc guarded $set via tenantSafeBulkWrite, preserving only-widens/ceiling/cleared-stays-permanent semantics - Log background index-build failures via Model 'index' listeners (previously swallowed silently, e.g. partialFilterExpression rejection on DocumentDB <5.0) - Add misc/documentdb live-compatibility harness + assessment (AWS-cited) - Fix latent file.spec helper bug: createdAt backdating was silently stripped by mongoose immutability Closes #14488 * 🍃 fix: Harden DocumentDB-Safe Updates Against Cross-Call Races Addresses Codex review on #14495: - decrementTagCounts: normalize-null / $inc / clamp-negative op triple so interleaved decrements of the same tag converge on max(0, ...) exactly as the serialized pipeline did (clamp keys on count < 0, not count < amount) - acceptTerms: guard the repeat-acceptance fallback with a non-null timestamp (exact complement of the claim guard) and retry the claim when a config/reset-terms.js reset races between the two updates, so acceptance never resurrects a reset cycle without a fresh audit timestamp * 👷 ci: Suppress Ignored-File Warnings in Changed-File ESLint Run Changed files under config-ignored paths (packages/data-schemas/misc/**) emit "File ignored" warnings that fail --max-warnings=0. |