mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 14:08:21 +00:00
* 🍃 test: Sweep Every data-schemas Method Against a Live Engine Adds `misc/documentdb/sweep.documentdb.spec.ts`: drives all 510 exported data-schemas methods against a real engine and records, per method, how many driver queries it issued and whether the engine rejected any of them. Every DocumentDB incompatibility found so far — pipeline updates, `$$REMOVE`, `$facet`, the transaction-probe false negative, the Mongoose-compiled mixed projection — was invisible until someone thought to look for its class. The sweep inverts that: the engine adjudicates whatever each method emits, known class or not. Query shapes are parsed before they are matched, so an empty database adjudicates most entry paths. Mechanics: arguments are synthesized from each function's parameter names, then repaired adaptively from the validation errors they provoke (ObjectId and number casts, required schema paths, object-parameter shapes) until the method reaches the database; the driver prototype is instrumented to count calls and capture server errors — including on cursor consumption, where parse errors actually surface — and duplicate-key violations are classified as benign. A method that never issues a query is reported un-adjudicated instead of silently green: coverage is visible, not assumed. Two modes with identical mechanics so matrices diff cleanly: in-memory MongoDB (`SWEEP_BASELINE=true`, free, CI-able) and live DocumentDB (`DOCUMENTDB_URI`). `SWEEP_REPORT_PATH` writes JSON for machine diffing; `DOCUMENTDB_STRICT=true` turns rejections into failures. First full run against live DocumentDB 5.0.0: 363 of 510 methods issued at least one query, zero engine rejections, zero divergences from the MongoDB baseline. The jest config gains the main config's transform-ignore patterns so the full method bundle (which pulls ESM markdown tooling) loads. * 🍃 fix: Adjudicate Index DDL, Real ACL Cascades, and a Run-Scoped Sweep Database Review round 1, all four findings real: - The driver instrumentation now wraps `createIndex`, `createIndexes`, `dropIndex`, `dropIndexes`, and `listIndexes` (cursor-wrapped), so an engine rejecting index DDL — which `recordAuditEntry` issues before any listed operation and then swallows — is adjudicated instead of surfacing as a false `no-queries` under STRICT. - The `removeAllPermissions` no-op override is gone: `createMethods` supplies a real implementation delegating to `deleteAclEntries`, so driven deletion cascades exercise their ACL operations (not-driven drops 147 → 142). - Each race timer is cleared when its attempt settles, instead of hundreds of referenced timers living until the shared deadline and the last one holding the process open after the report. - The live run connects to a run-scoped database (`librechat_sweep_<runId>`), so its `dropDatabase` teardown can never erase fixtures out from under the sibling live suites sharing the URI, nor any retained state in the URI's own database. * 🍃 fix: Attribute Straggler Queries by Async Context and Adjudicate Transaction Commits Review round 2, all three findings real: - Per-method attribution now rides AsyncLocalStorage instead of a module global: a method that outlives its timeout keeps recording queries and engine errors under ITS OWN row rather than corrupting whichever row the sweep advanced to, and a final reclassification pass upgrades any row whose engine error arrived after its outcome was assigned. - The session prototype's `commitTransaction`/`abortTransaction`/ `withTransaction` are instrumented: transaction-lifecycle rejections surface from the session, not a collection operation, and several methods convert them into domain errors — previously a real commit rejection classified as a mere `threw` and STRICT passed. - The compatibility doc's published numbers now come from the corrected-harness baseline (368 driven / 142 un-driven) and state explicitly that the recorded live-run matrix predates the instrumentation widening, so the next live run is authoritative for the newly adjudicated surfaces. * 🍃 fix: Settle Stragglers Before STRICT and Commit a Real Transaction on a Replica-Set Baseline Review round 3, all four findings real: - STRICT is now enforced against the FINALIZED rows: `afterAll` waits (bounded) for every timed-out invocation to settle, reclassifies rows whose engine errors arrived late, and asserts zero rejections — previously a straggler's rejection could arrive after its per-test assertion had already passed against the `timeout` outcome. - The baseline runs on a single-node replica set with the live branch's exact connection options (`autoIndex`/`autoCreate` off, run-scoped `dbName`): a standalone baseline would reject driven transactions for topology reasons, and asymmetric model auto-creation would attribute DDL in one mode and not the other — both manufacturing false matrix divergences. - `resolveMCPAuthorityProof` gets a seeded case (fixture ported from the compat suite, executed in tenant context via the new `within` wrapper on overrides): generic synthesis died on target validation before a session ever started, so the session instrumentation adjudicated nothing. It now drives 22 operations through the production path, transaction commit included, on both engines. * 🍃 docs: Sync the Published Baseline Counts With the Seeded Harness * 🍃 fix: Materialize Transaction-Read Collections and Settle Stragglers Before Teardown Review round 4, three findings real (the fourth — stale doc totals — was already fixed on the pushed head): - Straggler settlement moved BEFORE engine teardown: dropping the database or closing the client would abort outstanding work with closed-client errors that are not engine verdicts, leaving a timed-out row unadjudicated while STRICT passes. - The authority fixtures (sweep AND compat) materialize PluginAuth and Token before the transaction: `loadCurrentProof` reads both inside it, and DocumentDB rejects any in-transaction statement touching a non-existent collection — the rejection surfaces as `proof_unavailable` before the commit is reached. This is the mechanism behind the compat authority test failing on every live cluster run while passing against MongoDB; with the collections materialized, the next live run adjudicates the transaction itself instead of the missing namespace. - Engine errors are recognized by `instanceof MongoServerError` instead of a `name` equality, so server-originated subclasses (MongoWriteConcernError, MongoBulkWriteError) can no longer slip through the session wrapper as domain throws. * 🍃 fix: Materialize Group and Config for the Authority Transaction and Exclude Constructor Exports Review round 5, both findings real. The proof transaction also reads Group and Config; the sweep fixture materializes both (the compat fixture already creates a Config record and now materializes Group). SessionError — an error class exported alongside the session methods — is excluded from the method map, correcting the totals to 509 methods / 369 driven / 140 un-driven; the published baseline is synced in the same commit. * 🍃 fix: Build a Fresh Method Bundle per Case and Emit a Normalized Matrix Review round 6, both findings real: - Every row now invokes a freshly constructed method bundle. Factory-local caches leaked across alphabetically ordered rows — the session methods cache `sessionIndexesPromise`, so `ensureSessionIndexes` returned that settled promise and was misreported as `no-queries`. It now issues its 3 index operations, making the matrix order-independent (370 driven / 139 un-driven). - The report separates run metadata from a canonical `rows` payload with run-specific values normalized out (run id, generated ObjectIds and UUIDs), so the two engines' matrices diff cleanly instead of differing on every row that embeds a synthesized value in its error text. * 🍃 fix: Materialize Authority Snapshot Namespaces Before Its Transaction Amazon DocumentDB rejects any statement inside a transaction that touches a collection which does not exist, and `asMCPError` converts that server rejection into `proof_unavailable` — so on a deployment where no PluginAuth or Token row has ever been written, every MCP authority proof fails with an error naming nothing about the real cause. This is the mechanism behind the live-harness authority probe failing on every real cluster run since July while passing against MongoDB. `loadAuthoritativeSnapshot` now materializes the nine namespaces its snapshot reads before opening the transaction, once per process. MongoDB tolerates the in-transaction read either way, so nothing changes there; a create failure is logged at debug and left to surface from the transaction itself. The regression test drops the credential namespaces to reproduce the state and asserts the resolve recreates them and succeeds; it fails without the fix. * 🍃 docs: Record the Authoritative Hardened Live Run The hardened harness ran against a live DocumentDB 5.0.0 cluster: 370 of 509 methods drove at least one query, zero engine rejections, and the matrix is byte-identical to the MongoDB baseline — every outcome and query count matches. This run adjudicated index DDL, transaction commits and aborts, ACL cascades, and the authority snapshot transaction, none of which the first pre-hardening run could see. The same run turned the compatibility suite's MCP authority probe green for the first time on any real cluster, confirming the namespace-materialization diagnosis for the `proof_unavailable` failures seen since July. * 🍃 fix: Retry Snapshot Namespace Creation After a Transient Failure The catch suppressed every `createCollection` error, so a transient failure settled the once-per-process memo as successful: creation was skipped for the rest of the process, the transaction kept failing on the missing collection, and every later proof returned `proof_unavailable` even after the condition cleared. The existing reset-on-reject path could never fire because the inner promise never rejected. Only `NamespaceExists` (including a concurrent creator winning the race) is suppressed now; every other failure propagates, clears the memo, and lets the next request retry. The regression test rejects the first creation with a transient error, asserts the proof fails, then asserts the following request recreates the namespace and succeeds — it fails against the swallow-all form. * 🍃 fix: Throttle Snapshot Namespace Preflight Retries With a Bounded Cooldown Clearing the memo on every rejection made a persistent failure — an authorization error, say — turn each authority request into another serial collection preflight: a DDL retry storm precisely while the database is unhealthy, since the single-flight promise only coalesces requests while an attempt is pending. A failure is now recorded with its timestamp and replayed without touching the database until a bounded cooldown elapses (30s, overridable through the existing hooks for tests). Transient failures stay retryable, just not on every request. Both sides of the boundary are covered: one test asserts a persistent failure issues no further creations inside the cooldown, and the existing retry test runs with the cooldown disabled to assert recovery still works. The throttle test fails with the cooldown gate removed. * 🍃 fix: Instrument Db-Level Namespace DDL in the Sweep The authority path's namespace preflight calls `Model.createCollection()`, which delegates to `Db.createCollection` — not to any `Collection` operation the sweep wrapped. A DocumentDB rejection of that DDL would therefore be laundered by `asMCPError` into `proof_unavailable`, leaving the row classified as a mere `threw` while `DOCUMENTDB_STRICT=true` passed: the exact false-green class this suite exists to close, reintroduced by the preflight itself. `Db.createCollection`/`dropCollection`/`renameCollection` are now instrumented, and `NamespaceExists` (48) joins the benign server codes since an idempotent preflight against an existing collection is the expected outcome, not an incompatibility. The authority row's adjudicated query count rises from 22 to 31, which is those nine preflight operations becoming visible. |
||
|---|---|---|
| .. | ||
| documentdb | ||
| ferretdb | ||