mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 14:08:21 +00:00
10 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
080e5f500c
|
🏘️ fix: Resolve Tenant Profiles in Their Own Partition and Keep Role Operations in the Caller's (#17417)
Some checks failed
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Vote Gate / Gate (push) Waiting to run
Codegraph E2E Votes / vote (shard 1/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 10/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 2/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 3/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 4/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 5/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 6/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 7/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 8/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 9/10) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
- GET /resolved resolves a group or user profile in the partition its document belongs to, as
writes judge it: for an admin without a tenant reading a tenant's profile, that tenant's
librechat.yaml and __base__. GET /catalog resolves such a scope there too, instead of
answering 404, and GET /effective finds a user by id the way profiles are addressed and
resolves them in their own tenant: their role, groups and documents there.
- findPartitionUser becomes findPrincipalUser, which also returns the user's tenant.
- Admin role operations address the role of the caller's tenant partition: the role lookup
(getPartitionRole), listing, creation's name check, the members a rename migrates or a deletion
reassigns, the members listed and counted, and the ACL entries (deletePrincipalAclEntries) and
grants a deletion removes. A member added or removed by id must be a user of that partition.
- getRoleByName, which every request's permission checks read, reads the role of the caller's
partition too: outside a tenant and under the system context the platform-level one, never an
older tenant role of the same name. updateRoleByName without an id and updateAccessPermissions
for a role read without its _id write that role, so the legacy permission route changes the
platform role. The grants route checks a role exists with getPartitionRole.
- runInPartition moves a read into another partition only from outside a tenant, and rejects
leaving an active one.
- deleteGrantsForPrincipal and findGrantsForPrincipal without a tenantId reach only
platform-level grants, as moveGrantsToPrincipal does.
- updateRoleByName takes { id } to change one role document; renames, their rollback,
description updates and updateAccessPermissions write the document they read.
- The exported keyed rename helpers (applyConfigChange rename, renameConfigEntryTombstones)
refuse a new name no keyed path can address, as the admin routes do.
|
||
|
|
0716094734
|
🏷️ feat: Hide and Rename Custom Endpoints by Name in Config Profiles (#17396)
* 🏷️ feat: Hide and Rename Custom Endpoints by Name in Config Profiles Config profiles merge `endpoints.custom` over librechat.yaml entry by entry on `name`, but could not take one entry away. A tombstone that named one, such as `endpoints.custom["Name"]`, passed every check, was stored with a 200 and did nothing, because the merge removes object paths only and stops at arrays; a tombstone at a position (`endpoints.custom.0.headers`) was stored and did nothing either. Renaming a profile's own endpoint through PUT or PATCH lost its stored key, since the `renames` hint covered MCP servers only. POST `/fields/tombstone`, DELETE `/fields` and the `tombstone` and `unset` ops of a change batch now take a keyed path, `endpoints.custom["<name>"]` or `endpoints.custom["<name>"].<path>`, whose name is a JSON string compared as the merge folds it. A tombstone hides the inherited endpoint, or one of its fields or record keys, for the profile's principals and removes the profile's own entry or path; hiding an endpoint also hides the model specs that select it, whether librechat.yaml or a lower profile defines it. An unset resets the profile's own entry or path with the keyed tombstones at or below it, and writing an entry clears the tombstone that hides it whole. Malformed keyed paths, keyed paths after arrays not merged by key, the entry's name, its key, preview or destination fingerprint, keyed writes and positional tombstones answer 400, and the store's keyed mutations refuse a location whose stored tombstone would name another path. A `segments` target names keys as written on every route, so a segment such as `custom["Name"]` never reaches an entry past these guards: a write by it stores a key of that spelling and clears none of the entry's tombstones. A stored tombstone reads as keyed only after an array merged by key, so an older one such as `mcpServers.a["b"]` keeps hiding that server. PUT and PATCH `renames` also take `{ path: 'endpoints.custom', previousName, name }`: the renamed entry gets the stored ciphertext and preview, and the keyed tombstones of the old name move to the new one, as they do for a batch `rename`. The carried key stays bound to its destination (400 `CONFIG_SECRET_REENTRY_REQUIRED` when the renamed entry sends it elsewhere) and keeps the destination fingerprint it was entered with; a hint that cannot apply answers 400 `CONFIG_RENAME_INVALID` with a reason. Removing the destination of a profile's own keyed entry beside its stored key needs the key again, and removing an inherited one drops the inherited key at merge and lists it in `ignored`. The merge-key registry moves to its own module so mutations can match entries by identity without an import cycle. Meta reports `keyedTombstones` and `renameHints`, `rules.renameHintPaths` and `limits.maxEntryKeyLength`, and `docs/endpoint-overrides.md` documents the contract. * 🙈 fix: Scope Hidden Endpoint Allow-Lists and Refuse Duplicate Rename Sources - A hidden custom endpoint leaves endpoints.agents.allowedProviders as it leaves modelSpecs; scopeHiddenEndpointSpecs becomes scopeHiddenEndpointReferences. - modelSpecs collapses only when no spec and no added endpoint remain, so unrelated addedEndpoints survive when the hidden endpoint owned every spec (tenant scoping shares the helper). - An addedEndpoints or allowedProviders list that names only hidden endpoints is kept whole: an empty list allows every endpoint. - Two endpoints.custom rename hints from one previousName answer CONFIG_RENAME_INVALID duplicate_source; MCP hints may still duplicate a server. * 🔀 fix: Keep Keyed Tombstones Whole Across Dev's Patch Steps, Write Report and Resolved API - PATCH resets (dev's request-ordered steps) still move a renamed entry's keyed tombstones, in validation and in the stored state; lifted tombstones come from one data-schemas helper for both paths. - The write report of POST /fields/tombstone and DELETE /fields names a keyed location by its keyed path, collection, key and entryPath, as a change ref does, instead of the bare collection. - resolveConfigLayers scopes the model specs and allow-lists of hidden endpoints as getAppConfig does, so the resolved API keeps its parity. - GET /meta reports maxEntryKeyLength beside dev's catalogModelsTtlMs. |
||
|
|
babf6f908b
|
🧾 feat: Admin Config Write Report, Structured Issues and Deployment-Only Paths (#17395)
* 🧾 feat: Admin Config Write Report, Structured Issues and Deployment-Only Paths Admin config writes dropped what the API never stores (base-only sections, interface permissions, dedicated sections) with a 200 and no word of it, stored values nothing reads (boot-time and operator-only settings, overrides of MCP servers librechat.yaml runs as local processes, tombstones inside arrays), answered most 4xx without a stable code, returned validation issues without the bound or option that failed, and refused every admin write once the terms-acceptance modal was on for an admin who had not accepted it, including the write that turns it off. PUT, PATCH /fields, POST /fields/tombstone and DELETE /fields now answer with `applied` and `skipped` (with a reason per location); a whole `interface` PATCH keeps its UI fields; base-only paths are skipped on every route. A new DEPLOYMENT_ONLY_CONFIG_PATHS list (rate limits, CloudFront, permission leases, OpenAPI, skill deletion, the tool filters, the agents and MCP boot settings, `cache`) is skipped on write, kept at librechat.yaml's value by the merge and published in GET /meta. Writes and tombstones inside a librechat.yaml stdio MCP server answer 400 PROCESS_MCP_SERVER_OVERRIDE, and a whole-section value leaves those servers out. GET of a document lists what the merge leaves out of it (`ignored`, `ignoredBasis`). Validation issues carry allowlisted `params`; each refused allowedAddresses entry is one issue with a `reason`; agent recursion limits must be whole numbers of at least 1, and the runtime ignores any other value. The config handlers' 4xx carry a `code`, the capability middleware adds AUTH_REQUIRED/FORBIDDEN, and the admin config router opts out of the terms gate. PATCH checks a section manager on every section it names, skipped entries included, as /changes does. New meta flags: writeReport, issueParams, adminTermsExempt. * 🧪 test: Expect Issue Params in the Whole-Document Override Validation Scenario Validation issues now carry `params` (here the expected and received types), so the scenario that asserts the exact issue list for a rejected PUT expects them too. * 🧪 test: Cover Merged Read Codes, Tenant-Scoped Read Reports and Deployment-Only Resolution |
||
|
|
f05f50460a
|
🔭 feat: Admin Config Read APIs (Resolved and Effective Config, Scope Catalog, Env Status) (#17383)
* 🔭 feat: Admin Config Read APIs (Resolved and Effective Config, Scope Catalog, Env Status) Admin clients could read one config document at a time, or the deployment's merged config, but nothing showed what a profile or a user resolves to: a client had to re-implement the merge (Base-first precedence, merge keys, tombstones, the interface permission filter, base-only fields, runtime stops, secret rebinding, the env allowlist) to preview it, and had no way to see why part of a profile is ignored. Endpoint, model and tool pickers came from the end-user routes, called as the signed-in admin, so they showed the admin's own scope and keys and needed a browser User-Agent. The admin config API now has four read routes. GET /resolved/:principalType/:principalId and GET /effective?userId= return librechat.yaml, the documents that merge over it in merge order, each with the parts the merge does not apply and why, and the result, in librechat.yaml shape. They come from resolveConfigLayers, which tracks the same fold the served merge runs and is pinned to it by parity tests, and from the stored documents rather than the override cache. GET /catalog lists the endpoints (with the reason one is not served), models (served, configured and defaults), MCP servers and tools a scope serves, resolved for the scope rather than the caller and never with a user's stored key; GET /env-status reports whether each env name the config uses is set, never its value. Every read is filtered to the sections the caller can read, redacted as reads redact, and scoped to the caller's tenant. The admin Langfuse routes now say why a deployment offers no connection settings. Meta reports resolvedApi, catalogApi, envStatus and limits.catalogModelsTtlMs. * 🪞 fix: Report Duplicate Custom Endpoints from the Served Definition When librechat.yaml defines a custom endpoint name twice, or both `Ollama` and `ollama`, the catalog kept the first definition for a name's env names, reason and models, while `loadCustomEndpointsConfig` serves the last servable one. An entry could pair the served config of one definition with the env names of another. `resolveServedCustomEndpoints` now holds the runtime rule (the last definition with a name, key, base URL and models wins under its served name), and `loadCustomEndpointsConfig`, `loadConfigModels` and the catalog all use it. A name with no servable definition is reported from its last one. * ⏱️ feat: Make the Admin Catalog Model-List Lifetime Configurable `GET /api/admin/config/catalog` reused a scope's model lists for a fixed two minutes. The new librechat.yaml field `endpoints.catalogModelsTtlMs` sets that lifetime (milliseconds, `0` to one hour, two minutes by default); `0` resolves the lists on every read. It is a base-only path, so a profile cannot change it and `GET /meta` reports one value, now read from the deployment's config on each request as `limits.catalogModelsTtlMs`. |
||
|
|
4d061fdd46
|
🔄 fix: Judge Tenant Profiles on Their Own Base, Clear Config Caches on Principal Deletion, and Add a Config Reload Route (#17397)
Profile writes are judged in the tenant partition the profile merges in. For a role, or a
caller inside a tenant, that is the caller's partition. For a group or user profile written by
an admin without a tenant, it is the stored document's tenant: PUT, PATCH and POST …/changes
validate on librechat.yaml as that tenant resolves it with that tenant's __base__ over it, use
the same Base for the recheck after commit and for key binding, tombstones and field deletes
judge re-entry on that tenant's librechat.yaml, and every route reports `ignored` over that
tenant's Base. Each attempt reads the document in that partition only, and a document that
moved partition while the write ran answers 409 CONFIG_VERSION_CONFLICT. The config store's
partition rules are unchanged; another tenant's Base is read in that tenant's context.
Deleting a group, role or user clears the config caches when it removes or restores the
principal's profile (`invalidatingConfigStore`), awaited before the response and broadcast with
Redis as config writes do; a call that rejects clears too, since it may have committed.
POST …/changes lets `unset` remove any part of a top-level section configSchema does not
define; the section's own `unknown_section` issue is no longer blamed on the removal, and every
other op there still answers 400 UNKNOWN_SECTION.
POST /api/admin/config/reload, for an admin without a tenant holding broad manage:configs (a
tenant admin gets 403, since the reload reaches every tenant), runs the config cache clear without
a write, loads librechat.yaml again on the answering instance, and once it loaded publishes the
clear over Redis so every other instance loads the file on its next read. It uses
ConfigCacheInvalidation.reload, which reports the clear, the load and the publish instead of only
logging a failure (writes keep `invalidate`, which leaves the load to the next read). It answers
200 `{ success: true, broadcast }`, `broadcast` false without Redis; 422 CONFIG_RELOAD_INVALID
with each schema issue's location when the loader refuses the file, keeping the last good config
and publishing nothing; 500 CONFIG_RELOAD_FAILED when a clear fails or the config cannot be built
from the file; and 503 CONFIG_RELOAD_BROADCAST_FAILED when this instance reloaded but the publish
failed, since an instance that misses the broadcast keeps its cached librechat.yaml, which never
expires. Every reload that runs is audited as `config.reloaded`, a failed one with outcome
`failure` and its code. The meta reports `features.reloadRoute`. Writes still reload the file.
The e2e content-filter helper uses the route instead of a sentinel profile.
docs/mcp-overrides.md describes the current timing: the awaited clear, the Redis broadcast, what
still waits for a TTL, that a process the clear misses keeps its old librechat.yaml, and hiding
as an access boundary at checkout.
|
||
|
|
76e3c6b134
|
🪜 fix: Base-First Config Precedence, Profile Lifecycle and Role Renames That Keep Their Profile (#17352)
* 🪜 fix: Base-First Config Precedence, Profile Lifecycle and Role Renames That Keep Their Profile Config documents merge in one total order (`compareConfigPrecedence`): Base (`role` / `__base__`) first whatever its priority, then ascending priority, then role < group < user, then document `_id`. The merge, `getApplicableConfigs` and `listAllConfigs` all use it, so a tie no longer goes to the newest document whatever its type, and Base no longer outranks a profile through its priority. A profile that writes fields of a librechat.yaml MCP server therefore undoes a Base hide of that server for its principals. A write that creates a document without a priority gives it its principal type's default (role 10, group 20, user 100, Base 0), and a write that omits the priority keeps the stored one, so a broad PUT no longer resets a profile to 10. Langfuse writes no longer pass a priority. Profile writes (PUT, PATCH `/fields`, POST `/changes`) validate on librechat.yaml with the active Base document resolved over it (`resolveRawConfig`, the fold the merge itself runs), so a role write that is valid where it lands is accepted and one the merge would strip is refused. Base still validates on librechat.yaml alone, an inactive Base is ignored, and `/changes` still judges what librechat.yaml itself defines on the raw YAML. `PATCH /api/admin/config/:principalType/:principalId/priority { priority, expectedRevision? }` sets only the priority, for broad `manage:configs`, keeping overrides, tombstones and `isActive`; it is audited as `operation: 'priority'`. A write that would create a principal's first document (PUT, PATCH `/fields`, POST `/fields/tombstone`, POST `/changes` including a dry run or a priority-only batch) answers 404 `PRINCIPAL_NOT_FOUND` unless the role (exact name), group or user (lowercase ObjectId) exists in the caller's tenant partition, or at platform level outside a tenant (`principalExists`). Each route checks inside its compare-and-swap write, against the document the write replaces; the tombstone route now plans its write like the others, so `AdminConfigDeps` no longer takes `tombstoneConfigField`. A PUT with `expectedRevision: 'absent'` creates the profile even when it carries no overrides. Renaming a role moves its config profile, system grants and ACL entries to the new name in the caller's tenant partition. The profile moves through the store contract (`movePrincipal`: a create-only copy, then removal at the copied revision). A profile already at the new name answers 409 `ROLE_CONFIG_CONFLICT`, and any failed step rolls back the users, the role document and every move (500 `ROLE_RENAME_FAILED`). Profiles, grants and ACL entries that renames made before this change left at an old name are neither moved nor removed, and a role created under that name still picks them up. The Base principal id is a reserved role name (400 `ROLE_NAME_RESERVED`), and a legacy role by that name never moves or deletes Base. The admin config meta reports `deterministicPrecedence`, `priorityRoute`, `principalCheck` and `roleRenameMovesProfile`. * 🧪 test: Give the E2E Reload Sentinel a Role of Its Own Admin config writes now answer 404 PRINCIPAL_NOT_FOUND for a principal that does not exist, so the content-filter helpers' reload sentinel (a made-up user id) could no longer write and every spec that reloads librechat.yaml failed. The sentinel is now a role the helper creates once (409 when it already exists) and deletes after its profile in cleanup; no user is a member, so its empty profile changes no one's config. * 🛟 fix: Keep a Moved Profile Whose Removal Cannot Be Confirmed, Validate on the Caller's Base and Restore Rolled-Back Descriptions - `movePrincipal` no longer treats a failed read-back of the source, after its removal failed, as the source still being there. It keeps the copy, which may be the only one left, and rethrows the removal's failure instead of deleting the copy. - Profile writes read the Base validation floor from the caller's tenant partition, so a caller without a tenant validates on the platform-level Base and never on a tenant's. - A role rename that rolls back gives a role stored without a description the schema default back, instead of a `$set` of `undefined` that kept the new description. * 🧭 fix: Scope Profile Priority Changes to the Caller's Tenant Partition `setConfigPriority`, behind PATCH `/api/admin/config/:principalType/:principalId/priority`, read and wrote through the unpartitioned config store. A platform-level admin without a tenant runs in the default non-strict context, where the tenant-isolation plugin adds no predicate, so the route could reorder a tenant's profile for the same principal, and answer with it, when no platform-level profile existed. It now runs on the partitioned operations that the role rename and the partitioned `findConfigByPrincipal` use. The read is limited to the caller's partition: that tenant's document, or without a tenant only a platform-level one (no tenant, `null` or `''`). The write is pinned to the `_id` of the document that read returned. A tenantless caller therefore gets 404 `CONFIG_NOT_FOUND` for a profile only a tenant holds. An `expectedRevision` naming such a profile gets a revision conflict and changes nothing. Tests: `config.spec.ts` (data-schemas) adds "setConfigPriority in the caller tenant partition". It covers the platform profile changing alone, a profile that only a tenant holds being neither changed nor returned, and only the active tenant's profile changing inside a tenant. The tenant helpers are now shared with the rename suite. `config.precedence.integration.spec.ts` adds the same check through the handler. With the old wiring, a tenantless PATCH of a profile only tenant-a holds answers 200 and moves tenant-a's profile to 40. With the fix it answers 404 and tenant-a keeps 10. * 🧷 fix: Roll Back Only What a Rename Changed and Validate Profile Writes on the Base They Land Over - Grant moves report what they changed (re-keyed grants, and superseded grants whose removal was confirmed); the undo re-keys only grants still at the new name and re-creates a superseded grant only while the new name still holds the grant that superseded it, so a grant revoked during a rename is never resurrected. - The Mongoose binding settles a create whose acknowledgement was lost by the `_id` it chose (found at version 1: committed, returned), as the store contract now requires; a profile move whose copy committed finishes instead of leaving an orphan copy. The memory engine and the conformance suite cover it, plus a copy that cannot be withdrawn being logged. - A rename rollback moves back only the profile it moved (by identity, edits included), puts back only members still on the new name, and reverts the description only while it is still the one the rename set. - PUT, PATCH /fields and POST /changes read Base beside the profile on every attempt and read it again once the write commits; a write that Base changed under is judged on that Base and withdrawn (only at the revision it wrote) when it fails there. - The priority route rereads a lost race's current revision in the caller's tenant partition. |
||
|
|
4e0d9145e0
|
⚛️ feat: Atomic Admin Config Changes Endpoint (set, unset, tombstone, entry upsert/remove/rename) (#17328)
* 🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes Admin config writes went straight to MongoDB with dotted-path `$set`, `$unset`, `$pull` and `$addToSet`, and nothing checked `configVersion`. That caused six problems. * ⚛️ feat: Atomic Admin Config Changes Endpoint (set, unset, tombstone, entry upsert/remove/rename) Admin clients can change a principal's config overrides only one kind of change at a time: - PATCH `/fields` for sets, at most 100 entries; - POST `/fields/tombstone` for one tombstone; - DELETE `/fields` for one reset. * 🔁 fix: Reload librechat.yaml on Admin Config Writes Base-only sections such as `filters` and `messageFilter` live only in librechat.yaml, and an admin config write was how a running server picked up an edit to them. The coherent-cache change limited admin-write invalidation to the override and MCP caches, so the YAML base was never re-read after startup: the content-filter e2e suite, which activates filters by editing the YAML and writing a reload sentinel, saw `filters: undefined` from GET /base and every activation timed out. Admin config writes clear the YAML base config again, alongside the override and MCP caches, and the clear is broadcast with the others, so every instance reloads its YAML. The reload keeps the last good config on failure and republishes the tools derived from it, so the global tool cache is left for it to overwrite rather than emptied first. * 🧭 fix: Cap Dotted Field Path Depth and Hold Revision Preconditions on No-Op Writes A dotted fieldPath was split without the 32-segment cap that segments already had, so a deep path reached the recursive path writer and failed with a 500. Both spellings now share MAX_CONFIG_PATH_DEPTH and answer 400 CONFIG_PATH_INVALID / too_deep on PATCH, tombstone and DELETE /fields. Writes reduced to nothing (protected interface permissions, base-only or dedicated sections) answered 200 without checking expectedRevision. They now compare it with the stored document and answer 409 CONFIG_VERSION_CONFLICT with currentRevision on a mismatch, as the write itself would. * 📡 fix: Retry the Config Invalidation Subscription When Redis Comes Back A SUBSCRIBE that failed at startup (Redis unreachable, the client's retries spent) was only logged, and ioredis resubscribes only to channels it has joined, so the instance missed every other instance's config invalidation until it restarted. The subscription is now sent again each time the subscriber becomes ready until one succeeds; after that the client's own resubscribe takes over, and closing stops the retries. * 🧷 fix: Tighten Config Change Batches Around Array Entries, Cancelled Changes and MCP Tombstones - A tombstone with an index segment below a field configSchema does not define (an unknown key, or what an intersection or any holds) is refused as inside_array: the merge removes inherited values from objects only, so it was stored and reported applied while the element stayed visible. - unset joins set and rename in the merged-by-key position check: unset endpoints.custom.0.headers answers 400 indexed_merge_key_write, since the position can name another endpoint by the time the batch applies. upsertEntry and removeEntry address entries by key. - A batch whose changes cancel out (set interface.customWelcome, then unset interface) writes nothing: no empty document, revision bump, audit entry or invalidation. Its changes are reported as unchanged. - A whole-section mcpServers tombstone reports each librechat.yaml stdio server it leaves in place as ignored (process_mcp_server). - removeEntry no longer attributes the whole collection to itself, and existing issues of merged-by-key entries are matched by key rather than index, so removing an entry beside a legacy-invalid one (or shifting it) is no longer refused for that entry's issue. * 🪪 fix: Keep a Requested Priority on New Principals and Report Ignored MCP Renames POST …/changes discarded a broad manager's priority for a principal without a document when no change altered it: the batch answered written: false instead of creating the priority-only document PUT creates. An explicit priority on an absent principal now counts as a change. A rename under an MCP server librechat.yaml runs as a process persisted an override the runtime merge drops, but was not reported in ignored. It now is, like set and tombstone. An unset there stays unreported: it asks for the YAML value, which is what runs. * 🔁 fix: Drop YAML Reloads and Merges Overtaken by a Config Write Admin config writes clear the YAML base config on every instance, so a write is how a running server picks up an edit to librechat.yaml. Two races could undo that: - A base reload that read the old YAML before a write's clear cached it after the clear, and the base has no TTL, so the old base was served until the next write. - A request that read the old base before the clear, and was still resolving principals, cached its merge under the new override generation for the override TTL. The config service now keeps a base-config generation that `clearAppConfigCache` bumps. A reload publishes the base, the tools and the last good config only if its generation is still current, a read after a clear never joins a reload started before it, and the newer reload starts once the stale one settles so the loader's process-wide side effects end with the newest YAML. Merges capture the generation, base included, before they read the base config. Tests: - service.spec: a clear during a reload (stale success, stale failure, a read waiting on the newer reload) and a merge built on the pre-clear base. - invalidation.redis.spec: a write on one instance makes another reload librechat.yaml. - admin-config-reload.spec (api): every admin config write route, through the real router, handlers, database and config service, reloads librechat.yaml; a request with nothing to write does not. - invalidateConfigCaches.spec no longer asserts an order the parallel clears do not have. * 🧪 test: Reload librechat.yaml After a Committed Change Batch and Not After an Unwritten One * 🧮 fix: Validate Change Batches on Their End State and Report Base-Only Fields a Tombstone Keeps A batch's validation scope was built from every intermediate write, located in the state each change produced. A change a later one undid (set `balance.startBalance`, then restore `balance`) still widened the scope, so an issue the stored document already had there rejected an unrelated change, and an entry a later change moved (upsert one entry, then remove one before it) pointed its written index at a neighbour with a legacy issue. The written paths are now located in the batch's end state and kept only where that state differs from the stored document, with entries of merged-by-key arrays compared by key. A tombstone above a base-only field (`endpoints.agents`, `endpoints.agents.pullRequests`) is stored and hides the rest of the subtree, while the runtime merge puts back the provider, host, OAuth app and shared cache slots librechat.yaml sets. The response now lists those fields in `ignored` with reason `base_only_path`, as it already does for the stdio MCP servers a whole-section tombstone leaves in place. * 🧬 fix: Encode Config Keys in Objects Built in Another Realm The key encoder treated an object as plain only when its prototype was this realm's Object.prototype, and returned any other object unchanged, keys and all. Under Jest, structuredClone (which the secret pipeline uses) returns objects from the host realm, so the items of an array the pipeline had cloned, such as Azure groups or custom endpoints, reached the engine with their dotted keys raw. Node in production has one realm, but the encoding must not depend on prototype identity: a skipped object stores its dotted keys raw without an error. A plain object is now one whose prototype is null or a root Object.prototype from any realm. The encoding spec covers an object built in a vm context. * 🧪 test: Check Dotted Record Keys Reach the Store's Whole-Document Write dev's admin config route test asserted the keys on patchConfigFields, which the handlers no longer call; it now asserts them on the draft writeConfigDocument receives. * 🚩 feat: Publish the expectedRevision, prunesEmptyParents and awaitedInvalidation Config Flags GET /api/admin/config/meta now reports the three write-coherence behaviours this branch implements, so an admin client can drop the workarounds each one replaces: - expectedRevision: responses carry `revision`; PUT, PATCH /fields, POST /fields/tombstone and PATCH /active take `expectedRevision` in the body and both DELETE routes in the query. A mismatch answers 409 CONFIG_VERSION_CONFLICT with currentRevision, a malformed value 400 INVALID_EXPECTED_REVISION; unpinned writes re-read and re-apply up to 3 times. - prunesEmptyParents: a field delete or tombstone removes the objects it empties, stopping at arrays; shells-only documents count as empty for assign-only lifecycle calls. - awaitedInvalidation: writes answer after the config caches are cleared (broadcast with Redis), GET /base bypasses the merged-config cache, and a merge read before a clear is never cached. meta.spec probes each one against the behaviour: a stale and a malformed precondition on a PATCH, the store operations' unset and tombstone on nested objects and an array element, and a held invalidation, the GET /base read options and a merge raced by a clear. * 🔀 fix: Keep Config Keys Raw Where the Engine Stores Them, So Older Servers Read Them During Upgrades Servers before the key encoding (v0.8.9-rc1 among them) read every override key literally. The Mongoose binding encoded every write, so during a rolling upgrade the first write from an upgraded server turned a stored `gpt-4.1` or Bedrock inference profile into `gpt-4%2E1` for every server not yet upgraded, and an older server's restore of a deleted group or role config dropped the `keyEncoding` marker its schema lacks, after which upgraded servers read the encoded keys literally. The binding now sends keys raw first and encodes them only when the engine refuses a raw field name (DocumentDB, Cosmos, FerretDB or MongoDB 4.4 for `$` names): the refused write changed nothing, so it runs again encoded under the same revision precondition, and that process encodes later writes holding such keys directly. MongoDB 4.4+ stores dotted names (5.0+ also `$` names inside a value), so there an older server reads exactly what it wrote. A document whose raw keys spell escape sequences while none holds a raw `.`, `$` or NUL is still written encoded, since a raw read would decode it. A raw write clears the marker. Reads decode a marked document (raw keys an older server added beside encoded ones, or wrote under a marker it left, carry no escape sequence and read as stored) and an unmarked one unless some key holds a raw `.`, `$` or NUL, which no encoded document does; that reads an encoded document whose marker an older server dropped. Encoding stays in the binding for every engine that needs it (decision #15); the residual cases are in documentdb-compat.md. * ⏱️ fix: Remove Config Cache Writes a Clear Overtook in Flight With an asynchronous APP_CONFIG store (Redis or a custom Keyv store), a clear can bump the generation and finish its delete after a reload or merge checked the generation but before its `set` landed. The older value then repopulated `_BASE_` until the next write, or a merged config for the full override TTL. The base reload, the last-good restoration and the merge cache now check the generation again once their write lands and delete what they wrote if a clear overtook it. A deleted entry only costs the next read a reload or a merge; a newer base load cannot be removed this way, since it starts only after the stale one settles. * 🔐 fix: Require the Stored Web Search Key Again When a Change Batch Moves Its URL #17323 refuses a PUT, PATCH, tombstone or field delete that changes or removes a web search URL beside a stored API key unless the key is entered again, so a stored key cannot be sent to a destination it was not entered for. POST /changes skipped that check: a batch could set, reset or tombstone `webSearch.searxngInstanceUrl` (or replace `webSearch` without the key, which keeps the stored key) and point the stored key at another host. A batch is now judged on the document it would store, like PATCH: a URL that ends up beside a stored key it did not travel with answers 400 CONFIG_SECRET_REENTRY_REQUIRED and writes nothing, dry runs included. Only keys the request sent count as entered again; a secret a set kept from the stored document does not. * 🚩 feat: Publish the changesBatch Flag and the maxChangeOps Limit in Config Meta GET /api/admin/config/meta now reports `features.changesBatch: true` and `limits.maxChangeOps` (1000, from MAX_CONFIG_CHANGE_OPS), so the admin panel can gate its one-request save on the endpoint and size its batches from the server. The meta.spec probe checks the behaviour itself: an oversized batch answers TOO_MANY_OPS, a batch with one invalid op writes nothing, and a valid set plus tombstone lands as one write. |
||
|
|
0b4dc238b3
|
🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes (#17326)
* 🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes Admin config writes went straight to MongoDB with dotted-path `$set`, `$unset`, `$pull` and `$addToSet`, and nothing checked `configVersion`. That caused six problems. * 🔁 fix: Reload librechat.yaml on Admin Config Writes Base-only sections such as `filters` and `messageFilter` live only in librechat.yaml, and an admin config write was how a running server picked up an edit to them. The coherent-cache change limited admin-write invalidation to the override and MCP caches, so the YAML base was never re-read after startup: the content-filter e2e suite, which activates filters by editing the YAML and writing a reload sentinel, saw `filters: undefined` from GET /base and every activation timed out. Admin config writes clear the YAML base config again, alongside the override and MCP caches, and the clear is broadcast with the others, so every instance reloads its YAML. The reload keeps the last good config on failure and republishes the tools derived from it, so the global tool cache is left for it to overwrite rather than emptied first. * 🧭 fix: Cap Dotted Field Path Depth and Hold Revision Preconditions on No-Op Writes A dotted fieldPath was split without the 32-segment cap that segments already had, so a deep path reached the recursive path writer and failed with a 500. Both spellings now share MAX_CONFIG_PATH_DEPTH and answer 400 CONFIG_PATH_INVALID / too_deep on PATCH, tombstone and DELETE /fields. Writes reduced to nothing (protected interface permissions, base-only or dedicated sections) answered 200 without checking expectedRevision. They now compare it with the stored document and answer 409 CONFIG_VERSION_CONFLICT with currentRevision on a mismatch, as the write itself would. * 📡 fix: Retry the Config Invalidation Subscription When Redis Comes Back A SUBSCRIBE that failed at startup (Redis unreachable, the client's retries spent) was only logged, and ioredis resubscribes only to channels it has joined, so the instance missed every other instance's config invalidation until it restarted. The subscription is now sent again each time the subscriber becomes ready until one succeeds; after that the client's own resubscribe takes over, and closing stops the retries. * 🔁 fix: Drop YAML Reloads and Merges Overtaken by a Config Write Admin config writes clear the YAML base config on every instance, so a write is how a running server picks up an edit to librechat.yaml. Two races could undo that: - A base reload that read the old YAML before a write's clear cached it after the clear, and the base has no TTL, so the old base was served until the next write. - A request that read the old base before the clear, and was still resolving principals, cached its merge under the new override generation for the override TTL. The config service now keeps a base-config generation that `clearAppConfigCache` bumps. A reload publishes the base, the tools and the last good config only if its generation is still current, a read after a clear never joins a reload started before it, and the newer reload starts once the stale one settles so the loader's process-wide side effects end with the newest YAML. Merges capture the generation, base included, before they read the base config. Tests: - service.spec: a clear during a reload (stale success, stale failure, a read waiting on the newer reload) and a merge built on the pre-clear base. - invalidation.redis.spec: a write on one instance makes another reload librechat.yaml. - admin-config-reload.spec (api): every admin config write route, through the real router, handlers, database and config service, reloads librechat.yaml; a request with nothing to write does not. - invalidateConfigCaches.spec no longer asserts an order the parallel clears do not have. * 🧬 fix: Encode Config Keys in Objects Built in Another Realm The key encoder treated an object as plain only when its prototype was this realm's Object.prototype, and returned any other object unchanged, keys and all. Under Jest, structuredClone (which the secret pipeline uses) returns objects from the host realm, so the items of an array the pipeline had cloned, such as Azure groups or custom endpoints, reached the engine with their dotted keys raw. Node in production has one realm, but the encoding must not depend on prototype identity: a skipped object stores its dotted keys raw without an error. A plain object is now one whose prototype is null or a root Object.prototype from any realm. The encoding spec covers an object built in a vm context. * 🧪 test: Check Dotted Record Keys Reach the Store's Whole-Document Write dev's admin config route test asserted the keys on patchConfigFields, which the handlers no longer call; it now asserts them on the draft writeConfigDocument receives. * 🚩 feat: Publish the expectedRevision, prunesEmptyParents and awaitedInvalidation Config Flags GET /api/admin/config/meta now reports the three write-coherence behaviours this branch implements, so an admin client can drop the workarounds each one replaces: - expectedRevision: responses carry `revision`; PUT, PATCH /fields, POST /fields/tombstone and PATCH /active take `expectedRevision` in the body and both DELETE routes in the query. A mismatch answers 409 CONFIG_VERSION_CONFLICT with currentRevision, a malformed value 400 INVALID_EXPECTED_REVISION; unpinned writes re-read and re-apply up to 3 times. - prunesEmptyParents: a field delete or tombstone removes the objects it empties, stopping at arrays; shells-only documents count as empty for assign-only lifecycle calls. - awaitedInvalidation: writes answer after the config caches are cleared (broadcast with Redis), GET /base bypasses the merged-config cache, and a merge read before a clear is never cached. meta.spec probes each one against the behaviour: a stale and a malformed precondition on a PATCH, the store operations' unset and tombstone on nested objects and an array element, and a held invalidation, the GET /base read options and a merge raced by a clear. * 🔀 fix: Keep Config Keys Raw Where the Engine Stores Them, So Older Servers Read Them During Upgrades Servers before the key encoding (v0.8.9-rc1 among them) read every override key literally. The Mongoose binding encoded every write, so during a rolling upgrade the first write from an upgraded server turned a stored `gpt-4.1` or Bedrock inference profile into `gpt-4%2E1` for every server not yet upgraded, and an older server's restore of a deleted group or role config dropped the `keyEncoding` marker its schema lacks, after which upgraded servers read the encoded keys literally. The binding now sends keys raw first and encodes them only when the engine refuses a raw field name (DocumentDB, Cosmos, FerretDB or MongoDB 4.4 for `$` names): the refused write changed nothing, so it runs again encoded under the same revision precondition, and that process encodes later writes holding such keys directly. MongoDB 4.4+ stores dotted names (5.0+ also `$` names inside a value), so there an older server reads exactly what it wrote. A document whose raw keys spell escape sequences while none holds a raw `.`, `$` or NUL is still written encoded, since a raw read would decode it. A raw write clears the marker. Reads decode a marked document (raw keys an older server added beside encoded ones, or wrote under a marker it left, carry no escape sequence and read as stored) and an unmarked one unless some key holds a raw `.`, `$` or NUL, which no encoded document does; that reads an encoded document whose marker an older server dropped. Encoding stays in the binding for every engine that needs it (decision #15); the residual cases are in documentdb-compat.md. * ⏱️ fix: Remove Config Cache Writes a Clear Overtook in Flight With an asynchronous APP_CONFIG store (Redis or a custom Keyv store), a clear can bump the generation and finish its delete after a reload or merge checked the generation but before its `set` landed. The older value then repopulated `_BASE_` until the next write, or a merged config for the full override TTL. The base reload, the last-good restoration and the merge cache now check the generation again once their write lands and delete what they wrote if a clear overtook it. A deleted entry only costs the next read a reload or a merge; a newer base load cannot be removed this way, since it starts only after the stale one settles. |
||
|
|
9ea3c5647d
|
🔑 fix: Use Admin-Stored Web Search Credentials at Runtime (#17323)
* 🔑 fix: Use Admin-Stored Web Search Credentials at Runtime Use admin-stored (encrypted or literal) web search credentials as system credentials at runtime, add the Keenable key to the secret registry, bind a stored key to the layer that set its URL, and require re-entering the key when a write changes or removes that URL (checked atomically with a version precondition). * 🧹 fix: Keep Dotted-Key Write Guarantees on Version-Checked Web Search Writes A version-checked write the caller saw absent first creates an empty document. When the database then refuses the write (a key name it cannot store, answered 422 CONFIG_KEY_NOT_STORABLE), remove that document again so a refused write stores nothing, unless another write has changed it. Secret re-entry issues now carry `segments` like every other config issue. * 🚩 feat: Publish the webSearchStoredKeys Feature Flag in the Config Contract GET /api/admin/config/meta now reports `webSearchStoredKeys: true`, so an admin client can stop warning that only ${VAR} web search references work at runtime. meta.spec probes the behaviour behind it: an encrypted stored key reaches web search auth, the Keenable key is encrypted on write, and changing a URL beside a stored key requires entering the key again. * 🧪 test: Type the Refused-Write Driver Stub in the Config Methods Spec * 🧪 test: Expect the Write Precondition Arguments in the Admin Config Route Test The handler passes a session slot and an optional version precondition to patchConfigFields since the web search re-entry check; the route test added in #17322 asserted the earlier five-argument call. * 🧹 fix: Surface a Failed Cleanup of the Empty Config Document When a version-checked write the database refused cannot remove the empty document created for it, let that failure reach the handler (500) instead of logging it and answering 422 CONFIG_KEY_NOT_STORABLE, which promises that nothing was stored. * 📝 docs: State the Web Search Stored-Key Binding Rule Exactly in the webSearchStoredKeys Flag The flag's doc said a stored key goes only to a URL the same layer set. The merge rule keeps a stored key over an inherited ${VAR} or default URL, so the doc now names the exact rule: a stored key reaches a URL stored in config only when one layer set both, a later layer that changes, blanks or removes the URL drops the key, and a write that changes or removes the URL must enter the key again. * 🧹 refactor: Read the Secret Registry Through getConfigSecretRules Only The dev merge kept this branch's getConfigSecretFields and getConfigArraySecretFields beside dev's getConfigSecretRules, which returns the same registry data. Only the secret specs used the two extra accessors. They now read getConfigSecretRules().secretFields and .arraySecretFields, so the accessors and their exported ConfigSecretField and ArraySecretField interfaces (duplicates of AdminConfigSecretFieldRule and AdminConfigArraySecretFieldRule) are gone. |
||
|
|
2591a15c10
|
🔑 feat: Keep Dotted Record Keys in Admin Config Writes and Publish the Config Contract (#17322)
* 🔑 feat: Keep Dotted Record Keys in Admin Config Writes and Publish the Config Contract Keep dotted record keys in admin config write bodies behind a path-scoped sanitizer, reject $-prefixed and prototype keys after auth, report dotted_key issues with segments, and publish GET /api/admin/config/meta (features, rules, limits, versions) so clients can gate on backend capabilities. * 🧪 test: Expect Issue Segments in the Config Override e2e Scenario Override validation issues now carry `segments`; assert the full issue shape. |