Commit graph

10 commits

Author SHA1 Message Date
Danny Avila
080e5f500c
🏘️ fix: Resolve Tenant Profiles in Their Own Partition and Keep Role Operations in the Caller's (#17417)
Some checks failed
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Vote Gate / Gate (push) Waiting to run
Codegraph E2E Votes / vote (shard 1/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 10/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 2/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 3/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 4/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 5/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 6/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 7/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 8/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 9/10) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
- GET /resolved resolves a group or user profile in the partition its document belongs to, as
  writes judge it: for an admin without a tenant reading a tenant's profile, that tenant's
  librechat.yaml and __base__. GET /catalog resolves such a scope there too, instead of
  answering 404, and GET /effective finds a user by id the way profiles are addressed and
  resolves them in their own tenant: their role, groups and documents there.
- findPartitionUser becomes findPrincipalUser, which also returns the user's tenant.
- Admin role operations address the role of the caller's tenant partition: the role lookup
  (getPartitionRole), listing, creation's name check, the members a rename migrates or a deletion
  reassigns, the members listed and counted, and the ACL entries (deletePrincipalAclEntries) and
  grants a deletion removes. A member added or removed by id must be a user of that partition.
- getRoleByName, which every request's permission checks read, reads the role of the caller's
  partition too: outside a tenant and under the system context the platform-level one, never an
  older tenant role of the same name. updateRoleByName without an id and updateAccessPermissions
  for a role read without its _id write that role, so the legacy permission route changes the
  platform role. The grants route checks a role exists with getPartitionRole.
- runInPartition moves a read into another partition only from outside a tenant, and rejects
  leaving an active one.
- deleteGrantsForPrincipal and findGrantsForPrincipal without a tenantId reach only
  platform-level grants, as moveGrantsToPrincipal does.
- updateRoleByName takes { id } to change one role document; renames, their rollback,
  description updates and updateAccessPermissions write the document they read.
- The exported keyed rename helpers (applyConfigChange rename, renameConfigEntryTombstones)
  refuse a new name no keyed path can address, as the admin routes do.
2026-10-24 18:33:12 +02:00
Danny Avila
0716094734
🏷️ feat: Hide and Rename Custom Endpoints by Name in Config Profiles (#17396)
* 🏷️ feat: Hide and Rename Custom Endpoints by Name in Config Profiles

Config profiles merge `endpoints.custom` over librechat.yaml entry by entry on `name`, but
could not take one entry away. A tombstone that named one, such as `endpoints.custom["Name"]`,
passed every check, was stored with a 200 and did nothing, because the merge removes object
paths only and stops at arrays; a tombstone at a position (`endpoints.custom.0.headers`) was
stored and did nothing either. Renaming a profile's own endpoint through PUT or PATCH lost its
stored key, since the `renames` hint covered MCP servers only.

POST `/fields/tombstone`, DELETE `/fields` and the `tombstone` and `unset` ops of a change
batch now take a keyed path, `endpoints.custom["<name>"]` or
`endpoints.custom["<name>"].<path>`, whose name is a JSON string compared as the merge folds
it. A tombstone hides the inherited endpoint, or one of its fields or record keys, for the
profile's principals and removes the profile's own entry or path; hiding an endpoint also hides
the model specs that select it, whether librechat.yaml or a lower profile defines it. An unset
resets the profile's own entry or path with the keyed tombstones at or below it, and writing an
entry clears the tombstone that hides it whole. Malformed keyed paths, keyed paths after arrays
not merged by key, the entry's name, its key, preview or destination fingerprint, keyed writes
and positional tombstones answer 400, and the store's keyed mutations refuse a location whose
stored tombstone would name another path. A `segments` target names keys as written on every
route, so a segment such as `custom["Name"]` never reaches an entry past these guards: a write
by it stores a key of that spelling and clears none of the entry's tombstones. A stored
tombstone reads as keyed only after an array merged by key, so an older one such as
`mcpServers.a["b"]` keeps hiding that server.

PUT and PATCH `renames` also take `{ path: 'endpoints.custom', previousName, name }`: the
renamed entry gets the stored ciphertext and preview, and the keyed tombstones of the old name
move to the new one, as they do for a batch `rename`. The carried key stays bound to its
destination (400 `CONFIG_SECRET_REENTRY_REQUIRED` when the renamed entry sends it elsewhere)
and keeps the destination fingerprint it was entered with; a hint that cannot apply answers 400
`CONFIG_RENAME_INVALID` with a reason. Removing the destination of a profile's own keyed entry
beside its stored key needs the key again, and removing an inherited one drops the inherited
key at merge and lists it in `ignored`.

The merge-key registry moves to its own module so mutations can match entries by identity
without an import cycle. Meta reports `keyedTombstones` and `renameHints`,
`rules.renameHintPaths` and `limits.maxEntryKeyLength`, and `docs/endpoint-overrides.md`
documents the contract.

* 🙈 fix: Scope Hidden Endpoint Allow-Lists and Refuse Duplicate Rename Sources

- A hidden custom endpoint leaves endpoints.agents.allowedProviders as it
  leaves modelSpecs; scopeHiddenEndpointSpecs becomes
  scopeHiddenEndpointReferences.
- modelSpecs collapses only when no spec and no added endpoint remain, so
  unrelated addedEndpoints survive when the hidden endpoint owned every spec
  (tenant scoping shares the helper).
- An addedEndpoints or allowedProviders list that names only hidden endpoints
  is kept whole: an empty list allows every endpoint.
- Two endpoints.custom rename hints from one previousName answer
  CONFIG_RENAME_INVALID duplicate_source; MCP hints may still duplicate a
  server.

* 🔀 fix: Keep Keyed Tombstones Whole Across Dev's Patch Steps, Write Report and Resolved API

- PATCH resets (dev's request-ordered steps) still move a renamed entry's
  keyed tombstones, in validation and in the stored state; lifted
  tombstones come from one data-schemas helper for both paths.
- The write report of POST /fields/tombstone and DELETE /fields names a
  keyed location by its keyed path, collection, key and entryPath, as a
  change ref does, instead of the bare collection.
- resolveConfigLayers scopes the model specs and allow-lists of hidden
  endpoints as getAppConfig does, so the resolved API keeps its parity.
- GET /meta reports maxEntryKeyLength beside dev's catalogModelsTtlMs.
2026-10-24 10:44:57 +02:00
Danny Avila
babf6f908b
🧾 feat: Admin Config Write Report, Structured Issues and Deployment-Only Paths (#17395)
* 🧾 feat: Admin Config Write Report, Structured Issues and Deployment-Only Paths

Admin config writes dropped what the API never stores (base-only sections, interface
permissions, dedicated sections) with a 200 and no word of it, stored values nothing reads
(boot-time and operator-only settings, overrides of MCP servers librechat.yaml runs as local
processes, tombstones inside arrays), answered most 4xx without a stable code, returned
validation issues without the bound or option that failed, and refused every admin write
once the terms-acceptance modal was on for an admin who had not accepted it, including the
write that turns it off.

PUT, PATCH /fields, POST /fields/tombstone and DELETE /fields now answer with `applied` and
`skipped` (with a reason per location); a whole `interface` PATCH keeps its UI fields;
base-only paths are skipped on every route. A new DEPLOYMENT_ONLY_CONFIG_PATHS list (rate
limits, CloudFront, permission leases, OpenAPI, skill deletion, the tool filters, the agents
and MCP boot settings, `cache`) is skipped on write, kept at librechat.yaml's value by the merge
and published in GET /meta. Writes and tombstones inside a librechat.yaml stdio MCP server
answer 400 PROCESS_MCP_SERVER_OVERRIDE, and a whole-section value leaves those servers out.
GET of a document lists what the merge leaves out of it (`ignored`, `ignoredBasis`).
Validation issues carry allowlisted `params`; each refused allowedAddresses entry is one issue
with a `reason`; agent recursion limits must be whole numbers of at least 1, and the runtime
ignores any other value. The config handlers' 4xx carry a `code`, the capability middleware adds
AUTH_REQUIRED/FORBIDDEN, and the admin config router opts out of the terms gate. PATCH checks a
section manager on every section it names, skipped entries included, as /changes does. New meta
flags: writeReport, issueParams, adminTermsExempt.

* 🧪 test: Expect Issue Params in the Whole-Document Override Validation Scenario

Validation issues now carry `params` (here the expected and received types), so the scenario that
asserts the exact issue list for a rejected PUT expects them too.

* 🧪 test: Cover Merged Read Codes, Tenant-Scoped Read Reports and Deployment-Only Resolution
2026-10-24 10:01:13 +02:00
Danny Avila
f05f50460a
🔭 feat: Admin Config Read APIs (Resolved and Effective Config, Scope Catalog, Env Status) (#17383)
* 🔭 feat: Admin Config Read APIs (Resolved and Effective Config, Scope Catalog, Env Status)

Admin clients could read one config document at a time, or the deployment's merged config,
but nothing showed what a profile or a user resolves to: a client had to re-implement the
merge (Base-first precedence, merge keys, tombstones, the interface permission filter,
base-only fields, runtime stops, secret rebinding, the env allowlist) to preview it, and had
no way to see why part of a profile is ignored. Endpoint, model and tool pickers came from the
end-user routes, called as the signed-in admin, so they showed the admin's own scope and keys
and needed a browser User-Agent.

The admin config API now has four read routes. GET /resolved/:principalType/:principalId and
GET /effective?userId= return librechat.yaml, the documents that merge over it in merge order,
each with the parts the merge does not apply and why, and the result, in librechat.yaml shape.
They come from resolveConfigLayers, which tracks the same fold the served merge runs and is
pinned to it by parity tests, and from the stored documents rather than the override cache.
GET /catalog lists the endpoints (with the reason one is not served), models (served,
configured and defaults), MCP servers and tools a scope serves, resolved for the scope rather
than the caller and never with a user's stored key; GET /env-status reports whether each env
name the config uses is set, never its value. Every read is filtered to the sections the caller
can read, redacted as reads redact, and scoped to the caller's tenant. The admin Langfuse routes
now say why a deployment offers no connection settings. Meta reports resolvedApi, catalogApi,
envStatus and limits.catalogModelsTtlMs.

* 🪞 fix: Report Duplicate Custom Endpoints from the Served Definition

When librechat.yaml defines a custom endpoint name twice, or both `Ollama` and `ollama`, the
catalog kept the first definition for a name's env names, reason and models, while
`loadCustomEndpointsConfig` serves the last servable one. An entry could pair the served config
of one definition with the env names of another.

`resolveServedCustomEndpoints` now holds the runtime rule (the last definition with a name, key,
base URL and models wins under its served name), and `loadCustomEndpointsConfig`,
`loadConfigModels` and the catalog all use it. A name with no servable definition is reported
from its last one.

* ⏱️ feat: Make the Admin Catalog Model-List Lifetime Configurable

`GET /api/admin/config/catalog` reused a scope's model lists for a fixed two minutes. The new
librechat.yaml field `endpoints.catalogModelsTtlMs` sets that lifetime (milliseconds, `0` to one
hour, two minutes by default); `0` resolves the lists on every read. It is a base-only path, so a
profile cannot change it and `GET /meta` reports one value, now read from the deployment's config
on each request as `limits.catalogModelsTtlMs`.
2026-10-24 09:23:22 +02:00
Danny Avila
4d061fdd46
🔄 fix: Judge Tenant Profiles on Their Own Base, Clear Config Caches on Principal Deletion, and Add a Config Reload Route (#17397)
Profile writes are judged in the tenant partition the profile merges in. For a role, or a
caller inside a tenant, that is the caller's partition. For a group or user profile written by
an admin without a tenant, it is the stored document's tenant: PUT, PATCH and POST …/changes
validate on librechat.yaml as that tenant resolves it with that tenant's __base__ over it, use
the same Base for the recheck after commit and for key binding, tombstones and field deletes
judge re-entry on that tenant's librechat.yaml, and every route reports `ignored` over that
tenant's Base. Each attempt reads the document in that partition only, and a document that
moved partition while the write ran answers 409 CONFIG_VERSION_CONFLICT. The config store's
partition rules are unchanged; another tenant's Base is read in that tenant's context.

Deleting a group, role or user clears the config caches when it removes or restores the
principal's profile (`invalidatingConfigStore`), awaited before the response and broadcast with
Redis as config writes do; a call that rejects clears too, since it may have committed.

POST …/changes lets `unset` remove any part of a top-level section configSchema does not
define; the section's own `unknown_section` issue is no longer blamed on the removal, and every
other op there still answers 400 UNKNOWN_SECTION.

POST /api/admin/config/reload, for an admin without a tenant holding broad manage:configs (a
tenant admin gets 403, since the reload reaches every tenant), runs the config cache clear without
a write, loads librechat.yaml again on the answering instance, and once it loaded publishes the
clear over Redis so every other instance loads the file on its next read. It uses
ConfigCacheInvalidation.reload, which reports the clear, the load and the publish instead of only
logging a failure (writes keep `invalidate`, which leaves the load to the next read). It answers
200 `{ success: true, broadcast }`, `broadcast` false without Redis; 422 CONFIG_RELOAD_INVALID
with each schema issue's location when the loader refuses the file, keeping the last good config
and publishing nothing; 500 CONFIG_RELOAD_FAILED when a clear fails or the config cannot be built
from the file; and 503 CONFIG_RELOAD_BROADCAST_FAILED when this instance reloaded but the publish
failed, since an instance that misses the broadcast keeps its cached librechat.yaml, which never
expires. Every reload that runs is audited as `config.reloaded`, a failed one with outcome
`failure` and its code. The meta reports `features.reloadRoute`. Writes still reload the file.
The e2e content-filter helper uses the route instead of a sentinel profile.

docs/mcp-overrides.md describes the current timing: the awaited clear, the Redis broadcast, what
still waits for a TTL, that a process the clear misses keeps its old librechat.yaml, and hiding
as an access boundary at checkout.
2026-10-24 08:45:54 +02:00
Danny Avila
76e3c6b134
🪜 fix: Base-First Config Precedence, Profile Lifecycle and Role Renames That Keep Their Profile (#17352)
* 🪜 fix: Base-First Config Precedence, Profile Lifecycle and Role Renames That Keep Their Profile

Config documents merge in one total order (`compareConfigPrecedence`): Base (`role` /
`__base__`) first whatever its priority, then ascending priority, then role < group < user,
then document `_id`. The merge, `getApplicableConfigs` and `listAllConfigs` all use it, so a
tie no longer goes to the newest document whatever its type, and Base no longer outranks a
profile through its priority. A profile that writes fields of a librechat.yaml MCP server
therefore undoes a Base hide of that server for its principals. A write that creates a document
without a priority gives it its principal type's default (role 10, group 20, user 100, Base 0),
and a write that omits the priority keeps the stored one, so a broad PUT no longer resets a
profile to 10. Langfuse writes no longer pass a priority.

Profile writes (PUT, PATCH `/fields`, POST `/changes`) validate on librechat.yaml with the
active Base document resolved over it (`resolveRawConfig`, the fold the merge itself runs), so
a role write that is valid where it lands is accepted and one the merge would strip is refused.
Base still validates on librechat.yaml alone, an inactive Base is ignored, and `/changes` still
judges what librechat.yaml itself defines on the raw YAML.

`PATCH /api/admin/config/:principalType/:principalId/priority { priority, expectedRevision? }`
sets only the priority, for broad `manage:configs`, keeping overrides, tombstones and
`isActive`; it is audited as `operation: 'priority'`. A write that would create a principal's
first document (PUT, PATCH `/fields`, POST `/fields/tombstone`, POST `/changes` including a dry
run or a priority-only batch) answers 404 `PRINCIPAL_NOT_FOUND` unless the role (exact name),
group or user (lowercase ObjectId) exists in the caller's tenant partition, or at platform
level outside a tenant (`principalExists`). Each route checks inside its compare-and-swap
write, against the document the write replaces; the tombstone route now plans its write like
the others, so `AdminConfigDeps` no longer takes `tombstoneConfigField`. A PUT with
`expectedRevision: 'absent'` creates the profile even when it carries no overrides.

Renaming a role moves its config profile, system grants and ACL entries to the new name in the
caller's tenant partition. The profile moves through the store contract (`movePrincipal`: a
create-only copy, then removal at the copied revision). A profile already at the new name
answers 409 `ROLE_CONFIG_CONFLICT`, and any failed step rolls back the users, the role document
and every move (500 `ROLE_RENAME_FAILED`). Profiles, grants and ACL entries that renames made
before this change left at an old name are neither moved nor removed, and a role created under
that name still picks them up. The Base principal id is a reserved role name (400
`ROLE_NAME_RESERVED`), and a legacy role by that name never moves or deletes Base. The admin
config meta reports `deterministicPrecedence`, `priorityRoute`, `principalCheck` and
`roleRenameMovesProfile`.

* 🧪 test: Give the E2E Reload Sentinel a Role of Its Own

Admin config writes now answer 404 PRINCIPAL_NOT_FOUND for a principal that does not exist, so
the content-filter helpers' reload sentinel (a made-up user id) could no longer write and every
spec that reloads librechat.yaml failed. The sentinel is now a role the helper creates once (409
when it already exists) and deletes after its profile in cleanup; no user is a member, so its
empty profile changes no one's config.

* 🛟 fix: Keep a Moved Profile Whose Removal Cannot Be Confirmed, Validate on the Caller's Base and Restore Rolled-Back Descriptions

- `movePrincipal` no longer treats a failed read-back of the source, after its removal
  failed, as the source still being there. It keeps the copy, which may be the only one
  left, and rethrows the removal's failure instead of deleting the copy.
- Profile writes read the Base validation floor from the caller's tenant partition, so a
  caller without a tenant validates on the platform-level Base and never on a tenant's.
- A role rename that rolls back gives a role stored without a description the schema
  default back, instead of a `$set` of `undefined` that kept the new description.

* 🧭 fix: Scope Profile Priority Changes to the Caller's Tenant Partition

`setConfigPriority`, behind PATCH `/api/admin/config/:principalType/:principalId/priority`,
read and wrote through the unpartitioned config store. A platform-level admin without a
tenant runs in the default non-strict context, where the tenant-isolation plugin adds no
predicate, so the route could reorder a tenant's profile for the same principal, and answer
with it, when no platform-level profile existed.

It now runs on the partitioned operations that the role rename and the partitioned
`findConfigByPrincipal` use. The read is limited to the caller's partition: that tenant's
document, or without a tenant only a platform-level one (no tenant, `null` or `''`). The
write is pinned to the `_id` of the document that read returned. A tenantless caller
therefore gets 404 `CONFIG_NOT_FOUND` for a profile only a tenant holds. An
`expectedRevision` naming such a profile gets a revision conflict and changes nothing.

Tests: `config.spec.ts` (data-schemas) adds "setConfigPriority in the caller tenant
partition". It covers the platform profile changing alone, a profile that only a tenant holds
being neither changed nor returned, and only the active tenant's profile changing inside a
tenant. The tenant helpers are now shared with the rename suite.
`config.precedence.integration.spec.ts` adds the same check through the handler. With the old
wiring, a tenantless PATCH of a profile only tenant-a holds answers 200 and moves tenant-a's
profile to 40. With the fix it answers 404 and tenant-a keeps 10.

* 🧷 fix: Roll Back Only What a Rename Changed and Validate Profile Writes on the Base They Land Over

- Grant moves report what they changed (re-keyed grants, and superseded grants whose removal
  was confirmed); the undo re-keys only grants still at the new name and re-creates a
  superseded grant only while the new name still holds the grant that superseded it, so a
  grant revoked during a rename is never resurrected.
- The Mongoose binding settles a create whose acknowledgement was lost by the `_id` it chose
  (found at version 1: committed, returned), as the store contract now requires; a profile
  move whose copy committed finishes instead of leaving an orphan copy. The memory engine and
  the conformance suite cover it, plus a copy that cannot be withdrawn being logged.
- A rename rollback moves back only the profile it moved (by identity, edits included), puts
  back only members still on the new name, and reverts the description only while it is still
  the one the rename set.
- PUT, PATCH /fields and POST /changes read Base beside the profile on every attempt and read
  it again once the write commits; a write that Base changed under is judged on that Base and
  withdrawn (only at the revision it wrote) when it fails there.
- The priority route rereads a lost race's current revision in the caller's tenant partition.
2026-10-22 23:32:27 +02:00
Danny Avila
4e0d9145e0
⚛️ feat: Atomic Admin Config Changes Endpoint (set, unset, tombstone, entry upsert/remove/rename) (#17328)
* 🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes

Admin config writes went straight to MongoDB with dotted-path `$set`, `$unset`, `$pull` and `$addToSet`, and nothing checked `configVersion`. That caused six problems.

* ⚛️ feat: Atomic Admin Config Changes Endpoint (set, unset, tombstone, entry upsert/remove/rename)

Admin clients can change a principal's config overrides only one kind of change at a time:
- PATCH `/fields` for sets, at most 100 entries;
- POST `/fields/tombstone` for one tombstone;
- DELETE `/fields` for one reset.

* 🔁 fix: Reload librechat.yaml on Admin Config Writes

Base-only sections such as `filters` and `messageFilter` live only in librechat.yaml, and an admin
config write was how a running server picked up an edit to them. The coherent-cache change limited
admin-write invalidation to the override and MCP caches, so the YAML base was never re-read after
startup: the content-filter e2e suite, which activates filters by editing the YAML and writing a
reload sentinel, saw `filters: undefined` from GET /base and every activation timed out.

Admin config writes clear the YAML base config again, alongside the override and MCP caches, and
the clear is broadcast with the others, so every instance reloads its YAML. The reload keeps the
last good config on failure and republishes the tools derived from it, so the global tool cache is
left for it to overwrite rather than emptied first.

* 🧭 fix: Cap Dotted Field Path Depth and Hold Revision Preconditions on No-Op Writes

A dotted fieldPath was split without the 32-segment cap that segments already had, so a deep path reached the recursive path writer and failed with a 500. Both spellings now share MAX_CONFIG_PATH_DEPTH and answer 400 CONFIG_PATH_INVALID / too_deep on PATCH, tombstone and DELETE /fields.

Writes reduced to nothing (protected interface permissions, base-only or dedicated sections) answered 200 without checking expectedRevision. They now compare it with the stored document and answer 409 CONFIG_VERSION_CONFLICT with currentRevision on a mismatch, as the write itself would.

* 📡 fix: Retry the Config Invalidation Subscription When Redis Comes Back

A SUBSCRIBE that failed at startup (Redis unreachable, the client's retries spent) was only logged, and ioredis resubscribes only to channels it has joined, so the instance missed every other instance's config invalidation until it restarted. The subscription is now sent again each time the subscriber becomes ready until one succeeds; after that the client's own resubscribe takes over, and closing stops the retries.

* 🧷 fix: Tighten Config Change Batches Around Array Entries, Cancelled Changes and MCP Tombstones

- A tombstone with an index segment below a field configSchema does not define (an unknown key, or what an intersection or any holds) is refused as inside_array: the merge removes inherited values from objects only, so it was stored and reported applied while the element stayed visible.
- unset joins set and rename in the merged-by-key position check: unset endpoints.custom.0.headers answers 400 indexed_merge_key_write, since the position can name another endpoint by the time the batch applies. upsertEntry and removeEntry address entries by key.
- A batch whose changes cancel out (set interface.customWelcome, then unset interface) writes nothing: no empty document, revision bump, audit entry or invalidation. Its changes are reported as unchanged.
- A whole-section mcpServers tombstone reports each librechat.yaml stdio server it leaves in place as ignored (process_mcp_server).
- removeEntry no longer attributes the whole collection to itself, and existing issues of merged-by-key entries are matched by key rather than index, so removing an entry beside a legacy-invalid one (or shifting it) is no longer refused for that entry's issue.

* 🪪 fix: Keep a Requested Priority on New Principals and Report Ignored MCP Renames

POST …/changes discarded a broad manager's priority for a principal without a document when no change altered it: the batch answered written: false instead of creating the priority-only document PUT creates. An explicit priority on an absent principal now counts as a change.

A rename under an MCP server librechat.yaml runs as a process persisted an override the runtime merge drops, but was not reported in ignored. It now is, like set and tombstone. An unset there stays unreported: it asks for the YAML value, which is what runs.

* 🔁 fix: Drop YAML Reloads and Merges Overtaken by a Config Write

Admin config writes clear the YAML base config on every instance, so a write is how a running
server picks up an edit to librechat.yaml. Two races could undo that:

- A base reload that read the old YAML before a write's clear cached it after the clear, and the
  base has no TTL, so the old base was served until the next write.
- A request that read the old base before the clear, and was still resolving principals, cached
  its merge under the new override generation for the override TTL.

The config service now keeps a base-config generation that `clearAppConfigCache` bumps. A reload
publishes the base, the tools and the last good config only if its generation is still current,
a read after a clear never joins a reload started before it, and the newer reload starts once the
stale one settles so the loader's process-wide side effects end with the newest YAML. Merges
capture the generation, base included, before they read the base config.

Tests:
- service.spec: a clear during a reload (stale success, stale failure, a read waiting on the newer
  reload) and a merge built on the pre-clear base.
- invalidation.redis.spec: a write on one instance makes another reload librechat.yaml.
- admin-config-reload.spec (api): every admin config write route, through the real router,
  handlers, database and config service, reloads librechat.yaml; a request with nothing to write
  does not.
- invalidateConfigCaches.spec no longer asserts an order the parallel clears do not have.

* 🧪 test: Reload librechat.yaml After a Committed Change Batch and Not After an Unwritten One

* 🧮 fix: Validate Change Batches on Their End State and Report Base-Only Fields a Tombstone Keeps

A batch's validation scope was built from every intermediate write, located in the state each
change produced. A change a later one undid (set `balance.startBalance`, then restore `balance`)
still widened the scope, so an issue the stored document already had there rejected an
unrelated change, and an entry a later change moved (upsert one entry, then remove one before
it) pointed its written index at a neighbour with a legacy issue. The written paths are now
located in the batch's end state and kept only where that state differs from the stored
document, with entries of merged-by-key arrays compared by key.

A tombstone above a base-only field (`endpoints.agents`, `endpoints.agents.pullRequests`) is
stored and hides the rest of the subtree, while the runtime merge puts back the provider, host,
OAuth app and shared cache slots librechat.yaml sets. The response now lists those fields in
`ignored` with reason `base_only_path`, as it already does for the stdio MCP servers a
whole-section tombstone leaves in place.

* 🧬 fix: Encode Config Keys in Objects Built in Another Realm

The key encoder treated an object as plain only when its prototype was this realm's
Object.prototype, and returned any other object unchanged, keys and all. Under Jest,
structuredClone (which the secret pipeline uses) returns objects from the host realm, so the
items of an array the pipeline had cloned, such as Azure groups or custom endpoints, reached
the engine with their dotted keys raw. Node in production has one realm, but the encoding must
not depend on prototype identity: a skipped object stores its dotted keys raw without an error.

A plain object is now one whose prototype is null or a root Object.prototype from any realm.
The encoding spec covers an object built in a vm context.

* 🧪 test: Check Dotted Record Keys Reach the Store's Whole-Document Write

dev's admin config route test asserted the keys on patchConfigFields, which the handlers no
longer call; it now asserts them on the draft writeConfigDocument receives.

* 🚩 feat: Publish the expectedRevision, prunesEmptyParents and awaitedInvalidation Config Flags

GET /api/admin/config/meta now reports the three write-coherence behaviours this branch
implements, so an admin client can drop the workarounds each one replaces:

- expectedRevision: responses carry `revision`; PUT, PATCH /fields, POST /fields/tombstone and
  PATCH /active take `expectedRevision` in the body and both DELETE routes in the query. A
  mismatch answers 409 CONFIG_VERSION_CONFLICT with currentRevision, a malformed value 400
  INVALID_EXPECTED_REVISION; unpinned writes re-read and re-apply up to 3 times.
- prunesEmptyParents: a field delete or tombstone removes the objects it empties, stopping at
  arrays; shells-only documents count as empty for assign-only lifecycle calls.
- awaitedInvalidation: writes answer after the config caches are cleared (broadcast with Redis),
  GET /base bypasses the merged-config cache, and a merge read before a clear is never cached.

meta.spec probes each one against the behaviour: a stale and a malformed precondition on a
PATCH, the store operations' unset and tombstone on nested objects and an array element, and a
held invalidation, the GET /base read options and a merge raced by a clear.

* 🔀 fix: Keep Config Keys Raw Where the Engine Stores Them, So Older Servers Read Them During Upgrades

Servers before the key encoding (v0.8.9-rc1 among them) read every override key literally.
The Mongoose binding encoded every write, so during a rolling upgrade the first write from an
upgraded server turned a stored `gpt-4.1` or Bedrock inference profile into `gpt-4%2E1` for
every server not yet upgraded, and an older server's restore of a deleted group or role
config dropped the `keyEncoding` marker its schema lacks, after which upgraded servers read
the encoded keys literally.

The binding now sends keys raw first and encodes them only when the engine refuses a raw
field name (DocumentDB, Cosmos, FerretDB or MongoDB 4.4 for `$` names): the refused write
changed nothing, so it runs again encoded under the same revision precondition, and that
process encodes later writes holding such keys directly. MongoDB 4.4+ stores dotted names
(5.0+ also `$` names inside a value), so there an older server reads exactly what it wrote. A
document whose raw keys spell escape sequences while none holds a raw `.`, `$` or NUL is
still written encoded, since a raw read would decode it. A raw write clears the marker.

Reads decode a marked document (raw keys an older server added beside encoded ones, or wrote
under a marker it left, carry no escape sequence and read as stored) and an unmarked one
unless some key holds a raw `.`, `$` or NUL, which no encoded document does; that reads an
encoded document whose marker an older server dropped. Encoding stays in the binding for
every engine that needs it (decision #15); the residual cases are in documentdb-compat.md.

* ⏱️ fix: Remove Config Cache Writes a Clear Overtook in Flight

With an asynchronous APP_CONFIG store (Redis or a custom Keyv store), a clear can bump the
generation and finish its delete after a reload or merge checked the generation but before its
`set` landed. The older value then repopulated `_BASE_` until the next write, or a merged
config for the full override TTL.

The base reload, the last-good restoration and the merge cache now check the generation again
once their write lands and delete what they wrote if a clear overtook it. A deleted entry only
costs the next read a reload or a merge; a newer base load cannot be removed this way, since it
starts only after the stale one settles.

* 🔐 fix: Require the Stored Web Search Key Again When a Change Batch Moves Its URL

#17323 refuses a PUT, PATCH, tombstone or field delete that changes or removes a web search
URL beside a stored API key unless the key is entered again, so a stored key cannot be sent
to a destination it was not entered for. POST /changes skipped that check: a batch could set,
reset or tombstone `webSearch.searxngInstanceUrl` (or replace `webSearch` without the key,
which keeps the stored key) and point the stored key at another host.

A batch is now judged on the document it would store, like PATCH: a URL that ends up beside
a stored key it did not travel with answers 400 CONFIG_SECRET_REENTRY_REQUIRED and writes
nothing, dry runs included. Only keys the request sent count as entered again; a secret a set
kept from the stored document does not.

* 🚩 feat: Publish the changesBatch Flag and the maxChangeOps Limit in Config Meta

GET /api/admin/config/meta now reports `features.changesBatch: true` and
`limits.maxChangeOps` (1000, from MAX_CONFIG_CHANGE_OPS), so the admin panel can gate its
one-request save on the endpoint and size its batches from the server. The meta.spec probe
checks the behaviour itself: an oversized batch answers TOO_MANY_OPS, a batch with one invalid
op writes nothing, and a valid set plus tombstone lands as one write.
2026-10-22 13:38:06 +02:00
Danny Avila
0b4dc238b3
🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes (#17326)
* 🗄️ refactor: Portable Admin Config Store with Revisions, Key Encoding and Coherent Writes

Admin config writes went straight to MongoDB with dotted-path `$set`, `$unset`, `$pull` and `$addToSet`, and nothing checked `configVersion`. That caused six problems.

* 🔁 fix: Reload librechat.yaml on Admin Config Writes

Base-only sections such as `filters` and `messageFilter` live only in librechat.yaml, and an admin
config write was how a running server picked up an edit to them. The coherent-cache change limited
admin-write invalidation to the override and MCP caches, so the YAML base was never re-read after
startup: the content-filter e2e suite, which activates filters by editing the YAML and writing a
reload sentinel, saw `filters: undefined` from GET /base and every activation timed out.

Admin config writes clear the YAML base config again, alongside the override and MCP caches, and
the clear is broadcast with the others, so every instance reloads its YAML. The reload keeps the
last good config on failure and republishes the tools derived from it, so the global tool cache is
left for it to overwrite rather than emptied first.

* 🧭 fix: Cap Dotted Field Path Depth and Hold Revision Preconditions on No-Op Writes

A dotted fieldPath was split without the 32-segment cap that segments already had, so a deep path reached the recursive path writer and failed with a 500. Both spellings now share MAX_CONFIG_PATH_DEPTH and answer 400 CONFIG_PATH_INVALID / too_deep on PATCH, tombstone and DELETE /fields.

Writes reduced to nothing (protected interface permissions, base-only or dedicated sections) answered 200 without checking expectedRevision. They now compare it with the stored document and answer 409 CONFIG_VERSION_CONFLICT with currentRevision on a mismatch, as the write itself would.

* 📡 fix: Retry the Config Invalidation Subscription When Redis Comes Back

A SUBSCRIBE that failed at startup (Redis unreachable, the client's retries spent) was only logged, and ioredis resubscribes only to channels it has joined, so the instance missed every other instance's config invalidation until it restarted. The subscription is now sent again each time the subscriber becomes ready until one succeeds; after that the client's own resubscribe takes over, and closing stops the retries.

* 🔁 fix: Drop YAML Reloads and Merges Overtaken by a Config Write

Admin config writes clear the YAML base config on every instance, so a write is how a running
server picks up an edit to librechat.yaml. Two races could undo that:

- A base reload that read the old YAML before a write's clear cached it after the clear, and the
  base has no TTL, so the old base was served until the next write.
- A request that read the old base before the clear, and was still resolving principals, cached
  its merge under the new override generation for the override TTL.

The config service now keeps a base-config generation that `clearAppConfigCache` bumps. A reload
publishes the base, the tools and the last good config only if its generation is still current,
a read after a clear never joins a reload started before it, and the newer reload starts once the
stale one settles so the loader's process-wide side effects end with the newest YAML. Merges
capture the generation, base included, before they read the base config.

Tests:
- service.spec: a clear during a reload (stale success, stale failure, a read waiting on the newer
  reload) and a merge built on the pre-clear base.
- invalidation.redis.spec: a write on one instance makes another reload librechat.yaml.
- admin-config-reload.spec (api): every admin config write route, through the real router,
  handlers, database and config service, reloads librechat.yaml; a request with nothing to write
  does not.
- invalidateConfigCaches.spec no longer asserts an order the parallel clears do not have.

* 🧬 fix: Encode Config Keys in Objects Built in Another Realm

The key encoder treated an object as plain only when its prototype was this realm's
Object.prototype, and returned any other object unchanged, keys and all. Under Jest,
structuredClone (which the secret pipeline uses) returns objects from the host realm, so the
items of an array the pipeline had cloned, such as Azure groups or custom endpoints, reached
the engine with their dotted keys raw. Node in production has one realm, but the encoding must
not depend on prototype identity: a skipped object stores its dotted keys raw without an error.

A plain object is now one whose prototype is null or a root Object.prototype from any realm.
The encoding spec covers an object built in a vm context.

* 🧪 test: Check Dotted Record Keys Reach the Store's Whole-Document Write

dev's admin config route test asserted the keys on patchConfigFields, which the handlers no
longer call; it now asserts them on the draft writeConfigDocument receives.

* 🚩 feat: Publish the expectedRevision, prunesEmptyParents and awaitedInvalidation Config Flags

GET /api/admin/config/meta now reports the three write-coherence behaviours this branch
implements, so an admin client can drop the workarounds each one replaces:

- expectedRevision: responses carry `revision`; PUT, PATCH /fields, POST /fields/tombstone and
  PATCH /active take `expectedRevision` in the body and both DELETE routes in the query. A
  mismatch answers 409 CONFIG_VERSION_CONFLICT with currentRevision, a malformed value 400
  INVALID_EXPECTED_REVISION; unpinned writes re-read and re-apply up to 3 times.
- prunesEmptyParents: a field delete or tombstone removes the objects it empties, stopping at
  arrays; shells-only documents count as empty for assign-only lifecycle calls.
- awaitedInvalidation: writes answer after the config caches are cleared (broadcast with Redis),
  GET /base bypasses the merged-config cache, and a merge read before a clear is never cached.

meta.spec probes each one against the behaviour: a stale and a malformed precondition on a
PATCH, the store operations' unset and tombstone on nested objects and an array element, and a
held invalidation, the GET /base read options and a merge raced by a clear.

* 🔀 fix: Keep Config Keys Raw Where the Engine Stores Them, So Older Servers Read Them During Upgrades

Servers before the key encoding (v0.8.9-rc1 among them) read every override key literally.
The Mongoose binding encoded every write, so during a rolling upgrade the first write from an
upgraded server turned a stored `gpt-4.1` or Bedrock inference profile into `gpt-4%2E1` for
every server not yet upgraded, and an older server's restore of a deleted group or role
config dropped the `keyEncoding` marker its schema lacks, after which upgraded servers read
the encoded keys literally.

The binding now sends keys raw first and encodes them only when the engine refuses a raw
field name (DocumentDB, Cosmos, FerretDB or MongoDB 4.4 for `$` names): the refused write
changed nothing, so it runs again encoded under the same revision precondition, and that
process encodes later writes holding such keys directly. MongoDB 4.4+ stores dotted names
(5.0+ also `$` names inside a value), so there an older server reads exactly what it wrote. A
document whose raw keys spell escape sequences while none holds a raw `.`, `$` or NUL is
still written encoded, since a raw read would decode it. A raw write clears the marker.

Reads decode a marked document (raw keys an older server added beside encoded ones, or wrote
under a marker it left, carry no escape sequence and read as stored) and an unmarked one
unless some key holds a raw `.`, `$` or NUL, which no encoded document does; that reads an
encoded document whose marker an older server dropped. Encoding stays in the binding for
every engine that needs it (decision #15); the residual cases are in documentdb-compat.md.

* ⏱️ fix: Remove Config Cache Writes a Clear Overtook in Flight

With an asynchronous APP_CONFIG store (Redis or a custom Keyv store), a clear can bump the
generation and finish its delete after a reload or merge checked the generation but before its
`set` landed. The older value then repopulated `_BASE_` until the next write, or a merged
config for the full override TTL.

The base reload, the last-good restoration and the merge cache now check the generation again
once their write lands and delete what they wrote if a clear overtook it. A deleted entry only
costs the next read a reload or a merge; a newer base load cannot be removed this way, since it
starts only after the stale one settles.
2026-10-22 13:04:35 +02:00
Danny Avila
9ea3c5647d
🔑 fix: Use Admin-Stored Web Search Credentials at Runtime (#17323)
* 🔑 fix: Use Admin-Stored Web Search Credentials at Runtime

Use admin-stored (encrypted or literal) web search credentials as system
credentials at runtime, add the Keenable key to the secret registry, bind a
stored key to the layer that set its URL, and require re-entering the key when
a write changes or removes that URL (checked atomically with a version
precondition).

* 🧹 fix: Keep Dotted-Key Write Guarantees on Version-Checked Web Search Writes

A version-checked write the caller saw absent first creates an empty
document. When the database then refuses the write (a key name it cannot
store, answered 422 CONFIG_KEY_NOT_STORABLE), remove that document again
so a refused write stores nothing, unless another write has changed it.

Secret re-entry issues now carry `segments` like every other config
issue.

* 🚩 feat: Publish the webSearchStoredKeys Feature Flag in the Config Contract

GET /api/admin/config/meta now reports `webSearchStoredKeys: true`, so an
admin client can stop warning that only ${VAR} web search references work at
runtime. meta.spec probes the behaviour behind it: an encrypted stored key
reaches web search auth, the Keenable key is encrypted on write, and changing
a URL beside a stored key requires entering the key again.

* 🧪 test: Type the Refused-Write Driver Stub in the Config Methods Spec

* 🧪 test: Expect the Write Precondition Arguments in the Admin Config Route Test

The handler passes a session slot and an optional version precondition to
patchConfigFields since the web search re-entry check; the route test added
in #17322 asserted the earlier five-argument call.

* 🧹 fix: Surface a Failed Cleanup of the Empty Config Document

When a version-checked write the database refused cannot remove the empty
document created for it, let that failure reach the handler (500) instead
of logging it and answering 422 CONFIG_KEY_NOT_STORABLE, which promises that
nothing was stored.

* 📝 docs: State the Web Search Stored-Key Binding Rule Exactly in the webSearchStoredKeys Flag

The flag's doc said a stored key goes only to a URL the same layer set. The merge rule keeps a stored key over an inherited ${VAR} or default URL, so the doc now names the exact rule: a stored key reaches a URL stored in config only when one layer set both, a later layer that changes, blanks or removes the URL drops the key, and a write that changes or removes the URL must enter the key again.

* 🧹 refactor: Read the Secret Registry Through getConfigSecretRules Only

The dev merge kept this branch's getConfigSecretFields and getConfigArraySecretFields beside dev's getConfigSecretRules, which returns the same registry data. Only the secret specs used the two extra accessors. They now read getConfigSecretRules().secretFields and .arraySecretFields, so the accessors and their exported ConfigSecretField and ArraySecretField interfaces (duplicates of AdminConfigSecretFieldRule and AdminConfigArraySecretFieldRule) are gone.
2026-10-22 11:29:20 +02:00
Danny Avila
2591a15c10
🔑 feat: Keep Dotted Record Keys in Admin Config Writes and Publish the Config Contract (#17322)
* 🔑 feat: Keep Dotted Record Keys in Admin Config Writes and Publish the Config Contract

Keep dotted record keys in admin config write bodies behind a path-scoped
sanitizer, reject $-prefixed and prototype keys after auth, report dotted_key
issues with segments, and publish GET /api/admin/config/meta (features, rules,
limits, versions) so clients can gate on backend capabilities.

* 🧪 test: Expect Issue Segments in the Config Override e2e Scenario

Override validation issues now carry `segments`; assert the full issue shape.
2026-10-22 09:59:08 +02:00