Commit graph

736 commits

Author SHA1 Message Date
Danny Avila
596f806f60 🛡️ fix: Strict Opt-In Skills Activation per Agent (#12823)
* 🛡️ fix: Strict opt-in skills activation per agent

Skills were activating on every agent run that had the capability +
RBAC enabled, regardless of whether the user (ephemeral) or author
(persisted) had opted in. `scopeSkillIds(undefined)` fell through to
"full accessible catalog" whenever `agent.skills` was unset, which is
the default state for any agent created before skills existed and for
every ephemeral agent.

Activation now requires an explicit signal:
- Ephemeral agent → per-conversation skills badge toggle.
- Persisted agent → new `skills_enabled` master switch on the agent
  doc, surfaced as a toggle in the Agent Builder skills section.
  Enabled + empty/undefined allowlist = full accessible catalog;
  enabled + non-empty allowlist = narrow to those ids; disabled (or
  undefined) = no skills available, even if an allowlist is set.

Centralised the predicate in `resolveAgentScopedSkillIds` so the
primary-agent path, handoff/discovery, the subagent loop, and both
OpenAI controllers all share one source of truth. Frontend `$`
popover scope mirrors the same logic so the UI never offers skills
the backend would refuse to activate.

* test: mock resolveAgentScopedSkillIds in agent controller specs

* refactor: address review findings on skills opt-in PR

- AgentConfig: associate skills label with toggle via htmlFor for
  click/keyboard affordance; simplify Switch handler to Boolean(value).
- skills: mark scopeSkillIds as @internal so runtime callers continue
  to route through resolveAgentScopedSkillIds and inherit the activation
  predicate (ephemeral toggle, persisted skills_enabled).

* fix(agents): include skills_enabled in agent list projection

Without this field, agents loaded via the list endpoint hydrate into the
client agentsMap with skills_enabled === undefined, causing the `$`
skill popover to hide every skill on a fresh page load even when the
agent was saved with skills_enabled: true.

* fix(skills): fail closed for persisted agents during agentsMap hydration

Returning undefined while the agents map loads let the popover render the
full catalog for a persisted agent before we could read its
skills_enabled flag, so the user could pick a skill the backend would
then refuse for the turn. Match the strict opt-in contract by returning
[] until the map is authoritative.

* refactor(skills): extract skillsHintKey for readability

Replaces the nested ternary in the skills section JSX with a
pre-computed constant so the activation -> hint key mapping reads
top-down.

* refactor(skills): unflatten skillsHintKey to remove nested ternary
2026-04-25 04:02:01 -04:00
Danny Avila
d83cb84f59 🪆 feat: Subagent configuration in Agent Builder (#12725)
* 🪆 feat: Subagents configuration (isolated-context child agents)

Surfaces the new @librechat/agents `SubagentConfig` primitive in the Agent
Builder. Subagents let a supervisor delegate a focused subtask to a child
graph running in an isolated context window: verbose tool output stays in
the child, only a filtered summary returns to the parent.

Data model: new `subagents: { enabled, allowSelf, agent_ids }` on Agent,
wired through the Zod, Mongoose, and form schemas plus a new
`AgentCapabilities.subagents` capability (enabled by default).

Backend: `initialize.js` loads explicit subagent configs alongside handoff
agents, and drops subagent-only references from the parallel/handoff maps
so they don't leak into the supervisor's graph. `run.ts` emits
`SubagentConfig[]` on the primary `AgentInputs` — a self-spawn entry when
`allowSelf` is enabled plus one entry per configured agent.

UI: an "Advanced" panel section with an enable toggle, a self-spawn
toggle, and an agent picker (capped at 10). Enabling without adding
agents still yields self-spawn; disabling self-spawn with no agents shows
a warning. A capability flag gates the whole section.

* 🪆 feat: Stream subagent progress to UI (dialog + inline ticker)

Pairs with the @librechat/agents SDK change that forwards child-graph
events through the parent's handler registry (danny-avila/agents#107):

- Self-spawn and explicit subagents can now use event-driven tools,
  because child `ON_TOOL_EXECUTE` dispatches reach our ToolService via
  the parent's registered handler.
- The same forwarding path wraps the child's run_step / run_step_delta
  / run_step_completed / message_delta / reasoning_delta dispatches in
  a new `ON_SUBAGENT_UPDATE` envelope, with start/stop/error bookends.

Backend: `callbacks.js` registers an `ON_SUBAGENT_UPDATE` handler that
forwards each envelope straight to the SSE stream.

Frontend:
- `useStepHandler` consumes `ON_SUBAGENT_UPDATE` events and merges them
  into a per-tool_call Recoil atom (`subagentProgressByToolCallId`).
  First-seen `subagentRunId` claims the most-recent unclaimed `subagent`
  tool call in the active response message — a temporal mapping, no SDK
  wire-format change needed to correlate child runs with parent tool
  calls.
- New `SubagentCall` part component replaces the default `ToolCall`
  rendering when `toolCall.name === Constants.SUBAGENT`: compact status
  ticker showing the 3 most recent update labels, clickable to open a
  dialog with the full activity log + final markdown-rendered result.
- Adds `Constants.SUBAGENT`, `StepEvents.ON_SUBAGENT_UPDATE`, and
  `SubagentUpdateEvent` type in data-provider.

Tests:
- `packages/api npx jest run-summarization` — 23 pass
- `api npx jest initialize` — 16 pass
- `npm run build` — clean

Dependency note: bumps `@librechat/agents` to `^3.1.67-dev.1` — requires
the SDK PR (danny-avila/agents#107) to be merged to dev and published
before this PR merges. `ON_SUBAGENT_UPDATE` is absent from dev.0, so the
handler registration would be a no-op with the older SDK but would not
crash.

* 🪆 fix: address Codex review and review audit on subagents

Stacks on top of the SDK change in danny-avila/agents#107 (bumped to
`^3.1.67-dev.2`).

- **P1 (`initialize.js`)**: subagent-only agents were being deleted from
  both `agentConfigs` AND `agentToolContexts`. The tool-execute handler
  resolves execution context (agent, tool_resources, skill ACLs) from
  `agentToolContexts`, so explicit subagents would run without their
  configured resources and skip action tools. Now only `agentConfigs`
  is pruned — tool context stays intact.
- **P2 (`AgentSubagents.tsx`)**: toggling subagents off set the form
  field to `undefined`; `removeNullishValues` stripped it from the
  PATCH, leaving the server copy enabled. Now it persists an explicit
  `{ enabled: false, ... }` so the update actually clears state.

- **Finding 1 (MAJOR)** — `agent_ids` Zod schema gains `.max()` via a
  new `MAX_SUBAGENTS` export from `data-provider` (shared with the UI
  cap). Crafted payloads can't trigger hundreds of `processAgent`
  calls.
- **Finding 2 (MAJOR)** — `subagentProgressByToolCallId` atomFamily
  atoms are now tracked in a ref and reset from `clearStepMaps` via a
  `useRecoilCallback({ reset })`. No monotonic growth across a session.
- **Finding 3 (MAJOR)** — early-arriving `ON_SUBAGENT_UPDATE` events
  whose parent `tool_call_id` is not yet mapped are now buffered in
  `pendingSubagentBuffer` (keyed by `subagentRunId`) and replayed in
  arrival order once correlation completes. Mirrors the existing
  `pendingDeltaBuffer` pattern.
- **Finding 4 (MAJOR)** — switched to deterministic correlation via
  the new `parentToolCallId` that SDK `3.1.67-dev.2` threads through
  from `ToolRunnableConfig.toolCall.id`. Temporal fallback now iterates
  oldest-unclaimed-first (forward), matching tool-call creation order,
  so concurrent spawns map correctly.
- **Finding 6 (MINOR)** — `agent_ids` are deduped on the backend via
  `new Set(...)` before the load loop. Duplicates no longer produce
  duplicate `SubagentConfig` entries visible to the LLM.
- **Finding 7 (MINOR)** — events array inside each Recoil atom is
  capped at 200 entries. Long-running subagents no longer replay O(n)
  spreads on every update; the dialog log still shows the cap window.
- **Finding 8 (MINOR)** — documented: subagents are loaded only for
  the primary agent this release (handoff children get self-spawn but
  not explicit sub-subagents). In-code comment added so the next
  maintainer doesn't wonder.
- **Finding 9 (NIT)** — removed `{!isSubmitting && null}` dead code
  and the misleading announce-polite comment in `SubagentCall`.

- New `validation.spec.ts` — 9 tests covering the cap on
  `agent_ids.length` at the subagent schema, agent-create, and
  agent-update layers.
- `run-summarization` — 23 pass, `initialize` — 16 pass, total backend
  package: 103 pass across touched areas.

Findings 5 (component tests) and 10 (micro-allocation) are tracked
but deferred; the former needs a Recoil-RenderHook harness that isn't
in this PR's scope, and the latter has negligible impact (one `Array.from`
per subagent run).

* 🧪 test: integration coverage for subagent correlation + backend loading

Addresses the follow-up audit on #12725 with real-code tests (no mock
handlers, only the existing setMessages/getMessages spies and the
standard mongodb-memory-server harness).

Six new tests under a dedicated `describe('subagent loading')`:
- loads a configured subagent, populates `subagentAgentConfigs`, keeps
  it out of `agentConfigs`
- **P1 regression guard**: drives the real `toolExecuteOptions.loadTools`
  closure with the subagent id and asserts `loadToolsForExecution` is
  called with `agent: <subagent>`, `tool_resources`, `actionsEnabled`.
  If anyone deletes `agentToolContexts` again, this fails.
- dedup: three copies of the same id load the agent once
- overlap: agent referenced both as handoff target and subagent stays in
  `agentConfigs`
- capability gate: admin disabling `subagents` suppresses loading even
  when the agent has a config
- per-agent disable: `subagents.enabled: false` skips loading entirely

Five new tests under `describe('on_subagent_update event')` using a
real `RecoilRoot` and a companion `useRecoilCallback` reader so writes
from the hook are observable:
- deterministic correlation via `parentToolCallId` (happy path with
  SDK dev.2+)
- fallback: oldest-unclaimed tool call wins for concurrent spawns
  without `parentToolCallId`
- early-arrival buffer: updates with no mapping get buffered and
  replayed once the tool call appears
- event cap: 205 updates collapse to 200 retained, oldest dropped
- `clearStepMaps` resets tracked atoms back to their null default

- F2 — added explicit `// TODO` marker for handoff-subagent-loading
  extension (matches the comment that referenced it).
- F3 — dropped the unnecessary `MAX_SUBAGENTS as MAX_SUBAGENTS_CAP`
  alias; just import the constant directly.
- Bumped `@librechat/agents` to `^3.1.67-dev.3` to pick up the SDK's
  paired test additions.

- `api/server/services/Endpoints/agents/initialize.spec.js` — 22 pass
  (6 new + 16 existing)
- `packages/api/src/agents/validation.spec.ts` +
  `run-summarization.test.ts` — 103 pass
- `client/src/hooks/SSE/__tests__/useStepHandler.spec.ts` — 48 pass
  (5 new + 43 existing)

* 🪆 fix: strip parent run summary + discovered tools from subagent inputs

Codex P1 on #12725: `buildSubagentConfigs` reused the shared
`buildAgentInput` factory for each explicit child, and that factory
always stamps the parent run's `initialSummary` (cross-run conversation
summary) and `discoveredTools` (tool names the parent's LLM searched
earlier) onto every `AgentInputs` it returns. When subagents were
enabled on a conversation that had already been summarized, every
child inherited that summary — silently defeating the isolated-context
contract and burning extra tokens on unrelated prior chat.

Fix in `run.ts`: after `buildAgentInput(child)`, explicitly clear
`childInputs.initialSummary` and `childInputs.discoveredTools` before
attaching to the `SubagentConfig`. The parent keeps both — that's how
the supervisor receives cross-turn context — but the child starts
fresh.

Paired with danny-avila/agents#107 (bumped to `^3.1.67-dev.4`), which
adds the equivalent strip inside `buildChildInputs` to cover the
self-spawn path where the SDK clones parent `_sourceInputs` directly
and LibreChat never sees the intermediate shape. Belt and suspenders.

Regression test (new):
- `does NOT leak the parent run initialSummary into an explicit child
  (Codex P1 regression)` — sets `initialSummary` on the run, enables
  subagents with an explicit child, asserts the parent still has the
  summary but `childConfig.agentInputs.initialSummary` is `undefined`.
  Same for `discoveredTools`. 24 pass.

* 🪆 fix: capability gate applies to handoff agents + parallel subagent test

### Codex P2 — handoff agents kept `subagents` after capability disabled
The endpoint-level `AgentCapabilities.subagents` gate only cleared
`subagents` on `primaryConfig`. Handoff agents loaded into
`agentConfigs` retained their persisted `subagents.enabled: true`,
and because `run.ts` calls `buildSubagentConfigs` for every agent
input, self-spawn would still fire on a handoff target even when the
admin had disabled the capability globally.

Fix in `initialize.js`: after the subagent loading block, when the
capability is off, iterate `agentConfigs.values()` and clear
`subagents` + `subagentAgentConfigs` on every loaded config.

Regression test: `clears subagents on handoff agents too when
capability is disabled (Codex P2 regression)` — seeds a handoff target
with its own `subagents.enabled: true`, disables the capability at
the endpoint, asserts both primary AND handoff have `subagents`
undefined in the client args. 23 init tests pass.

### Parallel subagent correlation — user-requested verification
Added `keeps parallel subagent streams independent when events
interleave` to `useStepHandler.spec.ts`. Two `subagent` tool calls
seeded side by side, 6 interleaved `ON_SUBAGENT_UPDATE` envelopes
dispatched (a-start, b-start, a-step, b-step, a-stop, b-step), each
carrying its own `parentToolCallId`. Asserts each `tool_call_id`'s
Recoil bucket accumulates only its own run's events, statuses reflect
each run independently (`call_a` → stop, `call_b` → run_step), no
cross-contamination. 49 step-handler tests pass.

* 🪆 fix: SubagentCall detects cancelled / errored states (Codex P2)

Codex P2 on #12725: the old `running` check only consulted
`initialProgress` and the subagent's phase. A user stop, dropped
stream, or backend crash before a terminal `stop`/`error` envelope
arrived would leave the ticker permanently stuck on "working…". Other
*Call components (ToolCall.tsx) already model this via
`!isSubmitting && !finished` → cancelled.

Mirror that pattern. Re-introduce `isSubmitting` on `SubagentCallProps`
(the prop was dropped earlier as 'unused' — that was a bug) and resolve
status as a tri-state:

- `finished`  — initialProgress >= 1, or subagent `stop`/`error`
- `cancelled` — `!isSubmitting && !finished`
- `running`   — neither

New locale keys `com_ui_subagent_cancelled` + `com_ui_subagent_errored`
swap in the right header text per state.

Tests: new `SubagentCall.test.tsx` covers all four states with a real
`RecoilRoot` and a `useRecoilCallback` seeder — no mocked store — 5/5
pass. Includes an explicit P2 regression test that simulates the
`isSubmitting=false, progress.status='run_step', initialProgress<1`
scenario and asserts the cancelled label renders.

* 🪆 feat: semantic ticker + aggregated content-part dialog for subagents

Two rounds of feedback on #12725:

### Ticker — user-readable lines, not raw event names

The old ticker showed \`on_run_step\`, \`on_message_delta\`, etc. — not
meaningful to users. Replaced with \`buildSubagentTickerLines\`, a pure
helper that walks the \`SubagentUpdateEvent\` stream and emits:

- message/reasoning deltas → a single live "Writing: <last 60 chars>"
  (or "Reasoning: …") line that updates in place as chunks arrive
- run_step with tool_calls → "Using calculator(expression=42*58)" for
  a single call, "Using tool: a, b" for parallel (args dropped when
  multiple so the line stays short)
- run_step_completed → "calculator → 42*58 = 2436" (output truncated
  to 48 chars; falls back to "Tool X complete" when output is empty)
- error → "Error: <message>"
- start / stop / run_step_delta → suppressed (too granular / lifecycle-only)

Args and output pass through \`summarizeArgs\` / \`summarizeOutput\`
which flatten JSON to \`key=value\` pairs and head-truncate long
strings so a 200-line tool output never bloats the ticker.

### Dialog — aggregated content parts via leaf renderers

\`aggregateSubagentContent\` folds the raw event stream into
\`TMessageContentParts[]\` — text/reasoning delta streaks collapse into
single \`TEXT\` / \`THINK\` parts, tool calls become \`TOOL_CALL\` parts,
and \`run_step\` boundaries correctly break text runs around tool
calls. The dialog iterates those parts through a \`SubagentDialogPart\`
renderer that delegates to the existing \`Text\`, \`Reasoning\`, and
\`ToolCall\` leaf components — the same sub-components \`<Part />\` uses
— wrapped in a minimal \`MessageContext\` so reasoning expand state and
cursor animation work.

Leaf components are used directly rather than importing \`<Part />\`
itself to avoid a module cycle (Part → Parts/index → SubagentCall →
Part) and to sidestep a hypothetical nested-subagent rendering.

### Tests

- \`subagentContent.test.ts\` — 19 pure-function tests covering the
  aggregator (text concat, reasoning concat, tool call lifecycle,
  interleaving, phase suppression, late-arriving completions) and the
  ticker builder (live preview truncation, args/output snippets,
  parallel-call handling, output truncation, i18n formatter override).
- \`SubagentCall.test.tsx\` — 9 component tests: 5 status-resolution
  (existing) + 2 ticker (semantic text, delta collapse) + 2 dialog
  (aggregated parts routed to leaf renderers, raw-output fallback).

### Locale keys

New: \`com_ui_subagent_ticker_writing\`, \`…_reasoning\`, \`…_error\`,
\`…_using\`, \`…_using_with_args\`, \`…_tool_complete\`,
\`…_tool_output\`. Preserves i18n at the display layer while the
helper stays pure.

* chore: drop unused com_ui_subagent_activity_log locale key

The dialog no longer renders an "Activity log" section — the new
content-parts renderer replaced it. Also tweaks the dialog description
copy to match.

* 🪆 fix: subagent dialog order, persistence, auto-scroll, width

Follow-up pass addressing the four issues observed in real runs
against a live subagent-using parent.

### Aggregator ordering (reasoning appearing after text it preceded)

Reproducible pattern: LLM emits reasoning → text → tool call in that
order, but the dialog rendered text BEFORE reasoning in the content
array. Root cause: `aggregateSubagentContent` maintained `currentText`
and `currentThink` buffers in parallel and only flushed them at a
`run_step` boundary in a fixed (text, think) order, losing the actual
arrival order.

Fix: when a text chunk arrives, close any open think buffer first
(pushes it into the content array right then); symmetric for think →
text. Two new regression tests cover the exact reasoning → text →
tool_call sequence from the screenshot and the repeated
reasoning ↔ text flow across a turn.

### Content persists after completion (markdown not rendering when done)

`clearStepMaps` was calling `resetSubagentAtoms()` at stream end,
which wiped every `subagentProgressByToolCallId` entry. Once reset,
`contentParts.length === 0` and the dialog fell back to rendering the
raw `output` string with plain text — hence the literal `##`/`**` in
the completed-state screenshot. Stopped resetting; the atoms are
bounded per-call (200-event cap) and per-conversation (one per
subagent spawn) so growth matches the rest of the conversation state.
`resetSubagentAtoms` is kept for a future conversation-switch caller.

Also: routed the raw-`output` fallback (older subagent runs recorded
before the event forwarder existed) through the same
`SubagentDialogPart` → `Text` leaf that content parts use, so its
markdown renders the same way.

### Auto-scroll to bottom while running

Added a `scrollRef` on the dialog body and a `useEffect` that pins
`scrollTop = scrollHeight` while the dialog is open AND the subagent
is running. Triggers on `contentParts.length` (new tool calls / part
boundaries) and `events.length` (intra-part deltas) so the cursor
tracks text streaming. Disabled post-completion so re-opening a
finished run doesn't yank to the bottom.

### Wider dialog

Went from `max-w-2xl` (42rem / 672px — too cramped on maximized
laptop windows) to `w-[min(95vw,64rem)] max-w-[min(95vw,64rem)]`.
Narrow on phones, scales up to 64rem on desktop, always leaves a bit
of margin from the viewport edge. Bumped `max-h-[65vh]` on the scroll
area to give the extra width room to breathe vertically too.

### Tests

- `subagentContent.test.ts` — 21 pass (2 new ordering regressions).
- `useStepHandler.spec.ts` — 49 pass (1 updated to assert atoms are
  *preserved* on clearStepMaps).
- `SubagentCall.test.tsx` — 9 pass (unchanged; aggregator-level tests
  cover the ordering).

* 🪆 feat: persist subagent_content via SDK createContentAggregator

Per-request map of createContentAggregator instances keyed by the
parent's tool_call_id. ON_SUBAGENT_UPDATE handler feeds each event
into the matching aggregator (phase → GraphEvent mapping); AgentClient
harvests contentParts onto the subagent tool_call at message save so
the child's reasoning / tool calls / final text survive a page refresh.

Reusing the SDK's battle-tested aggregator instead of a bespoke one
keeps the persisted shape identical to the parent graph's output and
drops ~100 lines of custom aggregation code.

* 🪆 fix: incremental subagent aggregation + dialog render parity

**Disappearing tool_calls**: the Recoil atom trimmed events to a 200-long
rolling window, so verbose subagents could shed the `run_step` that
originally created a tool_call part — rebuilding content from the trimmed
window then produced only the surviving text/reasoning. Fix: fold each
envelope into `contentParts` incrementally in the atom as it arrives
(new `foldSubagentEvent` + cursor state). Event trim window now affects
only the ticker, never the dialog.

**Render parity**: dialog now applies `groupSequentialToolCalls` and
renders single parts through `Container` + grouped batches through
`ToolCallGroup` — same spacing and "Used N tools" collapsing the main
message view uses.

**Width**: `min(96vw, 80rem)` — wider on big screens, still responsive.

**Labels**: "Subagent: X" is jargon. Named subagents render as
`Running "{name}" agent` / `Ran "{name}" agent` (past tense on
completion); self-spawns use `Running subtask` / `Ran subtask` since
`Running "self" agent` reads badly.

* 🪆 polish: subagent dialog parity + agent avatar in header

**Labels**: drop "subtask" framing. Self-spawn shows `Running agent` /
`Ran agent` (past tense on completion); named subagents stay
`Running "X" agent` / `Ran "X" agent`.

**Dialog render parity**: stop wrapping every part in `Container`.
TEXT keeps its `Container` (gap-3 + `mt-5` sibling margin), THINK and
TOOL_CALL render bare so their own wrappers set the full-column width
the regular message view gives them — matches the main `<Part>` dispatch.
Outer scroll region now uses `px-4 py-3` padding and a
`max-w-full flex-grow flex-col gap-0` inner wrapper, mirroring the
`MessageParts` container the main conversation uses.

**Avatar**: header icon now renders the subagent's configured avatar
via `MessageIcon` when `useAgentsMapContext()` has the child agent,
falling back to the `Users` SVG (which keeps its running-state pulse).
Same icon-left-of-label pattern the tool UI uses.

* 🪆 polish: subagent group label, ticker throttle + tail-ellipsis, scroll button

**Grouped label**: ToolCallGroup now detects all-subagent batches and
labels them "Running N agents" / "Ran N agents" instead of "Used N
tools". Mixed batches keep the existing label. The tool-name summary
is suppressed for all-subagent groups (every entry dedupes to
"subagent", which adds nothing).

**Ticker width + tail-ellipsis**: raise the preview cap to 300 chars so
wide containers aren't half-empty, and flip the ticker `<li>` to
`dir="rtl"` so `text-overflow: ellipsis` clips the *oldest* characters
(visually the left edge) — the newest tokens stay pinned to the right
regardless of container width. Bidi lays out the Latin text LTR
internally, the rtl only affects which side gets the ellipsis.

**Throttle**: `useThrottledValue` hook (trailing-edge, 1.2s) smooths
the live `Writing: …` preview so tokens no longer strobe past the
eye faster than they can be read. Ref-based internals (not `useState`)
avoid infinite-update loops when the upstream value is a new-reference
each render; `NEGATIVE_INFINITY` sentinel ensures the very first value
passes through synchronously so tests and first paint aren't delayed.

**Scroll-to-bottom**: dialog tracks `isAtBottom` with a 120px
threshold; auto-scroll only engages when the user is already following
along, and a persistent jump-to-latest button appears whenever they
scroll up — no more fighting the auto-scroll to read back.

* 🪆 polish: snappier ticker, prefix-safe labels, agents icon, readable lines

**Ticker lines are now incrementally aggregated in the atom** — same
pattern as contentParts. The raw-events rolling window is gone; event
volume no longer caps what the ticker can display. Verbose subagents
that used to drop early tool_call lines out of the window now keep the
full 3-line history (using_tool, tool_complete, writing).

**Discriminated-union ticker lines** split a constant prefix (e.g.
"Writing:") from a tail-truncatable body. The prefix lives in a
`shrink-0` span so it never gets clipped when the body overflows; the
body uses `dir="rtl"` only on itself — scoped so non-streaming lines
(e.g. "Waiting for first update…") can't get their trailing ellipsis
flipped by bidi.

**Content-aware throttle**: 800ms interval (down from 1200ms), skipped
entirely while the live buffer is below 120 chars. Early tokens now
appear immediately — no more "Reasoning: I" sitting blank for a full
second before the next heartbeat. Once the preview is long enough to
fill the container, throttling kicks in at the tighter interval.

**Header label** is now a constant verb + optional muted sub-label.
Base reads "Running agent" / "Ran agent" / "Cancelled agent" / "Agent
errored" for every subagent; named subagents get the configured agent
name rendered to the right in secondary text (self-spawns and
unresolved names omit it — "Running self agent" is nonsense).

**ToolCallGroup** now detects `allSubagents` and swaps `StackedToolIcons`
for a single `Users` glyph — otherwise the group header shows a wrench
("tool") icon next to "Ran 5 agents", which reads wrong.

* 🪆 feat: delimiter-aware tool labels in ticker + full-width tool lines

New shared `parseToolName` helper in `client/src/utils/toolLabels.ts`
— single source of truth for splitting `<tool>_mcp_<server>` ids and
mapping native tool names (web_search, execute_code, …) to their
friendly translation keys. `ToolCallGroup` drops its inline copy and
pulls from this helper.

Ticker tool lines now use the shared parser + a new `ToolIdentifier`
sub-renderer so the live log reads like the main tool UI:

  - MCP tool  → `<server> · <code-badge:tool>` (e.g. "github · `search_code`")
  - Native   → friendly name from `TOOL_FRIENDLY_NAME_KEYS`
  - Unknown  → bare `<code>` badge of the raw id

The `using_tool` / `tool_complete` rows now render with a
`flex w-full items-baseline gap-1 overflow-hidden` layout matching
the writing/reasoning rows — they take the full container width
instead of collapsing to content size. Output snippets on
`tool_complete` get the same tail-side `dir="rtl"` ellipsis so the
newest characters stay flush-right when the container is narrow.

Dropped the now-unused template i18n keys
(`com_ui_subagent_ticker_using_with_args`,
`com_ui_subagent_ticker_tool_complete`,
`com_ui_subagent_ticker_tool_output`) in favor of tokens the JSX
composes structurally. Only English is touched per the project rule;
other locales follow externally.

* 🪆 fix: dialog scroll button + auto-scroll during streaming deltas

Two race/trigger bugs in the dialog's scroll behavior:

**Button never showed**: `addEventListener('scroll', …)` in a
`useEffect` ran before Radix's portal had actually committed the
scroll container, so `scrollRef.current` was still null — the listener
never attached, `isAtBottom` stayed stuck at its initial `true`, and
the jump-to-latest button was never rendered. Swap to React's
`onScroll` prop on the element itself so the handler wires up as part
of DOM commit, not a post-commit effect.

**Auto-scroll stalled during text streaming**: the pin-to-bottom
effect only re-fired on `contentParts.length` changes. Message/reasoning
deltas extend the last TEXT/THINK part's `.text` without changing the
array length — so the view would drift up as tokens piled in and
never catch back up. Replace the length-dep effect with a
`ResizeObserver` on the inner content div; every height change (new
part or in-place growth) triggers a scroll-pin when the user is still
at the bottom.

* 🪆 fix: drop leading ellipsis from ticker body

truncatePreview was prepending ... to the tail when the buffer
exceeded 300 chars. The component's CSS already produces a left-side
ellipsis for overflow via dir=rtl + text-overflow: ellipsis — stacking
a data-level ellipsis on top renders a stray dot character right after
the Writing: / Reasoning: label (Writing: .Sure!), which looks like a
typo to the reader.

Data now returns just the last 300 chars when truncating; CSS handles
the visual cue whenever the body actually overflows its container.

* 🪆 fix: Codex review — subagent isolation + concurrent-safe throttle

Three findings from the @codex review pass, all valid:

**P1 — buildAgentInput leaks parent discovered-tool state into subagent
children.** `buildAgentInput` mutates `agent.toolRegistry`
(`overrideDeferLoadingForDiscoveredTools` flips `defer_loading:true→false`
on tools the parent previously searched for) and appends those tools'
definitions to the returned `toolDefinitions` before the function
returns. `buildSubagentConfigs` was clearing the reported
`initialSummary` / `discoveredTools` fields on the returned
AgentInputs, but that happened post-return — the registry writes and
extra tool definitions persisted on the child, silently defeating
context isolation and inflating the child's prompt.

Fix: `buildAgentInput` now takes an `isSubagent` flag that gates the
registry-mutation block and omits `initialSummary` /
`discoveredTools` at the source. `buildSubagentConfigs` passes
`{ isSubagent: true }` for every explicit child; no post-hoc cleanup
needed.

**P2 — ToolCallGroup labels a finished subagent group as still
running when the child returned no output.** `getToolMeta` computed
`hasOutput` as `!!tc.output`, which is `false` for a completed
subagent that returned empty text (the UI already has an "empty
result" fallback for that case). `allCompleted` would stay `false`
and the group header stuck on "Running N agents" forever.

Fix: treat `tc.progress === 1` as completion too — progress is the
authoritative lifecycle signal, output is just content.

**P2 — useThrottledValue schedules `setTimeout` during render.**
Discarded renders under Strict Mode / Concurrent rendering would
leave orphan timers firing against stale trees.

Fix: move `setTimeout` into a `useEffect` keyed on `[value, intervalMs,
enabled]`. Render-time still mutates refs (idempotent), but timer
scheduling lives post-commit. Cleanup on unmount and on passthrough
transitions is preserved.

* 🪆 fix: Codex P2 — wipe subagent atoms on conversation switch

`clearStepMaps()` intentionally doesn't reset `subagentProgressByToolCallId`
so a user can reopen a completed subagent's dialog mid-conversation,
but `resetSubagentAtoms` was defined and never exposed / called —
so each completed run's aggregated `contentParts` + `tickerState`
stayed resident in the `atomFamily` for the whole app session.
Unbounded growth across multi-conversation sessions.

Expose `resetSubagentAtoms` from `useStepHandler` and fire it from
`useEventHandlers` whenever the URL's `conversationId` changes.
That's the correct cleanup boundary: historical subagent dialogs
rehydrate from persisted `subagent_content` on each `tool_call` at
message-save time, so wiping live atoms on switch doesn't lose any
viewable history — it just releases per-tool-call state that the
old conversation's components no longer subscribe to.

* 🪆 fix: Codex round 3 — subagent registry isolation + post-run label

Two more valid findings.

**P1 — parent-order registry mutation leaks into subagent inputs.**
`overrideDeferLoadingForDiscoveredTools` mutates `agent.toolRegistry`
in place (the Map *and* the LCTool objects inside it). When an agent
appears both as a handoff target (normal graph node) AND an explicit
subagent child, a subagent build that ran before the parent's build
captures a reference to the same registry — the parent's later mutation
leaks through to the child.

Fix: for subagent children (`isSubagent`), clone the `toolRegistry`
Map and shallow-clone each LCTool inside before returning the inputs.
`defer_loading` flips on parent-graph registry mutations can't
propagate across the clone boundary. `toolDefinitions` also gets a
shallow-copy pass so the same isolation holds for definitions the
child carries directly.

**P2 — "Running N agents" label stuck after cancel/error.**
ToolCallGroup's all-subagent label was gated only on `allCompleted`,
which requires every child to have `hasOutput || progress === 1`.
A subagent that gets cancelled (stream ends, no `stop` phase, no
output) never satisfies that — so even after `isSubmitting` flips
false, the header stays on "Running N agents" while each individual
card correctly shows "Cancelled agent".

Fix: derive a `subagentsDone` flag as `allCompleted || !isSubmitting`
and gate the past-tense label on that. Matches the tri-state each
SubagentCall card already resolves (finished / cancelled / running).

* 🪆 fix: Codex P2 — ACL-check subagents.agent_ids on create/update

Codex flagged that `subagents.agent_ids` was accepted as arbitrary
strings on the create/update routes while `edges` got a
`validateEdgeAgentAccess` pass — so users could save subagent
references to agents they can't VIEW. At runtime `initializeClient`'s
`processAgent` ACL gate silently drops those, so the persisted
configuration and the actual behavior diverged in a way that is
difficult to diagnose.

Refactor: extract the id-set → unauthorized-ids check into a shared
`collectUnauthorizedAgentIds`, wrap it with a dedicated
`validateSubagentAccess`, and plumb the same 403-on-failure response
the edge path already returns. Applied on both POST /agents and
PATCH /agents/:id.

* 🪆 fix: Codex round 5 — ACL-disable escape hatch + ticker order

Two valid findings.

**P1 — can't disable subagents after losing access to a child.**
The subagent ACL check ran on every create/update that echoed back
the `agent_ids` list, even when the user was explicitly disabling
the feature. The UI keeps the list intact when toggling `enabled:
false`, so a user who subsequently lost VIEW on any child would be
locked in a 403 loop — every edit (including the one that turns
subagents off) bounces.

Fix: gate the ACL check on `subagents.enabled !== false` at both
the POST /agents and PATCH /agents/:id handlers. Empty list stays
a no-op. Disabling the feature is always permitted.

**P2 — ticker fold merges out-of-order previews across delta
switches.** `foldSubagentEventIntoTicker` carried `textLineIdx` /
`thinkLineIdx` across a reasoning → text → reasoning transition,
so the second reasoning chunk appended to the original reasoning
line instead of starting a new chronological one.

Fix: close the opposite buffer + cursor when a delta-type switch
is detected (same rule the content-parts reducer already applies).
Added a regression test.

* 🪆 fix: Codex round 6 — preserve mid-stream atoms + honor sequential suppression

Two valid findings.

**P2 — atom reset fires on initial chat URL assignment.**
`useEventHandlers` initialized `lastConversationIdRef` from the URL's
current `paramId`, then reset subagent atoms whenever the ref and
`paramId` disagreed. For a brand-new conversation the URL stamp goes
from `undefined → "abc123"` while the first response is still
streaming, which used to drop subagent ticker/content state mid-run
and leave dialogs missing earlier updates.

Fix: only reset when *both* the old and new IDs are non-null and
differ — i.e. a user-initiated switch between two established
conversations. The initial assignment passes through without
clearing.

**P2 — ON_SUBAGENT_UPDATE bypassed `hide_sequential_outputs`.**
Every other streaming handler in `callbacks.js`
(`ON_RUN_STEP`, `ON_MESSAGE_DELTA`, etc.) gates emission on
`checkIfLastAgent` + `metadata?.hide_sequential_outputs`, but the
subagent forwarder did an unconditional `emitEvent` — so intermediate
agents in a sequential chain were leaking their children's activity
to the client even when the chain was configured to suppress
intermediates.

Fix: accept `metadata` and apply the same `isLastAgent ||
!hide_sequential_outputs` gate. Aggregation still runs regardless of
visibility (persistence + dialog depend on it); only the SSE forward
is suppressed.

* 🪆 fix: Codex P2 — gate subagent ACL check on endpoint capability

`validateSubagentAccess` ran on every create/update where
`subagents.enabled !== false`, regardless of the endpoint-level
`subagents` capability. When the capability is off at the appConfig
level, `initializeClient` already strips the `subagents` block at
runtime — so persisted `agent_ids` are inert — but the validation
could still 403 on a legacy record whose referenced child is no
longer viewable, blocking unrelated edits.

Fix: add `isSubagentsCapabilityEnabled(req)` that reads the agents
endpoint's capabilities from `req.config` and gate both the create
and update ACL checks on it. Capability-off environments can update
agents with stale `subagents` data freely; capability-on keeps the
full ACL protection.

* 🪆 fix: Codex P2 — reset subagent atoms on id→null navigation too

Previous guard (both-established) skipped the reset whenever
`paramId` became null/undefined, so navigating from an existing
chat to a "new chat" route left stale subagent progress resident
in the `atomFamily` until the user picked a specific different
chat.

Swap the both-established check for a one-time flag: skip only the
very first `undefined → id` transition (the brand-new-chat URL
stamp that happens mid-stream), then reset on any subsequent
change — id→id, id→null, null→id-after-reset. If the user started
on an established chat the flag is true at mount, so the guard is
a no-op and every navigation resets normally.

* 🪆 fix: Codex round 9 — subagent persistence gate + handoff children

Two valid findings.

**P1 — hide_sequential_outputs also gates persistence.**
The previous fix gated the SSE forward on
`isLastAgent || !hide_sequential_outputs` but still ran the
per-tool-call `createContentAggregator` aggregation unconditionally.
`finalizeSubagentContent` would then attach the hidden intermediate
agent's child reasoning / tool output to the saved message, so a
page refresh could reveal activity that was intentionally suppressed
live. Move the visibility gate to the top of the handler — hidden
agents now skip both aggregation and emission, so
"hide_sequential_outputs" is a consistent "don't record" rule for
subagent traces.

**P2 — handoff agents' explicit subagents were silently dropped.**
`initializeClient` only resolved `subagentAgentConfigs` for the
primary config, so an agent used via handoff that had its own
`subagents.agent_ids` saved in the builder would get self-spawn
only; every explicit child was quietly ignored, creating a
saved-config / runtime mismatch the user couldn't diagnose.

Extract the resolution into a shared `loadSubagentsFor(config)`
helper and invoke it for the primary and every handoff agent in
`agentConfigs`. The `edgeAgentIds` precomputation stays outside
the helper (it's loop-invariant). Capability-off shortcuts return
empty early so the existing strip-on-capability-off path still
holds.

* 🪆 fix: Codex P2 — recursive subagent build for multi-level delegation

Previously only the outer `agents[]` loop attached `subagentConfigs`
to its inputs, so a child used as a subagent (invoked via the
`subagent` tool) lost every explicit spawn target of its own. A
user-valid configuration like A → B → C would only run the top
layer; B could never actually delegate to C from inside A's run.

Recursively build `subagentConfigs` for each child inside
`buildSubagentConfigs`, passing the child's freshly-constructed
`childInputs` down so its own `subagents.enabled` children get
resolved too. Added cycle protection via an `ancestors` Set — a
configuration like A → B → A is safely cut off at the second
encounter of A rather than recursing forever (the existing
`child.id === agent.id` guard already prevents the direct self-loop).

* 🪆 fix: Codex P2 — reset subagent atoms on useEventHandlers unmount

The effect that resets subagent atoms only fired on `paramId` change,
so unmounting the chat container (route change away from /c) never
flushed the atoms. `knownSubagentAtomKeys` lives in a ref inside
`useStepHandler` — once the hook unmounts the ref is gone, so a
subsequent remount can't clean atoms it never registered.

Added a second `useEffect` that only runs cleanup on unmount (empty
deps aside from the stable `resetSubagentAtoms` callback). Keeps
`atomFamily` bounded across full route teardowns too.

* 🪆 fix: Codex round 13 — cyclic subagent guard + prefer persisted

Two valid findings.

**P1 — cyclic subagent ref reloads the primary.** A configuration
like `A ↔ B` (B lists A as its own subagent) would send
`loadSubagentsFor` down a path that couldn't find A in
`agentConfigs` (the primary isn't stored there), so it called
`processAgent(A)` a second time. That inserts a fresh config for
the primary id, which downstream duplicates in
`[primary, ...agentConfigs.values()]` and can replace the primary's
tool context with the reloaded copy.

Fix: short-circuit when a subagent ref points back at
`primaryConfig.id` — reuse the already-loaded primary config.
Primary is always an edge id so no pruning bookkeeping needed.

**P2 — live atom preferred over canonical persisted trace.** The
dialog picked `progress.contentParts` ahead of
`persistedContent`, but the Recoil bucket is best-effort — after
a disconnect/reconnect it can be stale or partial. The server's
`subagent_content` on the `tool_call` is the canonical record
refreshed on sync. Preferring live could hide completed
tool/reasoning history that was actually persisted.

Fix: flip the preference order. Persisted wins when it's
non-empty; live covers the mid-stream window (before the parent
message saves, persisted is empty) and the older-runs fallback.
Updated the test that enforced the old order to lock the new
semantics in (separate mid-stream live-fallback assertion kept).

* 🪆 fix: Codex P2 — subagent atom reset rule simplified to 'leaving established id'

The `hasEstablishedConversationRef` + check for initial undefined→id
covered the first navigation but missed the equivalent mid-stream
URL stamp when a user goes from an existing chat to a new chat and
sends a message there (`id → null → newId`). The null → newId
transition was still hitting the reset branch and wiping the
in-flight subagent ticker/content for that first turn.

Simpler rule: only reset when the PREVIOUS paramId is an established
id. Every transition AWAY from an established chat clears (id→id2,
id→null, id→undefined); every transition FROM null/undefined passes
through (initial mount, new-chat URL stamp mid-stream). Drop the
`hasEstablishedConversationRef` machinery in favor of that single
condition.

* 🪆 fix: Codex P2 — match runtime's strict subagent enable check in ACL

Runtime (`initializeClient` + `run.ts`) treats `subagents?.enabled`
as a truthy predicate — `undefined`, `null`, missing, and `false`
all short-circuit. The ACL gate was using `!== false` which
accepted `undefined` / missing as "enabled" and could 403 a payload
whose subagent tool would be inert at runtime.

Swap both create and update to `enabled === true`. Only a strictly-
enabled payload triggers the ACL check; the disable path (`false`)
still passes through so a user who lost VIEW on a child can still
save the disable edit.

* 🪆 fix: Codex P2 — reject missing subagent references with 400

`validateSubagentAccess` collapsed through `collectUnauthorizedAgentIds`,
which returns an empty list for ids with no DB record — so typos and
references to deleted agents passed validation silently, and
`initializeClient` later dropped them at runtime. Saved config would
then list spawn targets that the backend never honored, a hard-to-
diagnose drift.

Refactor the helper into `classifyAgentReferences(ids, …)` which
returns `{ missing, unauthorized }` separately. `validateEdgeAgentAccess`
keeps its old semantics (missing is intentional — a self-referential
`from` names the agent being created). `validateSubagentReferences`
surfaces both buckets so the create/update handlers can 400 on
missing and 403 on unauthorized with distinct error messages and
`agent_ids` lists.

* 🪆 polish: tighten subagent dialog grid gap to gap-2

OGDialogContent's grid default is `gap-4`, which renders the title,
description, and scroll area as three visually separated panels.
Drop to `gap-2` so they read as one block.

* 🪆 polish: swap Subagents above Handoffs in Advanced panel

Subagents is the more common knob users reach for, so show it first.
Handoffs keep the same Controller wiring, just move below.
2026-04-25 04:02:01 -04:00
Danny Avila
91cd3f7b7c 🧽 refactor: Skills polish: precedence-aware body validation, controller drop logs, SkillPills rename (#12760)
Post-merge sanity-review cleanup on top of #12746:

- `createSkill` / `updateSkill` now parse SKILL.md body's always-apply
  status once and reuse it for both validation and derivation (was
  parsing the same YAML block twice per call).
- Body-inline `always-apply:` validation becomes precedence-aware: a
  caller sending an explicit top-level `alwaysApply` or a structured
  `frontmatter['always-apply']` no longer gets rejected for a typo in
  the body — the body value is never consulted at derivation time when
  a higher-precedence source wins. New tests cover the three relevant
  interactions (explicit+body-typo, frontmatter+body-typo, body-only
  typo still rejects).
- OpenAI and Responses controllers now emit a `logger.warn` when
  `injectSkillPrimes` drops always-apply primes to stay under
  `MAX_PRIMED_SKILLS_PER_TURN`. `injectSkillPrimes` already logs
  internally; the controller-level warn adds endpoint context so
  operators can identify which path hit the cap at a glance. Mirrors
  AgentClient's existing log.
- Rename `ManualSkillPills` → `SkillPills` (component + type + file +
  test + all JSDoc references). The component handles both manual and
  always-apply pills now; the original name was carried over from the
  manual-only Phase 3 and misleads new readers.
- Drive-by fix: declare `appConfig = req.config` at the top of
  `createResponse` in `responses.js` — it was used unqualified on
  lines 381/396, which silently evaluated to `undefined` (via optional
  chaining) and disabled the skills-capability check on the Responses
  endpoint. Pre-existing, surfaced by lint on the touched file.
2026-04-25 04:02:01 -04:00
Danny Avila
dfc3dfa57f 📍 feat: always-apply frontmatter: auto-prime skills every turn (#12746)
* 🔁 refactor: Rebase always-apply work onto merged structured-frontmatter columns

Phase 6 (disable-model-invocation / user-invocable / allowed-tools)
landed first on feat/agent-skills. Reconcile this branch with the new
mainline:

- Thread alwaysApplySkillPrimes through unionPrimeAllowedTools alongside
  manualSkillPrimes, applying the combined MAX_PRIMED_SKILLS_PER_TURN
  ceiling before loading tools.
- Add `_id` to ResolvedAlwaysApplySkill to match Phase 6's
  ResolvedManualSkill shape (read_file name-collision protection).
- Register 'always-apply' in ALLOWED_FRONTMATTER_KEYS / FRONTMATTER_KIND
  so Phase 6's validator recognizes it.
- Drop frontmatter from the listSkillsByAccess projection; the backfill
  helper remains as defensive code but its read path is no longer
  exercised on summary rows (no legacy rows exist — the branch never
  shipped), saving ~200KB per page.
- Retire the corresponding "backfills legacy on summaries" test.
- Plumb listAlwaysApplySkills through the JS controllers + endpoint
  initializer so the always-apply resolver sees a real DB method.

* 🧹 fix: Dedupe manual/always-apply overlap, share YAML util, tidy comments

Addresses review findings:

- Cross-list dedup: when a user $-invokes a skill that is also marked
  always-apply, the always-apply copy is now dropped so the same
  SKILL.md body never primes twice in one turn. Manual wins (explicit
  intent, closer to the user message). Dedup runs in both
  initializeAgent (so persisted user-bubble pills stay in sync) and
  injectSkillPrimes (defense-in-depth at splice time). New test cases
  cover single-overlap, partial-overlap, and dedup-before-cap.
- DRY: extract stripYamlTrailingComment to
  packages/data-schemas/src/utils/yaml.ts; packages/api/src/skills/import.ts
  now imports the shared helper. Also drop the redundant inner
  stripYamlTrailingComment call inside parseBooleanScalar — the call
  site already strips.
- Mark injectManualSkillPrimes as @deprecated in favor of
  injectSkillPrimes (kept for external consumers of @librechat/api).
- Document SKILL_TRIGGER_MODEL as forward-looking plumbing for the
  model-invoked path rather than leaving it as a bare unused export.
- Replace the stale "frontmatter is included" comment on
  listSkillsByAccess with an accurate explanation of why it was
  intentionally excluded.

* 🔒 fix: Include always-apply primes in skillPrimedIdsByName + clear alwaysApply on body opt-out

Two bugs flagged by Codex review:

P1 (read_file): `manualSkillPrimedIdsByName` only carried manual-invocation
primes, so an always-apply skill with `disable-model-invocation: true`
was blocked from reading its own bundled files, and same-name collisions
could resolve to a different doc than the one whose body got primed.
- Rename `buildManualSkillPrimedIdsByName` → `buildSkillPrimedIdsByName`
  (accepts both manual + always-apply prime arrays).
- Rename the configurable field `manualSkillPrimedIdsByName` →
  `skillPrimedIdsByName` throughout the plumbing (skillConfigurable.ts,
  handlers.ts, CJS callers, tests).
- Overlap resolution: manual wins on the rare edge case where the same
  name appears in both arrays (upstream dedup should prevent this, but
  defensive merging treats manual as authoritative).
- New tests: (1) gate-relaxation fires for always-apply primes, (2) `_id`
  pinning works for always-apply same-name collisions.

P2 (updateSkill): when a body update had no `always-apply:` key,
`extractAlwaysApplyFromBody` returned `absent` and the column was left
untouched. A skill that was once `alwaysApply: true` would keep
auto-priming even after its SKILL.md no longer declared the flag.
- Treat `absent` as a positive "not always-apply" declaration when the
  body is explicitly submitted; flip the column to `false`.
- Explicit top-level `alwaysApply` still wins (three-source precedence
  unchanged).
- New tests: body removes key → false, body has no frontmatter at all →
  false, explicit + body-without-key → explicit wins.

* 🧵 refactor: Collapse duplicate prime types + tighten parse + test hygiene

Sanity-check review follow-ups:

- Collapse `ResolvedManualSkill` / `ResolvedAlwaysApplySkill` into a
  single `ResolvedSkillPrime` canonical interface with two backward-
  compatible type aliases. Both resolvers feed the same pipeline stages
  (injectSkillPrimes, unionPrimeAllowedTools, buildSkillPrimedIdsByName);
  the per-source distinction lives on `additional_kwargs.trigger`, not
  on the resolver output.
- Move the `always-apply` branch in `parseFrontmatter` to operate on the
  raw post-colon text. The outer `unquoteYaml` was fine today because
  it's idempotent on non-quoted strings, but running it twice (once per
  line, once after stripping the inline comment) would be fragile if the
  unquoter ever grows richer YAML-escape handling.
- Add the missing `alwaysApplyDedupedFromManual: 0` field to the
  `injectSkillPrimes` mocks in `openai.spec.js` and `responses.unit.spec.js`
  so they match the full `InjectSkillPrimesResult` contract.
- Insert the blank line between the `unionPrimeAllowedTools` and
  `resolveAlwaysApplySkills` describe blocks.

* 🔧 fix(tsc): Cast mock.calls via `unknown` for strict tuple destructure

`getSkillByName.mock.calls[0]` is typed as `[]` by jest's generic default;
a direct cast to `[string, ..., ...]` fails TS2352 under `--noEmit` even
though the runtime shape matches. Go through `as unknown as [...]` like
the earlier test in the same file so CI's type-check step stays green.

* 🪢 fix: Propagate skillPrimedIdsByName into handoff agent tool context

Handoff agents go through the same `initializeAgent` flow as the primary
(with `listAlwaysApplySkills` now plumbed), so they resolve their own
`manualSkillPrimes` and `alwaysApplySkillPrimes` — but the
`agentToolContexts.set(...)` for handoff agents didn't carry
`skillPrimedIdsByName` into the per-agent context.

That meant `handleReadFileCall` fell back to the full ACL set + a
`prefer*` flag for handoff agents: same-name collisions could resolve to
a different doc than the one whose body got primed, and a
`disable-model-invocation: true` skill primed via manual `$` or
always-apply inside the handoff flow would be blocked from reading its
own bundled files.

Build the map via `buildSkillPrimedIdsByName(config.manualSkillPrimes,
config.alwaysApplySkillPrimes)` for every handoff tool context so
`read_file` behaves identically across primary and handoff agents.
2026-04-25 04:02:00 -04:00
Danny Avila
82173f7b91 🛡️ feat: Persist & enforce disable-model-invocation / user-invocable / allowed-tools (#12745)
* 🧬 feat: Persist `disable-model-invocation` / `user-invocable` / `allowed-tools`

Adds first-class columns mirroring the three runtime-enforced frontmatter
fields, with a `deriveStructuredFrontmatterFields` helper that maps from
frontmatter at create/update time and re-syncs (via `$unset`) when fields
are removed. `listSkillsByAccess` projection includes them so the Phase 6
catalog filter and popover filter can both read off the summary row.

Marks `invocationMode` as @deprecated on `TSkill` and the
`InvocationMode` enum — the runtime now reads the persisted pair instead.

* 🛡️ feat: Enforce frontmatter at runtime (catalog, skill tool, manual resolver, tool union)

Wires the persisted columns into actual runtime behavior across all four
invocation paths:

- `injectSkillCatalog` excludes `disableModelInvocation: true` skills
  before catalog formatting — they cost zero context tokens and stay
  invisible to the model.
- `handleSkillToolCall` rejects with a clear error when the model names
  a skill marked `disable-model-invocation: true` (defends against a
  stale-cache or hallucinated invocation getting past the catalog
  filter).
- `resolveManualSkills` skips `userInvocable: false` skills with a warn
  log so an API-direct caller can't bypass the popover-side filter.
- `unionPrimeAllowedTools` collects skill-declared `allowed-tools` minus
  what's already on the agent; `initialize.ts` re-runs `loadTools` for
  the extras and merges resulting `toolDefinitions` into the agent's
  effective set for the turn. Tool-name resolution is tolerant —
  unknown names silently drop with a debug log so cross-ecosystem
  skills referencing yet-to-be-implemented tools (Claude Code's
  `edit_file`, etc.) import without breaking. The agent document is
  never modified; the union is turn-scoped.

Helper exports (`unionPrimeAllowedTools`) are structured so Phase 5's
always-apply primes flow through the same union (combined
`[...manualPrimes, ...alwaysApplyPrimes]`) once the resolver lands.

Skill handler wire format gains the three fields so clients can render
them on detail / list views.

* 🎛️ feat: `$` popover reads `userInvocable` instead of UI-only `invocationMode`

Replaces the phase-1 UI-only `invocationMode` check with the persisted
`userInvocable` field (mirrors the `user-invocable` frontmatter). Skills
authored with `user-invocable: false` no longer surface in the popover;
the backend resolver enforces the same rule for defense-in-depth.

Default-visible behavior is preserved: skills without an explicit
`userInvocable` value (older rows, freshly imported skills that don't
declare the field) stay visible — only an explicit `false` hides them.

Test fixture updated to reflect the new field.

* 🔧 fix: Address Phase 6 review findings

Codex P2 + reviewer #1: Single `loadTools` call with the union of
`agent.tools + allowed-tools`. The earlier two-call approach dropped
`userMCPAuthMap` / `toolContextMap` / `actionsEnabled` from the
skill-added pass — an MCP tool gained via `allowed-tools` would be
visible to the model but fail at execution without per-user auth
context. Resolution of `manualSkillPrimes` is hoisted before
`loadTools` so the union can be computed up-front; the dropped-tools
debug log now compares loaded vs. requested across the single call.

Codex P3 + reviewer #2: `injectSkillCatalog.activeSkillIds` now
includes `disable-model-invocation: true` skills. The runtime ACL
check in `handleSkillToolCall` previously couldn't reach the explicit
"cannot be invoked by the model" rejection because the broader access
set excluded those skills. Catalog text and tool registration still
gate on the visible subset (zero-context-token guarantee preserved);
only the per-user `isActive` filter is a hard exclusion now.

Reviewer #1 (try/catch around loadTools, MAJOR): A single bad
`allowed-tools` entry from a shared skill could crash the entire turn.
Now wrapped — on failure with extras, retry with just `agent.tools`
and continue (the dropped-tools debug log surfaces what vanished). If
the retry-without-extras still throws, propagate; the agent's own
tools are the load-bearing surface.

Reviewer #3 (integration tests, MAJOR): Added six tests in
`initialize.test.ts` covering the full `allowed-tools` loading path:
union pass-through, no-extras short-circuit, agent-baseline dedup,
loadTools throw + retry, propagated throw without extras, and the
empty-tools edge case.

Smaller cleanups bundled in:
- Reviewer #4: Moved `logger` import to the package-imports section
  (was wedged among local imports).
- Reviewer #5: Removed unused index on `disableModelInvocation`
  (filtering happens application-side in `injectSkillCatalog`; index
  cost write overhead for zero query benefit).
- Reviewer #6: Swapped order of `userInvocable` and body checks in
  `resolveManualSkills` so the more authoritative author-decision
  reason surfaces first when both apply.
- Reviewer #8: Documented the `allowedTools` enforcement gap on the
  schema + type — model-invoked skills (mid-turn `skill` tool calls)
  do NOT trigger tool union, since adding tools after the graph
  starts would require a rebuild. Manual / always-apply (Phase 5)
  primes are the supported paths.
- Reviewer #9: Renamed `dmi` / `ui` / `at` locals to
  `disableModelInvocationRaw` / `userInvocableRaw` / `allowedToolsRaw`
  in `deriveStructuredFrontmatterFields`.

Reviewer #7 (DRY shared `getSkillByName` return type) deferred —
field sets diverge meaningfully across the three call sites (handler
needs `body + fileCount`; resolver needs `author + allowedTools +
userInvocable`; the InitializeAgentDbMethods contract needs the
superset). A `Pick<>`-based consolidation is a follow-up cleanup.

* 🔧 fix: Address codex iter 2 — catalog quota + duplicate-name dedup

P1: `injectSkillCatalog` cap now counts only model-visible skills, not
the merged active set. The previous behavior let a tenant with many
`disable-model-invocation: true` rows near the top of the cursor
exhaust the 100-slot quota before any invocable skill got scanned —
the catalog could end up empty even though invocable skills existed
further down the paginated results. `MAX_CATALOG_PAGES` stays the
ceiling on scan budget; only `visibleCount` drives the early-exit on
quota fill.

P2: When an invocable and a `disable-model-invocation: true` skill
share a name, drop the disabled doc(s) from `activeSkillIds`. Without
this dedup, `getSkillByName` (which sorts by `updatedAt` desc) could
pick the disabled doc and every model call to the cataloged name
would fail with "cannot be invoked by the model" instead of executing
the visible skill. When ONLY a disabled doc exists for a name, it
stays in `activeSkillIds` so the explicit-rejection error path still
fires for hallucinated invocations.

Tests: 3 new cases in `injectSkillCatalog` covering (a) cap counted
on visible skills only, (b) same-name collision drops disabled doc,
(c) sole-disabled-name case keeps the disabled doc.

* 🔒 fix: Apply `disable-model-invocation` gate to read_file too (codex iter 3 P1)

`activeSkillIds` is shared between the `skill` and `read_file` handlers.
The skill-tool gate was applied last iteration, but `handleReadFileCall`
authorized purely on `getSkillByName(..., accessibleIds)` — so a model
that learned a hidden skill's name (stale catalog or hallucination)
could still read its `SKILL.md` body or bundled files via `read_file`,
defeating the contract. Same explicit rejection now fires from both
handlers; no change needed to the ACL set itself (disabled docs stay
in `activeSkillIds` so the explicit error path keeps firing).

Two new tests in `handlers.spec.ts` cover the read_file gate and
regression-protect the happy path.

* 🔧 fix: Address codex iter 4 — manual-prime exception + legacy frontmatter backfill

P1: Scope the `read_file` `disableModelInvocation` gate to AUTONOMOUS
model probes only. A user-invoked `$` skill that is also marked
`disable-model-invocation: true` had its bundled `references/*` /
`scripts/*` files unreadable, leaving the manually-primed skill body
referencing files the model couldn't load. Now the handler bypasses
the gate when the skill name appears in `manualSkillNames` (the
per-turn allowlist threaded from `manualSkillPrimes` →
`agentToolContexts` → `enrichWithSkillConfigurable` →
`mergedConfigurable`). Defense-in-depth: the bypass is scoped to the
specific names in the allowlist; a different disabled skill name is
still rejected.

P2: Read-time fallback for legacy skills authored before Phase 6
landed the structured columns. `user-invocable: false` /
`disable-model-invocation: true` set in `frontmatter` (the validator
already accepted those keys) but with no derived column would
incorrectly evaluate as "user-invocable / model-allowed" until a save
backfilled the columns. New `backfillDerivedFromFrontmatter` helper
fills undefined columns from frontmatter at read time in both
`getSkillByName` and `listSkillsByAccess` — column wins when both are
set, frontmatter fills the gap when only it's set. No DB writes; the
next `updateSkill` naturally persists. `listSkillsByAccess` projection
expanded to include `frontmatter` (bounded by validator, payload
impact small) so summaries can also be backfilled.

Sticky-primed disabled skills (ones invoked in prior turns of the
same conversation) are not yet in the manual-prime allowlist — same-
turn manual invocation is the load-bearing path codex flagged; the
sticky-turn case is a known limitation tracked for a follow-up.

Tests: 2 new in handlers.spec.ts (manual-prime allows + name-scoped
block holds), 3 new in skill.spec.ts (legacy backfill via
getSkillByName + listSkillsByAccess + column-wins precedence).

* 🔧 fix: Address codex iter 5 — propagate manualSkillNames + keep read_file

P1: `enrichWithSkillConfigurable` is also called from `openai.js` and
`responses.js` (the OpenAI Responses + completions endpoints). Both
were ignoring the new `manualSkillNames` parameter, which meant the
manual-prime exception in the `read_file` gate (iter 4) only worked
on the agents endpoint. Now all three call sites pass
`primaryConfig.manualSkillPrimes?.map(p => p.name)` so manual `$`
invocations of disabled skills work consistently across endpoints.

P2: When every accessible skill is `disable-model-invocation: true`,
the catalog text and `skill` tool are correctly omitted (no model-
reachable targets) — but `read_file` and `bash_tool` MUST still be
registered. A user manually invoking such a skill gets its SKILL.md
body primed into context; if the body references `references/foo.md`
or `scripts/run.sh`, those reads need a registered tool. Restructured
`injectSkillCatalog` so `skill` registration is gated on
`catalogVisibleSkills.length > 0` while `read_file` (always) and
`bash_tool` (when codeEnvAvailable) register whenever any active
skill is in scope.

Tests: existing all-disabled test rewritten to assert read_file IS
registered + skill is NOT; new test confirms bash_tool joins it
when codeEnvAvailable.

* 🔧 fix: Address codex iter 6 — name-collision consistency via preferInvocable

P2a (resolveManualSkills): a name collision between an older
user-invocable doc and a newer non-user-invocable doc made manual `$`
invocation silently no-op. The popover surfaced the older invocable
doc; resolver looked it up by name; `getSkillByName` returned the
newer non-invocable doc; resolver skipped on `userInvocable: false`.

P2b (handler / runtime ACL): with same-name duplicates (e.g. older
invocable + newer disabled), the manual prime resolved to one doc
while later `read_file` / `skill` execution resolved a different doc
through `activeSkillIds`. Model could follow one SKILL.md body while
reading files from a different skill.

Both root-cause: `getSkillByName` always returned the newest match
and let the caller filter, but with collisions the newest can be
something the caller didn't want.

Fix: extend `getSkillByName` with `options.preferInvocable`. When
true, prefer the newest doc satisfying BOTH `userInvocable !== false`
AND `disableModelInvocation !== true` (with frontmatter backfill);
fall back to the newest match otherwise. Fast path preserved when
caller doesn't opt in.

Callers passing `preferInvocable: true`:
- `resolveManualSkills` — picks the popover-visible invocable doc
  even when a newer disabled / non-user-invocable duplicate exists.
- `handleSkillToolCall` — keeps execution aligned with the catalog;
  falls back to the disabled doc only when no invocable variant
  exists (so the explicit "cannot be invoked by the model" gate
  still fires for the hallucinated-disabled-name case).
- `handleReadFileCall` — same alignment, plus the manual-prime
  exception added in iter 4 still applies.

Tests:
- 2 new in skill.spec.ts (preferInvocable picks invocable when
  collision exists; falls back to newest when no clean-invocable
  exists).
- 1 new in skills.test.ts (resolver passes preferInvocable through).
- 2 new in handlers.spec.ts (skill tool + read_file pass it).
- Existing initialize.test.ts assertion updated for the new option.

* 🔧 fix: Address codex iter 7 — split preferInvocable into per-axis flags

The previous unified `preferInvocable` filter required BOTH
`userInvocable !== false` AND `disableModelInvocation !== true`. That
was wrong for the model paths: `userInvocable: false` skills are
model-only and remain valid `skill` / `read_file` invocation targets.
A duplicate-name scenario where the newer cataloged doc was model-
only would let the older user-invocable variant shadow it on every
model call.

Split the option into two independent axes:
- `preferUserInvocable` — for manual paths (`$` popover). Skips docs
  with `userInvocable: false`. Disable-model-invocation status is
  irrelevant; iter 4 explicitly supports manual prime of disabled
  skills.
- `preferModelInvocable` — for model paths (`skill` / `read_file`
  handlers). Skips docs with `disableModelInvocation: true`. User-
  invocable status is irrelevant; model-only skills are valid here.

Both flags fall back to the newest match when no preferred doc
exists, so the explicit-rejection error paths still fire correctly
in the sole-disabled-name case.

Callers updated:
- `resolveManualSkills` → `preferUserInvocable: true`
- `handleSkillToolCall` / `handleReadFileCall` → `preferModelInvocable: true`

Tests:
- New spec test for preferModelInvocable not filtering on userInvocable.
- Existing preferInvocable test renamed/split to cover the new axes.
- New test asserts preferUserInvocable still returns disabled docs
  (preserves iter 4 manual-disabled support).
- Caller tests assert each path passes the right single flag and
  does NOT pass the wrong one.

* 🔧 fix: TypeScript type-check failure in handlers.spec.ts (CI green)

`jest.fn(async () => ...)` without explicit args infers an empty tuple
for the call signature, so `mock.calls[0][2]` flagged as "Tuple type
'[]' has no element at index '2'." Cast to `unknown[]` then narrow to
the expected option shape. Behavior unchanged.

Caught by the `Type check @librechat/api` CI step
(.github/workflows/backend-review.yml).

* 🔧 fix: Address codex iter 8 — undefined-result fallback + read_file alignment

P1 (loadTools returning undefined): Production loaders
(`createToolLoader` in `initialize.js` / `openai.js` /
`responses.js`) wrap `loadAgentTools` in try/catch and return
`undefined` on failure rather than throwing. Without explicit
handling, my iter-1 try/catch only fired for thrown errors — a
silent-failure on a skill-added tool would fall through to the
empty fallback and silently DROP the agent's baseline tools for
the turn (much worse than just losing the extras). Added an
`undefined`-result branch that retries with just `agent.tools`,
mirroring the throw branch. Test pins both behaviors.

P2 (read_file alignment with manual prime): When a skill is in
this turn's `manualSkillNames`, the `read_file` handler now uses
`preferUserInvocable` instead of `preferModelInvocable`. Same
name-collision rule as `resolveManualSkills`, so the doc whose
files get read is the same doc whose body got primed. For
autonomous probes (skill not in `manualSkillNames`), the handler
keeps `preferModelInvocable` to align with the catalog the model
saw. Two new tests cover both branches and regression-protect that
the wrong flag isn't passed.

* 🔧 fix: Address codex iter 9 — pin read_file lookup to primed skill _id

P1 (manually-primed disabled IDs were dropped from activeSkillIds):
The `executableSkills` dedup in `injectSkillCatalog` correctly drops
`disable-model-invocation: true` duplicates when an invocable doc
shares the name — but `resolveManualSkills` legitimately primes
disabled docs (iter 4 supports manual `$` invocation of disabled
skills). When the resolver primed a disabled doc, the read_file
handler couldn't find it in the (deduped) `activeSkillIds` and
either resolved a different same-name skill or returned not-found.

Fix: `ResolvedManualSkill` now carries `_id`; the legacy `initialize.js`
/ `openai.js` / `responses.js` controllers build a
`manualSkillPrimedIdsByName` map and `enrichWithSkillConfigurable`
passes it into `mergedConfigurable`. `handleReadFileCall` now pins
its lookup's `accessibleIds` to `[primedId]` whenever the requested
skill is in the map. The constrained set guarantees the lookup
returns the EXACT doc the resolver primed — body/files come from the
same source even when same-name duplicates exist or the dedup
removed the prime's id from `activeSkillIds`.

Autonomous read_file probes (skill not in the manual-primed map)
keep the full ACL set + `preferModelInvocable` so they align with
the catalog the model saw and the disabled-only case still fires
the explicit-rejection gate.

Test fixture changes flow from `_id` becoming required on
`ResolvedManualSkill`. `buildSkillPrimeContentParts` /
`injectManualSkillPrimes` widen their param types to `Pick<...>`
because they only read `name` / `body` and shouldn't force test
literals to invent placeholder ids.

* 🧹 fix: Address independent reviewer findings (DRY + types + tests + docs)

Sanity-pass review surfaced 7 findings; addressed 6 (the 7th — DRY
on inline `getSkillByName` return types — is acknowledged tech debt
deferred to a follow-up).

#1 [MAJOR, DRY]: The 4-line `manualSkillPrimedIdsByName` map
construction was duplicated across 4 CJS call sites (openai.js,
responses.js x2, initialize.js). Extracted `buildManualSkillPrimedIdsByName`
helper in `skillDeps.js`; all four sites now call the helper. If
`ResolvedManualSkill` ever renames `_id` or gains identifying fields,
only the helper changes.

#2 [MINOR, type safety]: `handleReadFileCall` was casting a hex string
to `Types.ObjectId[]` via `as unknown as`, relying on mongoose's
auto-cast in `$in` queries. Replaced with `new Types.ObjectId(...)`
so any future consumer comparing with `.equals()` / `===` gets the
correct value type. Imported `Types` as a value (was type-only).

#5 [MINOR, test gap]: Added a test for the worst-case silent-failure
path — both the union and base-only `loadTools` calls return undefined.
The agent gets no tools but the turn doesn't crash hard; pinning
that contract.

#4 [MINOR, performance]: Added a TODO on the `listSkillsByAccess`
projection noting the `frontmatter` field can be dropped once a
write migration backfills all pre-Phase-6 skills' columns. ~2KB/skill
× 100/page is wasted bandwidth post-backfill.

#6 [NIT, docs]: `backfillDerivedFromFrontmatter` JSDoc said "Pure"
right before "mutates its undefined fields in place". Replaced with
"Side-effect-free w.r.t. the DB (no writes), but mutates its argument
in place" which describes both halves accurately.

#7 [NIT, test determinism]: Replaced `await new Promise(r => setTimeout(r, 5))`
in two same-name collision tests with explicit `updateOne` setting
`updatedAt: new Date(Date.now() - 1000)` on the older doc. Removes
the wall-clock race on fast CI runners. The pagination test (line
480) still uses setTimeout — that test is pre-existing and order
is incidental, not load-bearing.

Existing test fixtures updated to use valid 24-char hex ObjectIds
(required by the iter-9 test that constructs a real `ObjectId`).

#3 [MINOR, deferred]: Inline `getSkillByName` return type duplicated
across `handlers.ts`, `initialize.ts`, `skills.ts`. Reviewer
acknowledged this as deferred; field sets diverge across call sites
(handler needs `fileCount`, resolver needs `author`/`allowedTools`).
A `Pick<>`-based consolidation is a clean follow-up.
2026-04-25 04:02:00 -04:00
Danny Avila
539c4c7e4d 🎬 feat: Prime Manually-Invoked Skills via $ Popover (#12709)
* 🎬 feat: Prime Manually-Invoked Skills via $ Popover

Lands the backend for manual skill invocation, making the $ popover
deterministically prime SKILL.md before the LLM turn instead of leaving
the model to discover the skill via the catalog.

Flow: popover drains pendingManualSkillsByConvoId on submit, attaches
names to the ask payload, controllers forward to initializeAgent, and
initialize resolves each name to its body (ACL + active-state filtered,
reusing the same rules as catalog injection). AgentClient splices the
primes as meta HumanMessages before the user's current message.

- Extract primeManualSkill / resolveManualSkills in packages/api/src/agents/skills.ts
  and reuse primeManualSkill inside handleSkillToolCall for a single shape source.
- Thread manualSkills + getSkillByName through InitializeAgentParams / DbMethods
  and all three initializeAgent call sites (initialize.js, responses.js, openai.js).
- Splice HumanMessage primes in client.js chatCompletion after formatAgentMessages,
  shifting indexTokenCountMap so hydrate still fills fresh positions correctly.
- Carry isMeta / source / skillName in additional_kwargs for downstream filtering.

* 🛡️ fix: Scope manual skill primes to single-agent + cap resolver input

Two follow-ups to the Phase 3 priming path flagged in Codex review.

Multi-agent runs: skipping the splice when agentConfigs is non-empty.
`initialMessages` is shared across every agent in `createRun`, so splicing
a skill body there would bypass Phase 1's per-agent `scopeSkillIds`
contract — a handoff / added-convo agent with a different skill scope
would see content its configuration excludes. Warn + skip is the minimal
correct behavior; lifting this to per-agent initial state is a follow-up.

Input bounding: `resolveManualSkills` now truncates to `MAX_MANUAL_SKILLS`
(10) after dedup, with a warn listing the dropped tail. Controllers only
validate `Array.isArray(req.body.manualSkills)`, so a crafted payload
could otherwise fan out into an unbounded `Promise.all` of concurrent
`getSkillByName` DB lookups. Cap lives in the resolver so every caller
(including future `always-apply` in Phase 5) inherits it.

* 🧪 refactor: Testable Helpers + Payload Validation for Manual Skill Primes

Follow-ups from the comprehensive review. No behavior change for the
happy path — these are architectural and defensive improvements that
shrink the JS surface in /api, tighten the request-body contract, and
cover the delicate splice logic with proper unit tests.

- Extract `injectManualSkillPrimes` into packages/api/src/agents/skills.ts
  so the message-array splice and `indexTokenCountMap` shift are unit-
  testable in TS. client.js now calls the helper. Tests pin the `>=`
  vs `>` boundary condition — a regression here would silently corrupt
  token accounting for every message after the insertion point.
- Extract `extractManualSkills(body)` and use in all three controllers
  (initialize.js, responses.js, openai.js). Replaces copy-pasted
  `Array.isArray(...) ? ... : undefined` with a helper that also filters
  non-string / empty elements — closes a type-safety gap where a crafted
  payload like `{"manualSkills": [123, {"$gt":""}]}` would otherwise reach
  `getSkillByName` and waste DB round-trips.
- Rename `primeManualSkill` → `buildSkillPrimeMessage`. The helper serves
  three invocation modes (`$` popover, `always-apply`, model-invoked);
  the old name misled readers coming from `handleSkillToolCall`.
- Add `loadable.state === 'hasValue'` guard in `drainPendingManualSkills`
  — defensive, since the atom has a synchronous `[]` default, but the
  previous `.contents` cast would have been unsound under loading/error.
- Document why `resolveManualSkills` honors the active-state filter even
  for explicit `$` selections (Phase 2 popover filter + API-direct
  hardening).
- Remove stray `void Types;` in initialize.test.ts — `Types` is already
  consumed elsewhere in that test.

* 🔖 refactor: Single source for the skill-message source marker

Export `SKILL_MESSAGE_SOURCE = 'skill'` and use it in both construction
paths that stamp skill-primed messages — `buildSkillPrimeMessage` (for
the model-invoked tool path) and `injectManualSkillPrimes` (for the
user-invoked splice path). Downstream filtering and telemetry read this
marker, so the two paths must agree; keeping the literal in one place
removes the risk of them drifting when Phase 5's `always-apply` adds a
third caller.

* ♻️ refactor: Drop Multi-Agent Guard + Review Polish

- Remove the multi-agent skip in `AgentClient.chatCompletion`. Leaking
  primes to handoff / added-convo agents via shared `initialMessages` is
  the agents SDK's concern to scope; this layer should just inject and
  let the graph handle agent-scoped state. The guard was well-intended
  but produced a silent-drop UX where `$skill` in a multi-agent run did
  nothing.
- Bound the `[resolveManualSkills] Truncating ...` warn output to the
  first 5 dropped names plus a count suffix. A malicious payload of
  1000 names was previously spilling all ~990 names into the log line.
- Remove dead `?? []` from the `hasValue`-guarded loadable read in
  `drainPendingManualSkills` — the atom always yields a string[] when
  resolved, so the nullish fallback was unreachable.
- Reorder skills.ts imports to follow the style guide: value imports
  shortest-to-longest (`data-schemas` → `langchain/core/messages` →
  multi-line `@librechat/agents`), type imports longest-to-shortest.

* 🧠 fix: Strip Skill Primes from Memory Window + Unbreak CI Mocks

Two fixes after the last push.

CI unbreak: `responses.unit.spec.js` and `openai.spec.js` mock
`@librechat/api` and the mock didn't expose the new `extractManualSkills`
symbol, so every test in those files crashed before reaching the
`recordCollectedUsage` assertion. Added `extractManualSkills: jest.fn()`
returning `undefined` to both mocks; the controllers now no-op on
manualSkills as the tests expect.

Codex P2: `runMemory` passes `messages` straight through to the memory
processor, so after the splice in `injectManualSkillPrimes`, SKILL.md
bodies ride along as if they were real user chat. That pollutes memory
extraction with synthetic instruction content and crowds out real turns
from the window.

- Export `isSkillPrimeMessage(msg)` from `packages/api/src/agents/skills.ts`
  — a predicate keyed on the shared `SKILL_MESSAGE_SOURCE` marker.
- Filter `chatMessages = messages.filter(m => !isSkillPrimeMessage(m))`
  at the top of `runMemory` before the window-sizing logic. Keeps the
  primes visible to the LLM (they still ride in `initialMessages`) but
  invisible to the memory layer.
- 5 new tests for the predicate covering marker-present, plain messages,
  different source, non-object inputs, and array filter integration.

* 📜 feat: Show Skill-Loaded Cards for Manually-Invoked Skills

The $ popover was priming SKILL.md bodies into the turn but leaving no
visible trace on the assistant response — from the user's view it looked
like the `$name ` cosmetic text did nothing. Now each manually-invoked
skill renders the same "Skill X loaded" tool-call card that model-invoked
skills already produce via PR #12684's SkillCall renderer.

Approach: post-run prepend to `this.contentParts`. The aggregator owns
per-step indices during the run, so pre-seeding collides; waiting until
`await runAgents(...)` returns lets the graph settle before synthetic
parts slot in at the front.

- Export `buildSkillPrimeContentParts(primes, { runId })` from
  `packages/api/src/agents/skills.ts`. Returns completed tool_call parts
  (`progress: 1`, args JSON-encoded with `{skillName}`, output matching
  the model-invoked path's wording) that the existing `SkillCall.tsx`
  renderer draws identically.
- In `AgentClient.chatCompletion`, prepend the built parts to
  `this.contentParts` immediately after `await runAgents`. Persistence
  and the final-event reconcile come for free — `sendCompletion` already
  reads `this.contentParts` verbatim.
- Card ordering: skills appear first in the assistant message, reflecting
  that priming ran before the LLM's turn.

Live-during-streaming cards are a separate follow-up — the graph's
index-based aggregator makes that a bigger lift and this change delivers
the core UX win without fighting the stream ordering.

6 new unit tests covering part shape, args JSON contract, output text,
unique IDs, empty input, and startOffset ID differentiation.

* ⚡ feat: Emit Optimistic Skill Cards + Wire Primes in OpenAI/Responses

Two follow-ups from testing.

Optimistic card emit: the main chat path was only showing "Skill X
loaded" cards at final-reconcile time, so the user saw nothing happen
until the stream finished. Now emit synthetic ON_RUN_STEP +
ON_RUN_STEP_COMPLETED events right before `runAgents` starts — same
pattern the MCP OAuth flow uses in `ToolService` — so the cards appear
immediately. The graph's content at index 0 may overwrite them during
streaming, but the post-run `contentParts` prepend (unchanged) restores
them on final reconcile.

OpenAI + Responses parity: both controllers were resolving
`manualSkillPrimes` via `initializeAgent` but never injecting them into
`formattedMessages` before the run. Manual invocation silently did
nothing on `/v1/chat/completions` and the Responses API path. Now both
call `injectManualSkillPrimes` on the formatted messages so the model
sees SKILL.md bodies on every path. LibreChat-style card SSE events
don't apply to these OpenAI-shaped responses, so the live-emit is
chat-path-only.

- Export `buildSkillPrimeStepEvents(primes, { runId })` from
  `packages/api/src/agents/skills.ts`. Uses `Constants.USE_PRELIM_RESPONSE_MESSAGE_ID`
  by default so the frontend maps events to the in-flight preliminary
  response message, matching the OAuth emitter.
- In `AgentClient.chatCompletion`, emit via `sendEvent` (or
  `GenerationJobManager.emitChunk` in resumable mode) after
  `injectManualSkillPrimes` runs, before the LLM turn begins.
- Wire `injectManualSkillPrimes` into `openai.js` + `responses.js` after
  `formatAgentMessages`. Refactored the destructure to `let` on
  `indexTokenCountMap` so the injector's returned map is usable.
- 8 new unit tests covering the step-event builder: pair cardinality,
  default/custom runId, TOOL_CALLS shape + JSON args, progress:1 on
  completion, index ordering, stepId/toolCallId pairing, empty input.

* 🎯 fix: Route Skill Prime Events to the Real Response + Sparse-Array Offset

Two bugs in the optimistic-card emit from the last pass.

1. Wrong runId. The events used `USE_PRELIM_RESPONSE_MESSAGE_ID` (the
   MCP OAuth pattern), but OAuth emits DURING tool loading — before the
   real response messageId exists. By the time skill priming fires, the
   graph is about to emit with `this.responseMessageId`, so the PRELIM
   runId orphaned every card onto the client's placeholder response
   entry in `messageMap`, separate from the one the LLM's events were
   building. Net effect: cards never rendered mid-stream.

   Now passing `this.responseMessageId` — the same ID `createRun`
   receives — so synthetic and real steps land on the same `messageMap`
   entry.

2. Index 0 collision. With the runId fixed, card-at-0 would have hit
   `updateContent`'s type-mismatch guard when the LLM's text delta
   arrived at the same index, suppressing the whole text stream.

   New `SKILL_PRIME_INDEX_OFFSET` = 100 placed on both the live SSE
   emit and the server-side `contentParts` assignment. Sparse array
   during streaming renders as `[llm_text, ..., card]` (skip-holes via
   `Array#filter` / `Array#map`). `filterMalformedContentParts` from
   `sendCompletion` compacts to dense `[text, card]` before persistence,
   so streaming UI and saved message agree on order — no finalize
   reorder jank. Post-run switches from `contentParts.unshift` to
   `contentParts[OFFSET + i] = part` to mirror the live placement.

- Add `startIndex` option to `buildSkillPrimeStepEvents` with
  `SKILL_PRIME_INDEX_OFFSET` default. Export the constant from
  `@librechat/api` so `client.js` can reuse it for the post-run splice.
- Update the existing index-ordering test to the new default and add a
  new test for the explicit `startIndex` override.

* 🎗️ feat: Replace \$skill-name Text with Pills on the User Message

The `$skill-name ` cosmetic text the popover was inserting into the
textarea had two problems: it lingered in the user message forever (the
card is a more meaningful marker), and it implied that free-form text
invocation like \"\$foo help me\" should work — which it doesn't, and
supporting it would mean another parsing layer nobody asked for.

Dropped the textarea insertion. Visual confirmation after submit now
comes from a compact `ManualSkillPills` row on the user bubble that
self-extinguishes once the backend's live skill-card stream
(`buildSkillPrimeStepEvents` from the last commit) populates the sibling
assistant response. Multiple skills render as multiple pills — the atom
was already a string array, so multi-select works for free.

- `SkillsCommand.tsx`: select handler no longer writes to the textarea.
  Still drops the trigger `$` via `removeCharIfLast`, still pushes to
  `pendingManualSkillsByConvoId`, still flips `ephemeralAgent.skills`.
- `families.ts`: new `attachedSkillsByMessageId` atomFamily keyed by
  user messageId. `useChatFunctions.ask` writes the drained skill list
  here on every fresh submit (regenerate/continue/edit still skip).
- `ManualSkillPills.tsx` renders pills conditionally: hidden when the
  message isn't a user message, when no skills are attached, or when
  the sibling assistant response already carries a `skill` tool_call
  content part (the live card took over). Reads messages via React Query
  so we don't re-render on every message-state keystroke.
- `Container.tsx` mounts the pills above the user message text, parallel
  to the existing `Files` slot.
- Updated the SkillsCommand select-flow spec to assert the textarea is
  cleared of `$` instead of populated with `\$name `. 5 new tests for
  `ManualSkillPills` covering empty state, non-user message guard,
  multi-skill rendering, the skill-card hide condition, and the
  text-only-content-doesn't-hide case.

* 🎛️ feat: Manual Skills as Persisted Message Field + Compose-Time Chips

Three problems with the previous pass:
1. Cards rendered BELOW the LLM text on the assistant message (and
   stayed there on reload) because the sparse index-100 offset put them
   after the model's content. Now back to `unshift` — cards at the top,
   same as before the live-emit detour.
2. Pills on the user message disappeared the moment the live card
   arrived, so users barely saw them. The live-emit channel also added
   meaningful complexity and relied on a per-message Recoil atom that
   had no clean cleanup story.
3. No visual cue at all during new-chat compose — the `$name ` text was
   removed, the submitted-message pills weren't there yet, and the
   popover closes after selection. User had no way to see what they'd
   queued up before sending.

New architecture: `manualSkills` is a first-class field on `TMessage`,
persisted by the backend on the user message. `ManualSkillPills` reads
straight from `message.manualSkills` — no atom, no sibling-lookup — so
pills survive reload, show in history, and stay for the lifetime of the
message. Compose-time chips above the textarea read the existing
`pendingManualSkillsByConvoId` atom and let users × skills out before
submitting.

Backend reverts:
- `client.js`: dropped the `ON_RUN_STEP` live-emit loop, restored
  `this.contentParts.unshift(...primeParts)` so cards sit at the top of
  the persisted assistant response.
- `skills.ts`: removed `buildSkillPrimeStepEvents` and
  `SKILL_PRIME_INDEX_OFFSET` (both unused now). `GraphEvents`,
  `StepTypes`, and `Constants` imports went with them. Removed 8 tests.

Field persistence:
- `tMessageSchema` gains `manualSkills: z.array(z.string()).optional()`.
- Mongoose message schema gains `manualSkills: { type: [String] }` with
  matching `IMessage` TS field.
- `BaseClient.js` reads `req.body.manualSkills` on user-message save,
  filters to non-empty strings, pins onto `userMessage` before
  `saveMessageToDatabase`. Mirrors the existing `files` pattern right
  above it. Runtime resolution still reads top-level `req.body.manualSkills`
  — persistence and resolution are separate concerns.

Frontend:
- `useChatFunctions.ask` sets `currentMsg.manualSkills` directly; the
  drained atom value goes onto the message, not a separate atom.
  Removed the `attachSkillsToMessage` Recoil callback.
- `ManualSkillPills`: pure render of `message.manualSkills`. No more
  `useQueryClient`, no sibling scan, no atom read. Loses the
  auto-hide-when-card-arrives behavior — pills stay on the user
  bubble, cards live on the assistant bubble, both are informative.
- Dropped the `attachedSkillsByMessageId` atomFamily and its export.
- New `PendingManualSkillsChips` above the textarea reads the
  compose-time atom and renders chips with × to remove. Mounted in
  `ChatForm` right after `TextareaHeader`. Naturally hides on submit
  when the atom drains.

Tests: updated `ManualSkillPills` suite to the new field-based reads
(5 passing). New `PendingManualSkillsChips` suite covering empty state,
multi-chip render, single × removal, and full-clear (4 passing).
Backend suite trimmed to 89 (was 97) from the step-events test
removal — no regressions on the remaining helpers.

* 🧪 feat: Assistant-Side Skill-Loading Chips + Pill Padding

Two small UX fixes on top of the field-on-message architecture.

Pill padding: bumped the user-side `ManualSkillPills` from `py-0.5` to
`py-1` on each chip and added `py-0.5` to the wrapper so the row
breathes a little without feeling tall.

Mid-stream indicator: new `InvokingSkillsIndicator` mirrors the parent
user message's `manualSkills` onto the assistant bubble as transient
"Running X" chips while the real card is in flight. Renders above
`ContentParts` in `MessageParts`. Hides itself when the assistant's
own `content` grows a `skill` tool_call — the authoritative card from
`buildSkillPrimeContentParts.unshift` is showing, so the placeholder
steps aside. No SSE emit, no aggregator injection, no index
collision with the LLM's streaming content: just a render slot keyed
off the parent's field.

Why not stream the cards live: whichever content index we'd choose
either blocks the LLM's text stream (`updateContent` type-mismatch at
index 0) or lands below the response after sparse compaction (index
100+). Mirroring the parent field sidesteps the aggregator entirely
and gives the user an immediate "skill is loading" signal that
naturally gives way to the real card at finalize.

Covers the gap the user flagged: pills on the user message said "I
asked for these" but nothing on the assistant side said "we're
working on it" until the stream finished. 5 new tests for the
indicator: user-msg guard, missing parent-field guard, multi-chip
render, hides-on-card-landing, orphan-parent guard.

* 🔁 fix: Indicator Visibility + Carry Manual Skills Through Regenerate/Edit

Two bugs.

Indicator never rendered: `InvokingSkillsIndicator` looked up the parent
user message via `queryClient.getQueryData([QueryKeys.messages, convoId])`,
but on a new chat the React Query cache is keyed by `"new"` (the URL
`paramId`) until the server assigns a real conversation ID — while
`message.conversationId` on the assistant message is already the server
ID. Lookup missed, `skills.length === 0`, nothing rendered. Switched
to `useChatContext().getMessages()`, which reads from the same
`paramId` the rest of the UI uses, so new-chat and existing-chat cases
both resolve to the correct message list.

Regenerate / save-and-submit dropped manual skills: the compose-time
`pendingManualSkillsByConvoId` atom is drained on the first submit,
so replaying that turn later found an empty atom and sent `manualSkills: []`.
The pills were still on the user bubble, so from the user's point of
view the model was running primed — but the backend saw nothing and
produced an unprimed response.

- Added `overrideManualSkills?: string[]` to `TOptions`. Callers with a
  reference message pass its persisted `manualSkills`; `useChatFunctions.ask`
  uses the override verbatim when present, otherwise falls back to the
  existing drain-or-empty logic.
- `regenerate` in `useChatFunctions` passes `parentMessage.manualSkills`
  — the user message being regenerated has the field persisted by the
  backend, so the second turn primes the same skills as the first.
- `EditMessage.resubmitMessage` covers both edit branches:
  - User-message save-and-submit: forwards the edited message's own
    `manualSkills` so the new sibling turn primes identically.
  - Assistant-response edit: forwards the parent user message's
    `manualSkills` for the same reason.

Indicator test suite converted from `@tanstack/react-query` harness to
a jest-mocked `useChatContext().getMessages()`. 6 tests (was 5), added
a cache-miss case.

* 🧭 fix: Drive Mid-Stream Skill Chips from Submission Atom, Not Message Lookup

Message-ID-keyed lookups kept racing the stream: the user message flips
from its client-side intermediate UUID to the server-assigned ID mid-run,
conversation IDs flip from the URL `paramId="new"` to the real convo
ID on brand-new chats, and the React Query cache splits briefly between
the two. Previous attempts — direct `queryClient.getQueryData` and then
`useChatContext().getMessages()` — each missed a different window.

`TSubmission.manualSkills` is already populated at `ask()` time and the
submission atom (`store.submissionByIndex(index)`) is the single stable
anchor across the whole lifecycle: set once at submit, lives through
every SSE event, cleared when the stream ends. No ID lookups, no cache
timing.

- `InvokingSkillsIndicator` now reads `submissionByIndex(index)` via
  Recoil. Shows chips when:
    • the message is assistant-side,
    • a submission is in flight with non-empty `manualSkills`,
    • the assistant's `parentMessageId` matches
      `submission.userMessage.messageId` (so chips appear only on the
      bubble for the current turn, never on siblings),
    • the assistant's own content doesn't yet carry a `skill`
      tool_call (real card takes over from the server's post-run
      `contentParts.unshift`).
- Drops the `useChatContext().getMessages()` dependency and the
  `useQueryClient` dependency before that. No more lookups by
  conversationId or messageId.

Test suite now mocks `useChatContext` to supply `index: 0` and seeds
the `submissionByIndex(0)` atom via Recoil initializer. 6 cases cover
user-side, no-submission history, empty `manualSkills`, multi-chip
render, hides-on-card-landing, and wrong-turn guard.

* 🌱 fix: Seed Response manualSkills in createdHandler, Indicator Becomes Pure

The mid-stream indicator kept getting wired off state I don't own: first
`queryClient.getQueryData` (raced the new-chat paramId flip), then
`useChatContext().getMessages()` (same cache, same race), then
`useRecoilValue(submissionByIndex)` (pulled every message into the
submission subscription — re-renders all indicators on any submission
change, exactly the "limit hooks in rendering" concern).

Cleanest path is the one the user pointed at: the submission owns the
data, `useSSE` / `useEventHandlers` owns the save points, so seed the
field ONTO the response message at the save site and let the indicator
be a pure prop-read.

- `createdHandler` now writes `manualSkills` onto the initial response
  from `submission.manualSkills` at the moment the placeholder enters
  the messages array. The field rides through the normal mutation
  pipeline via spreads (`useStepHandler` response creation,
  `updateContent` result returns) — no special handling needed.
- `InvokingSkillsIndicator` drops the Recoil / context / queryClient
  reads. Pure function of `message`: if assistant, has `manualSkills`,
  and `content` hasn't grown a `skill` tool_call yet, render chips.
  Only `useLocalize` left, which was already unavoidable for the i18n
  string.
- Renders decouple: no single state change (`submissionByIndex` flip,
  React Query cache update) forces every indicator in the message list
  to re-render anymore. Only the message whose prop changed re-runs.

Finalize story unchanged: server's `responseMessage` doesn't carry the
frontend-only `manualSkills` field, so `finalHandler`'s replacement
drops it — but by then the real `skill` tool_call is in `content`
and the indicator's content-scan hides itself anyway.

Test suite back to pure prop mocks: 7 cases covering user-guard,
no-seed, multi-chip render, skill-card-hide, non-skill-tool-call-keeps,
text-only-keeps, and missing message.

* 🪞 fix: Render Skill Indicator Inside ContentParts, Adjacent to Parts

The indicator still wasn't showing because even though MessageParts
mounted it as a sibling of ContentParts, ContentParts is a `memo`'d
component that owns the only rendering path that refreshes in lockstep
with content deltas. Mounting above it put the indicator one layer
further out — reachable, but not exercised on the same render cycle
that processes the streaming `message` prop.

Moved the indicator into ContentParts itself, rendered at the top of
both the sequential and parallel branches. Reads the `message` prop
(newly threaded through as an optional prop alongside `content`), so:

- Same render cycle as Parts — updates from the SSE pipeline flow
  through the same pathway.
- Lives outside the `content.map`, so delta-driven content reshuffles
  never wipe it.
- Still a pure prop-read inside the indicator itself (no Recoil,
  queryClient, context hooks). The only dep is `useLocalize`.

Thread:
- `ContentPartsProps` gains `message?: TMessage`.
- `MessageParts` passes `message={message}` through, drops its own
  indicator mount + import.
- `ContentParts` renders `<InvokingSkillsIndicator message={message} />`
  in both the parallel-content and sequential-content branches, right
  under `MemoryArtifacts` and before the empty-cursor / parts map.

Companion data flow (unchanged): `createdHandler` seeds
`initialResponse.manualSkills` from `submission.manualSkills`; the
field rides through `useStepHandler` via spreads; indicator hides on
`skill` tool_call landing in `content`.

* 🔎 refactor: Narrow Skill Components to Scalar skills Prop, Kill Memo Churn

Passing the full `message` object into presentational components busts
`React.memo` shallow comparisons every time the message reference changes
for unrelated reasons. Swap to scalar `skills?: string[]` throughout:

- `InvokingSkillsIndicator`: props-only (`skills?: string[]`); visibility
  logic (user-vs-assistant, skill tool_call arrival) now lives in the
  caller so this stays pure presentational.
- `ManualSkillPills`: props-only (`skills?: string[]`).
- `ContentParts`: takes `manualSkills?: string[]` scalar, computes
  `showInvokingSkills` once per render from `manualSkills` + content scan
  for the `skill` tool_call, then mounts the indicator with `skills=`
  prop in both parallel and sequential branches.
- `MessageParts`: passes `manualSkills={message.manualSkills}` through
  to `ContentParts`.
- `Container`: passes `skills={message.manualSkills}` to `ManualSkillPills`.
- Tests updated to exercise the narrowed prop surface.

* 📜 feat: Mid-Stream Skill Cards via SkillCall, Drop Custom Indicator

Instead of a separate `InvokingSkillsIndicator` chip component, render
pending skill placeholders through the existing `SkillCall` renderer —
same component the backend's finalized prime part uses. The loading
visual (`progress < 1` + empty output → pulsing "Running X") and the
completed visual ("Ran X") now come from one source of truth.

`ContentParts` computes `pendingSkillNames` from `manualSkills` minus
any `skill` tool_call already in `content` (dedupe by `args.skillName`
since the synthetic's id differs from the real one). Those names
render through a separate slot ABOVE the Parts iteration — not
prepended to the content array, which would shift React keys on
every downstream streaming text / tool part and force unmount/remount
mid-stream.

When the real prime `tool_call` lands at finalize (backend unshifts to
content[0..]), `collectExistingSkillNames` picks it up, the pending
set empties, and the real part takes over rendering in the Parts
iteration. Layout is identical either way because primes are always
at the top of content.

- `InvokingSkillsIndicator.tsx` + test deleted (no longer referenced)
- `ContentParts.tsx` renders `<SkillCall .../>` directly for pending
  names, mirrors `Part.tsx`'s usage of the same component
- `createdHandler` doc comment updated to reflect the new flow

* ✂️ fix: Render Interim Skill Cards From manualSkills Only, Leave Content Untouched

Previous revision read `content` to de-dupe pending cards against real
`skill` tool_calls, so any optimistic skill part streamed from the
backend would race our placeholder off the screen mid-turn — exactly
the "getting overridden" symptom.

Now: interim `SkillCall` cards are driven purely by the response
message's `manualSkills` field. `content` is never inspected here,
so no backend delta can pull the cards down. The field is now seeded
directly onto the assistant placeholder in `useChatFunctions` (not
only in `createdHandler`) so the cards appear from the first render,
before the `created` SSE event round-trip.

Lifecycle:
- `useChatFunctions` puts `manualSkills` on the freshly-minted
  `initialResponse` — cards render the instant the placeholder lands.
- `createdHandler` keeps its own re-seed (idempotent; safe) so a
  regenerate / save-and-submit flow that hits that path still works.
- `useStepHandler` spread operations preserve the field through every
  content update.
- `finalHandler` replaces the message with the server-backed
  `responseMessage` (no `manualSkills`) — cards disappear, and the
  real `skill` tool_call part in `content` takes over.

ContentParts changes:
- Drop `collectExistingSkillNames` / `parseJsonField` dedupe path.
- `renderPendingSkills` reads only `manualSkills` + `isCreatedByUser`.
- Simpler control flow — one boolean (`hasPendingSkills`) gates the
  early return, one function renders.

* 🩹 fix: Codex Review Resolutions — Localization, Guards, Tests, Docs

Addresses seven findings from comprehensive code review:

Finding 1 (MAJOR) — Document sticky re-priming as intentional
- `buildSkillPrimeContentParts`: expanded doc comment explaining
  synthetic `skill` tool_calls persist and get re-primed on every
  subsequent turn via `extractInvokedSkillsFromPayload` (shape parity
  with model-invoked skills). This matches the UX: the assistant
  skill card is a visible, persistent signal that the skill is active
  for the conversation. Not a bug — called out explicitly so future
  maintainers don't mistake it for one.

Finding 2 (MAJOR) — Add ContentParts render tests
- New `ContentParts.test.tsx` with 7 cases covering the interim skill
  card logic: assistant-only rendering, user-message suppression,
  undefined-content safety, parallel+sequential branch integration,
  progress<1 (pending) state. Child components mocked so the test
  exercises only the branching and prop wiring ContentParts owns.

Finding 3 (MINOR) — Localize hardcoded aria-labels
- Added `com_ui_skills_manual_invoked` + `com_ui_skills_queued` keys.
- Reused existing `com_ui_remove_skill_var` for the remove-button
  aria-label.
- `PendingManualSkillsChips` and `ManualSkillPills` now call
  `useLocalize()`. Test mocks updated to the label-echo pattern.

Finding 4 (MINOR) — Max-length guard in `extractManualSkills`
- New `MAX_SKILL_NAME_LENGTH = 200` constant and filter. Blocks a
  crafted payload like `{ manualSkills: ['a'.repeat(100000)] }` from
  reaching `getSkillByName` / Mongo's query planner.

Finding 5 (NIT) — `BaseClient.js` comment contradicted itself
- Rewrote to call the filter what it is: defense-in-depth on top of
  Mongoose schema validation, not a redundant second layer.

Finding 6 (NIT) — `ManualSkillPills` now wrapped in `React.memo`
- Consistent with peer components (`PendingManualSkillsChips`,
  `ContentParts`). Rendered inside `Container`, which re-renders on
  every content update, so the memo is a real cycle savings.

Finding 7 (NIT) — Redundant guard in `ContentParts.renderPendingSkills`
- Collapsed the duplicate null-check by computing `pendingSkills` as
  a `useMemo`'d array (`[]` when not applicable), and mapping
  directly. `hasPendingSkills` now derives from the array length —
  one source of truth, no redundant gate inside the render function.

* 🔧 fix: Update ParallelContent to Handle Optional Content Prop

Modified the `ParallelContentRendererProps` to make the `content` prop optional, ensuring safer access within the component. Adjusted the calculation of `lastContentIdx` to handle cases where `content` may be undefined, preventing potential runtime errors. This change enhances the robustness of the component when dealing with varying message structures.

* 🎯 fix: Thread manualSkills Through ContentRender — The Real Renderer

This is why the interim skill cards never appeared across many rounds of
iteration: `ContentRender.tsx` (the memo'd renderer used by most paths,
including the agents endpoint) was calling `ContentParts` without the
`manualSkills` prop. Only `MessageParts.tsx` had it wired up — and
that's not the component that actually renders the assistant response
in production.

Two fixes:
1. Pass `manualSkills={msg.manualSkills}` to the `ContentParts` call.
2. Extend the `areContentRenderPropsEqual` memo comparator to include
   `manualSkills.length`, otherwise a message update that adds the
   field (seeded by `useChatFunctions` on the initialResponse) would
   be bailed out by the memo and never re-render.

Verified the two ContentParts call sites are now consistent; Container
usages for `ManualSkillPills` on the user side were already correct.

* 🧹 polish: Address Audit Follow-Up (F1/F3/F6)

F1 — Clarify sticky re-priming opt-out path.
  The previous comment said "regenerate without the pick" as one
  opt-out, but `useChatFunctions.regenerate` forwards the original
  picks via `overrideManualSkills`, so regeneration alone keeps the
  skill sticky. Updated to: edit the originating message to remove
  the pills and resubmit, or start a new conversation.

F3 — Add DOM-order assertions to the parallel + sequential tests.
  The two "alongside" tests verified both elements existed but
  didn't pin the ordering contract. Both now use
  `compareDocumentPosition` to assert the pending SkillCall
  precedes the real content, matching the backend semantic
  (`contentParts.unshift(...primeParts)` puts primes at the top).

F6 — Fix package import order in PendingManualSkillsChips.
  `recoil` (58 chars) was listed before `lucide-react` (45 chars)
  which violates the "shortest to longest after react" rule in
  AGENTS.md. Swapped order; no behavior change.

F2 / F4 / F5 from the audit were confirmed as non-issues
(React-safe empty map, cosmetic test-mock artifact, accepted
memo tradeoff) and require no change.

* ✨ feat: Dedicated PendingSkillCall + Running→Ran Transition on Real Content

UX polish on the interim skill card now that it's actually rendering:

1. New `PendingSkillCall` component (mirrors `SkillCall` visually but
   drops the expand affordance). `SkillCall`'s underlying `ProgressText`
   always renders a chevron + clickable button when any input is
   present, which on a card with empty output points at nothing —
   misleading cursor:pointer and a no-op toggle. The pending variant
   has only the icon + label, no button wrapper, no chevron.

2. "Running X" → "Ran X" transition when real content lands.
   `ContentParts` computes `hasRealContent` (any non-text part, or a
   text part with non-empty content — placeholder empty-text parts
   don't count) and passes `loaded={hasRealContent}` to
   `PendingSkillCall`. Matches what users see for model-invoked skills
   as they finish priming: pulsing shimmer → static icon.

3. Cleanup:
   - Dropped direct `SkillCall` import from `ContentParts` (replaced
     by `PendingSkillCall`). `SkillCall` is still used by `Part` for
     real `skill` tool_call content parts — no behavior change there.
   - Removed the now-redundant explicit `manualSkills` assignment
     in `createdHandler`. `useChatFunctions` seeds the field on
     `initialResponse` at construction, so the `...submission.initialResponse`
     spread already carries it through — the re-assignment was
     defensive belt-and-suspenders doing the same work twice. Comment
     rewritten to describe the actual lifecycle.

Tests updated to the new component (12/12 pass): two new cases pin
the loaded-state transition (unloaded when content has no real parts,
flips to loaded once a non-empty text part lands).
2026-04-25 04:02:00 -04:00
Danny Avila
3f1bde4831 🪆 feat: Compose Agent Scope and Active-State Filters in $ Popover (#12708)
* feat: compose per-agent scope and per-user active-state filters in $ popover

Stack two runtime-truth filters on top of the existing `isUserInvocable`
check so the `$` skill popover matches what will actually be available at
turn time:

- Per-agent skill scope from `agent.skills` (resolved via useChatContext
  + useAgentsMapContext), mirroring backend `scopeSkillIds` semantics:
  `undefined`/`null` → no scope, `[]` → empty, non-empty → intersection.
- Per-user ownership-aware active state via `useSkillActiveState().isActive`.

The filters are composed in a pure, exported `filterSkillsForPopover`
helper so the agent-scope ∩ active ∩ invocable matrix can be unit-tested
without rendering the component. Short-circuits on cheapest check first
(agent scope → active → invocation mode).

Backend still enforces both filters at runtime; this PR is a UX mirror so
users do not see popover entries that would be filtered out by the time
the LLM turn begins.

* refactor: thread agentId into SkillsCommand as a prop

Drop the direct `useChatContext()` call inside SkillsCommand in favor of
receiving `agentId` from ChatForm. The parent already subscribes to the
conversation via its single useChatContext call, and SkillsCommand is
wrapped in React.memo — threading the id as a prop means the popover only
re-renders when agent_id actually changes instead of on every unrelated
conversation-shape mutation. Mirrors the pattern AttachFileChat already
uses for `conversation` / `agentId`.

No behavior change; filter semantics and test coverage are identical.

* fix: fail closed when agent skill scope cannot be resolved

Previously, if `conversation.agent_id` was set but the agents map had no
entry for it (hydration pending, query failure, or missing VIEW access),
the popover treated the scope as `undefined` and showed the full ACL
catalog. That leaks options the backend will reject at turn time, the
opposite of what this phase is meant to do.

Distinguish the unresolved cases ("map undefined" and "agent not in map")
from the intentionally-unconfigured case ("agent exists, no `skills`
field") and return `[]` for the former, preserving the backend semantics
of `scopeSkillIds` only for the latter. Adds two tests covering both
fail-closed branches.

* fix: surface agent.skills in list projection and treat ephemeral ids as unscoped

Two holes flagged on the earlier fail-closed commit:

1. The list-agents projection in `getListAgentsByAccess` omitted the
   `skills` field, so `agentsMap[agentId].skills` was always undefined
   and the popover fell back to the full ACL catalog for every scoped
   agent — the opposite of this phase's intent. Add `skills: 1` to the
   projection and lock it in with a new test.

2. Conversations can carry ephemeral agent ids (e.g. `ephemeral` after
   switching off the agents endpoint) that intentionally don't live in
   the agents map. The prior fail-closed branch blanked the popover in
   those cases. Treat anything that doesn't start with `agent_` as
   unscoped via the existing `isEphemeralAgent` helper so the popover
   shows the full ACL-visible catalog, matching how no-agent convos
   already behave.

Frontend: 18 tests pass (adds one ephemeral-id case).
Backend: 10 getListAgentsByAccess tests pass (adds one projection case).

* refactor: pass through hydration race, only fail closed when map is authoritative

Split the two "cannot resolve scope" cases that were previously collapsed
onto the same fail-closed branch:

- `agentsMap === undefined` means the agents list query has not settled
  yet. Return `undefined` (full catalog). The map typically hydrates well
  before the first `$` open, and the backend still scopes at turn time —
  blanking the popover during a sub-second race produces worse UX with
  no security benefit.
- `agentsMap` is populated but the agent is absent means the agent was
  deleted or the user's VIEW access was revoked mid-session. That is
  authoritative missing state, so keep the fail-closed behavior — the
  full catalog would be misleading.

Updates the associated test case to assert the hydration-race path now
shows the catalog, and rewrites the in-code comment to distinguish the
two branches.

* refactor: drop `as string` in agent scope memo by tightening the guard

`isEphemeralAgent` returns true for null/undefined so the original code
was runtime-safe, but its signature returns plain `boolean` rather than
a type predicate, so TypeScript never narrowed `agentId` to `string` and
the subsequent map lookup required an `as string` assertion. Split the
guard into `!agentId || isEphemeralAgent(agentId)` so the narrowing falls
out naturally and the assertion can be removed.

No behavior change; 18 tests pass.
2026-04-25 04:02:00 -04:00
Danny Avila
9225a279eb 🎚️ feat: Per-User Skill Active/Inactive Toggle with Ownership-Aware Defaults (#12692)
* feat: per-user skill active/inactive toggle with ownership-aware defaults

- Add `skillStates` map (Record<string, boolean>) to user schema for
  per-user active/inactive overrides on skills
- Add `defaultActiveOnShare` to interface.skills config (default: false)
  so admins can control whether shared skills auto-activate
- Add GET/POST /api/user/settings/skills/active endpoints with validation
- Add React Query hooks with optimistic mutations for skill states
- Add useSkillActiveState hook with ownership-aware resolution:
  owned skills default active, shared skills default inactive
- Add toggle switch UI to SkillListItem and SkillDetail components
- Filter inactive skills in injectSkillCatalog before agent injection
- Add localization keys for active/inactive labels

* fix: use Record instead of Map for IUser.skillStates

Mongoose .lean() flattens Map to a plain object, causing type
incompatibility with IUser in methods that return lean documents.

* fix: address review findings for skill active states

- Fail-closed when userId is absent: filter rejects all shared skills
  instead of passing them through unfiltered (Codex P1)
- Validate Mongoose Map key characters (reject . and $) in controller
  to return 400 instead of a 500 from schema validation (Codex P2)
- Block toggle while initial skill states query is loading to prevent
  overwriting server-side overrides with an empty snapshot (Codex P2)
- Extract shared SkillToggle component, eliminating duplicate toggle
  markup in SkillListItem and SkillDetail (Finding #3)
- Move skill state query/mutation hooks from Favorites.ts to
  Skills/queries.ts per feature-directory convention (Finding #4)
- Fix hardcoded English aria-label in SkillListItem by passing the
  localized string from the parent SkillList (Finding #5)
- Fix inline arrow in SkillList render loop: pass stable callback
  reference so SkillListItem memo() is not invalidated (Finding #1)
- Extract toRecord() helper in controller to DRY the Map-to-Object
  conversion (Finding #6)
- Remove Promise.resolve wrapping synchronous config read (Finding #8)
- Remove unused TUpdateSkillStatesRequest type (Finding #12)

* fix: forward tabIndex on SkillToggle to preserve list keyboard nav

The original inline toggle had tabIndex={-1} so the row itself
remained the sole tab target. The extraction into SkillToggle
dropped this prop, making every list toggle a tab stop. Add an
optional tabIndex prop and pass -1 from SkillListItem.

* fix: plumb skillStates to all agent entry points, isolate toggle keydown

- Add skillStates/defaultActiveOnShare loading to openai.js and
  responses.js controllers so shared-skill activation is respected
  across all agent entry points, not just initialize.js (Codex P1)
- Stop keydown propagation on SkillToggle so Enter/Space does not
  bubble to the parent row's navigation handler (Codex P2)

* fix: paginate catalog fetch and serialize toggle writes

- Paginate listSkillsByAccess (up to 10 pages of 100) until the active
  catalog quota is filled, so inactive shared skills in recent positions
  do not starve active owned skills past the first page (Codex P1)
- Extend listSkillsByAccess interface with cursor/has_more/after for
  catalog pagination
- Serialize skill-state writes via a ref queue: one in-flight request
  at a time, with the latest desired state sent when the previous one
  settles. Prevents last-response-wins races where an older request
  overwrites newer toggles (Codex P2)

* fix: share write queue across hook instances, block toggle on fetch error

- Move the write queue from a per-instance useRef to a module-scoped
  object so every mount of useSkillActiveState (SkillList, SkillDetail,
  etc.) serializes against the same in-flight slot. Prior per-instance
  queues allowed two components to race full-map POSTs (Codex P1)
- Extend the toggle guard beyond isLoading: also block when isError is
  true or data is undefined. Prevents a failed GET from seeding a
  toggle with an empty baseline that would wipe server-side overrides
  on the next successful POST (Codex P1)

* fix: stale closure, orphan cleanup, and cap-error UX

- Read toggle baseline from React Query cache via queryClient.getQueryData
  instead of the captured skillStates closure. The closure can be stale
  between onMutate's setQueryData and the next render, so rapid successive
  toggles would build on old state and drop earlier changes (Codex P1)
- Surface the MAX_SKILL_STATES_EXCEEDED error code with a specific toast
  key (com_ui_skill_states_limit) so users understand the 200-cap rather
  than seeing a generic error
- Prune orphaned entries (skillIds whose Skill doc no longer exists) on
  both GET and POST in SkillStatesController. Self-heals over time
  without needing cascade-delete hooks or a migration job. Uses one
  indexed Skill._id query per request

* test: pin skill active-state precedence with unit tests

Extract the active-state resolution logic from a closure inside
injectSkillCatalog into an exported resolveSkillActive helper, then
cover every branch of the precedence matrix:

- Fails closed when userId is absent (even with defaultActiveOnShare=true)
- Explicit override wins over ownership and config (both true and false)
- Owned skills default to active when no override is set
- Shared skills default to defaultActiveOnShare value
- Undefined skillStates behaves identically to an empty object
- defaultActiveOnShare defaults to false when omitted
- Owned skills ignore defaultActiveOnShare entirely

Closes Finding #2 from the pre-rebase comprehensive review. Mirrors
the existing scopeSkillIds test style; injectSkillCatalog now calls
resolveSkillActive instead of inlining the closure.

* refactor: limit skill active toggle to detail header, drop label

- Remove the per-row toggle from SkillListItem and the active-state
  plumbing (hook call, isSkillEnabled/onToggleEnabled/toggleAriaLabel
  props) from SkillList. The detail view is now the single place to
  change a skill's active state
- Drop dim/muted styling for inactive skills in the sidebar: without
  a control there, the visual indication has nowhere to land
- Resize SkillToggle to match neighbor buttons: outer h-9 container,
  h-6 w-11 track with size-5 knob, no label span. The 'Active' /
  'Inactive' text that accompanied the detail-view toggle is removed
- Remove the now-unused label prop and tabIndex prop (the tabIndex
  existed only for the list-row context) from SkillToggle. Drop the
  onKeyDown stopPropagation for the same reason
- Remove now-orphaned com_ui_skill_active / com_ui_skill_inactive
  translation keys

* style: shrink SkillToggle track to h-5 w-9 with size-4 knob

Container stays at h-9 to match neighbor button heights. The toggle
track itself drops from h-6 w-11 to h-5 w-9, with a size-4 knob
travelling 1.125rem on activation. Visually lighter inside the row.

* fix: remove redundant skillStates entries that match the resolved default

When a toggle lands on the ownership/config default, delete the key
from the map instead of persisting `{id: defaultValue}`. Without this,
a user toggling a skill off and back on would leave `{id: true}` for
an owned skill (whose default is already true), silently consuming a
slot against the 200-entry cap. Repeated round-trip toggles could
exhaust the quota with zero meaningful overrides (Codex P2).

Preserves the exceptions-list invariant that the runtime-resolution
design depends on.

* fix: prune before enforcing skill-state cap; reject non-ObjectId keys

Reorder the update controller so pruneOrphans runs before the 200-cap
check. Without this, a user near the cap with some orphaned entries
(skills deleted since their last GET) could send a payload that would
pass after pruning but gets rejected by the raw-size check first.

Add a sanity cap on raw payload size (2 * MAX_SKILL_STATES) so abusive
inputs do not reach the DB query, and enforce the real cap on the
pruned result instead.

Harden pruneOrphans: the earlier early-return path could pass
non-ObjectId keys through unchanged. Now only valid ObjectIds are
returned, and the Skill-model-unavailable fallback filters by format.

Also add isValidObjectIdString validation at the input boundary so
malformed (but otherwise non-Mongo-unsafe) keys never reach persistence
(Codex P2 x2).

* fix: enforce active filter at execute time, prune revoked shares, scope queue per user

P1: injectSkillCatalog now returns activeSkillIds (the filtered set
that appears in the catalog). initializeAgent uses that set as the
stored accessibleSkillIds on the initialized agent, so getSkillByName
at runtime cannot resolve a deactivated skill — even if the LLM
hallucinates a name or the user invokes by direct-invocation shorthand.
Previously the executor authorized against the full ACL set, bypassing
the active-state guarantee (Codex P1).

P2: pruneOrphans now checks user access via findAccessibleResources
in addition to skill existence. When a share is revoked, the user's
skillStates entry for that skill had no cleanup path and silently
consumed the 200-cap. Self-heals on both GET and POST. One extra ACL
query per settings read/write; scoped to a single user so no N-user
amplification (Codex P2).

P2: the write queue moves from a single module-scoped object to a Map
keyed by userId. Logout/login in the same tab can no longer flush the
previous user's pending snapshot under the new session's auth. Each
userId gets its own pending/inFlight slot; the in-flight request
retains its original auth via the cookie already attached when sent,
so the race window closes (Codex P2).

* refactor: extract skillStates helpers to packages/api; add tests; polish

Address the remaining valid findings from the comprehensive review:

- Extract toRecord, loadSkillStates, validateSkillStatesPayload, and
  pruneOrphanSkillStates into packages/api/src/skills/skillStates.ts
  as TypeScript. The controller in /api shrinks to a ~90-line thin
  wrapper that builds live dependency adapters for Mongoose + the
  permission service (Review #2 DRY, #3 workspace boundary)

- Replace the triplicated 12-line skillStates loading block in
  initialize.js, openai.js, and responses.js with a single call to
  loadSkillStates from @librechat/api. One helper, three sites

- Swap console.error for the project logger in the controller
  (Review #7)

- Remove the redundant INVALID_KEY_PATTERN regex: a valid ObjectId
  cannot contain . or $, so isValidObjectIdString already covers it
  (Review #11)

- Parameterize the 200-cap error toast with {{0}} interpolation
  driven by the error response's `limit` field, so future changes to
  MAX_SKILL_STATES update the UI message automatically (Review #12)

- Add 24 unit tests for the new skillStates helpers (toRecord,
  resolveDefaultActiveOnShare, loadSkillStates, validateSkillStates-
  Payload, pruneOrphanSkillStates) covering success paths, malformed
  input, cap boundaries, and parallel-query behavior (Review #4)

- Add 10 tests for injectSkillCatalog pagination covering empty
  accessible set, missing listSkillsByAccess, single-page filter,
  owned-vs-shared defaults, explicit-override precedence, multi-page
  collection, MAX_CATALOG_PAGES safety cap, early termination on
  has_more=false, additional_instructions injection, and fail-closed
  without userId (Review #5)

Total test count: 60 (was 26 on this surface).

* fix: rename skillStates ValidationError to avoid barrel-export collision

packages/api/src/types/error.ts already exports a ValidationError
(MongooseError extension). Re-exporting a different shape from
skills/skillStates.ts through the skills barrel caused TS2308 in CI
because the root index re-exports both. Rename to
SkillStatesValidationError to keep the exports disjoint.

* refactor: tighten tests and absorb caller guard into loadSkillStates

Address the followup review findings:

- Add optional `accessibleSkillIds` param to loadSkillStates so the
  helper short-circuits to defaults when no skills are accessible.
  All three controllers drop the residual 7-line conditional wrapper
  in favor of a single destructured call (Review #2)

- Remove the unreachable `typeof key !== 'string'` check from
  validateSkillStatesPayload: Object.entries always yields string
  keys per the JS spec (Review #3)

- Replace the two `as unknown as` agent casts in the injectSkillCatalog
  tests with a `makeAgent()` factory typed directly as the function's
  parameter shape (Review #4)

- Tighten the MAX_CATALOG_PAGES assertion from `toBeLessThanOrEqual(11)`
  to `toHaveBeenCalledTimes(10)` — the loop deterministically makes
  exactly 10 page fetches before hitting the cap (Review #1)

- Rewrite the parallel-execution test for pruneOrphanSkillStates using
  deferred promises instead of microtask-order assertions. The test
  now inspects `toHaveBeenCalledTimes(1)` on both mocks after a single
  Promise.resolve() yield, pinning Promise.all usage without relying
  on push-order into a shared array (Review #5)

- Evict stale writeQueue entries on user change via a module-scoped
  `lastSeenUserId` sentinel. When a different user's toggle is the
  first one after a logout/login, the previous user's queue entry is
  deleted. Keeps the Map bounded without adding hook-instance effect
  cleanup (Review #6)

* fix(test): mock loadSkillStates in openai and responses controller specs

The prior refactor replaced the inline 12-line skillStates loading
block with a call to loadSkillStates from @librechat/api. Both
controller spec files mock @librechat/api as a flat object, so any
new named import from that package is undefined in the test env.
Calling `await loadSkillStates(...)` threw before recordCollectedUsage
ran, surfacing as "undefined is not iterable" on the test's array
destructure of `mockRecordCollectedUsage.mock.calls[0]`.

Add the missing mock to both spec files alongside the existing
scopeSkillIds stub.

* fix: abandon stale skillStates write queues on user switch

Close the cross-session leak window where an in-flight flush loop
still holds a reference to a previous user's queue: it could fire its
next mutateAsync under the new session's auth cookies and persist
the stale snapshot to the new user's document (Codex P1).

Add an `abandoned` flag on `WriteQueue`. Three mechanisms cooperate:

- `getWriteQueue` marks every non-active queue abandoned when the
  user differs from the last-seen identity (pre-existing eviction
  site, now more aggressive).
- A `useEffect` on `userId` calls the same abandonment pass on every
  render with a new active identity, covering the window between
  logout/login and the new user's first toggle (when `getWriteQueue`
  would otherwise not fire).
- The flush loop checks `!queue.abandoned` in its while condition so
  the second and later iterations exit without firing another
  `mutateAsync` after the session changes.

The first iteration's in-flight request (already dispatched under the
original user's cookies) still runs to completion or failure on its
own — only the subsequent iterations, which are the dangerous ones,
are blocked.
2026-04-25 04:02:00 -04:00
Danny Avila
3e064c2f2b 🎯 feat: Per-Agent Skill Selection in Builder and Runtime Scoping (#12689)
* feat: per-agent skill selection in builder and runtime scoping

Wire skills persistence on the Agent model and enable the skills
section in the agents builder panel. At runtime, scope the skill
catalog to only the skills configured on each agent (intersected
with user ACL). When no skills are configured, the full user catalog
is used as the default. The ephemeral chat toggle overrides per-agent
scoping to provide the full catalog.

* fix: add scopeSkillIds to @librechat/api mock in responses unit test

The test mocks @librechat/api but was missing the newly imported
scopeSkillIds, causing createResponse to throw before reaching the
assertions. Added a passthrough mock that returns the input array.

* fix: scope primeInvokedSkills by agent's configured skills

primeInvokedSkills was receiving the full unscoped accessibleSkillIds,
bypassing the per-agent skill scoping applied to initializeAgent. This
allowed previously invoked skills from message history to be resolved
and primed even when excluded from the agent's configured skill set.

Apply the same scopeSkillIds filtering to match the initializeAgent
calls, so skill resolution is consistent across catalog injection
and history priming.

* fix: preserve agent skills through form reset and union prime scope

Two related bugs in the per-agent skill selection flow:

1. resetAgentForm dropped the persisted skills array because the generic
   fall-through at the end of the loop excludes object/array values.
   Combined with composeAgentUpdatePayload always emitting skills, this
   caused any save of a previously-configured agent to silently overwrite
   skills with an empty array. Add an explicit case for skills mirroring
   the agent_ids handling.

2. primeInvokedSkills processes the full conversation payload, including
   prior handoff-agent invocations. Scoping it to only primaryAgent.skills
   meant a skill invoked by a handoff agent in a prior turn could not be
   resolved when the current primary agent had a different scope, leaving
   message history reconstruction incomplete. Union the per-agent scoped
   accessibleSkillIds across primary plus all loaded handoff agents so
   any skill any active agent could invoke is resolvable from history.

* fix: mark inline skill removals as dirty

The inline X button on the skills list called setValue without
shouldDirty: true, so removing a skill via this control did not
mark the skills field as dirty in react-hook-form state. When a
user removed a skill with the X button and also staged an avatar
upload in the same save, isAvatarUploadOnlyDirty returned true and
onSubmit short-circuited to avatar-only upload, silently dropping
the PATCH that would persist the skill removal.

The dialog path (SkillSelectDialog) already passes shouldDirty: true
on add/remove; this aligns the inline control with that behavior.

* fix: restore full ACL scope for primeInvokedSkills history reconstruction

Reverting the earlier scoping of primeInvokedSkills to the active-agent
union. That change conflated runtime invocation scoping (which correctly
gates what the model can call now) with history reconstruction (which
restores bodies the model already saw in prior turns).

Per-agent scoping still applies at:
- Catalog injection (injectSkillCatalog via initializeAgent)
- Runtime invocation (handleSkillToolCall via enrichWithSkillConfigurable,
  using each agent's scoped accessibleSkillIds in agentToolContexts)

History priming is a read of past context, not a grant of new capability.
Scoping it causes historical skill bodies to vanish from formatAgentMessages
when an agent's skills list is edited mid-conversation or when the ephemeral
toggle flips, which breaks message reconstruction and drops code-env file
continuity for /mnt/data/{skillName}/ references. The user's ACL-accessible
set is the correct and sufficient gate for history reconstruction.

* fix: close openai.js skill gap and pin undefined vs [] semantics

Three related gaps surfaced in review:

1. api/server/controllers/agents/openai.js was a third skill resolution
   site alongside responses.js and initialize.js, but still used the old
   activation gate (required ephemeralAgent.skills === true) and never
   passed accessibleSkillIds through scopeSkillIds. Per-agent scoping
   silently did not apply on this route. Mirror the same pattern used
   in responses.js so all three routes behave identically.

2. scopeSkillIds previously collapsed undefined and [] into the same
   "full catalog" fallback, making it impossible for a user to express
   "this agent has no skills." Tighten the semantics before any data
   is written under the old behavior:
     - undefined / null = not configured, full catalog
     - []              = explicitly none, returns []
     - non-empty       = intersection with ACL-accessible set
   Update defaultAgentFormValues.skills from [] to undefined so a brand
   new agent whose skills UI was never touched does not accidentally
   persist "explicit none" on first save (removeNullishValues strips
   undefined from the payload server side).

3. Add direct unit tests for scopeSkillIds covering all five cases
   (undefined, null, empty, disjoint, overlap, exact match, empty
   accessible set). 16 tests total in skills.test.ts pass.

* fix: add scopeSkillIds to @librechat/api mock in openai unit test

Same pattern as the earlier responses.unit.spec.js fix: the test mocks
@librechat/api with an explicit object, so each newly imported symbol
must be added to the mock. Without scopeSkillIds, OpenAIChatCompletion
controller throws on destructuring before reaching recordCollectedUsage,
causing the token usage assertions to fail.
2026-04-25 04:02:00 -04:00
Danny Avila
64ec5f18b8 ⚙️ feat: Skill runtime integration: catalog, tools, execution, file priming (#12649)
* feat: Skill runtime integration — catalog injection, tool registration, execute handler

Wires the @librechat/agents SkillTool primitive into LibreChat's agent runtime:

**Enums:**
- Add `skills` to AgentCapabilities + defaultAgentCapabilities

**Data layer:**
- Add `getSkillByName(name, accessibleIds)` — compound query that
  combines name lookup + ACL check in one findOne

**Agent initialization (packages/api/src/agents/initialize.ts):**
- Accept `accessibleSkillIds` param and `listSkillsByAccess` db method
- Query accessible skills, format catalog via `formatSkillCatalog()`,
  append to `additional_instructions` (appears in agent system prompt)
- Register `SkillToolDefinition` + `createSkillTool()` when catalog
  is non-empty (tool appears in model's tool list)
- Store `accessibleSkillIds` and `skillCount` on InitializedAgent

**Execute handler (packages/api/src/agents/handlers.ts):**
- Add `getSkillByName` to `ToolExecuteOptions`
- `handleSkillToolCall()` intercepts `Constants.SKILL_TOOL`:
  extracts skillName, loads body from DB with ACL check,
  substitutes $ARGUMENTS, returns ToolExecuteResult with
  injectedMessages (skill body as isMeta user message)

**Caller wiring:**
- initialize.js: query skill IDs via findAccessibleResources,
  pass to initializeAgent + store on agentToolContexts,
  add getSkillByName to toolExecuteOptions,
  pass accessibleSkillIds through loadTools configurable
- openai.js + responses.js: same pattern for their flows

Requires @librechat/agents >= 3.1.65 (PR #91 exports).

* feat: Skills toggle in tools menu + backend capability gating

Frontend:
- Add skills?: boolean to TEphemeralAgent type
- Add LAST_SKILLS_TOGGLE_ to LocalStorageKeys for persistence
- Add skillsEnabled to useAgentCapabilities hook
- Add skills useToolToggle to BadgeRowContext with localStorage init
- New Skills.tsx badge component (Scroll icon, cyan theme,
  permission-gated via PermissionTypes.SKILLS)
- Add skills entry to ToolsDropdown with toggle + pin
- Render Skills badge in BadgeRow ephemeral section

Backend:
- Extract injectSkillCatalog() into packages/api/src/agents/skills.ts
  (reduces initializeAgent module size, reusable helper)
- initializeAgent delegates to helper instead of inline block
- Capability-gate the findAccessibleResources query:
  - Agents endpoint: checks AgentCapabilities.skills in admin config
  - OpenAI/Responses controllers: checks ephemeralAgent.skills toggle
- ACL query runs once per run, result shared across all agents

* refactor: remove createSkillTool() instance from injectSkillCatalog

SkillTool is event-driven only. The tool definition in toolDefinitions
is sufficient for the LLM to see the tool schema. No tool instance is
needed since the host handler intercepts via ON_TOOL_EXECUTE before
tool.invoke() is ever called.

Removes tools from InjectSkillCatalogParams/Result, drops the
createSkillTool import.

* feat: skill file priming, bash tool, and invoked skills state

Multi-file skill support:
- New primeSkillFiles() helper (packages/api/src/agents/skillFiles.ts)
  uploads skill files + SKILL.md body to code execution environment
- handleSkillToolCall primes files on invocation when skill.fileCount > 0,
  returns session info as artifact so ToolNode stores the session
- Skill-primed files available to subsequent bash/code tool calls

Bash tool auto-registration:
- BashExecutionToolDefinition added alongside SkillToolDefinition when
  skills are enabled, giving the model a bash tool for running scripts

Conversation state:
- Add invokedSkillIds field to conversation schema (Mongoose + Zod)
- handleSkillToolCall updates conversation with $addToSet on success
- Enables re-priming skill files on subsequent runs (future)

Dependency wiring:
- Pass listSkillFiles, getStrategyFunctions, uploadCodeEnvFile,
  updateConversation through ToolExecuteOptions
- Pass req and codeApiKey through mergedConfigurable
- All three controller entry points wired (initialize.js, openai.js,
  responses.js)

* fix: load bash_tool instance in loadToolsForExecution, remove file listing

- Add createBashExecutionTool to loadToolsForExecution alongside PTC/ToolSearch
  pattern: loads CODE_API_KEY, creates bash tool instance on demand
- Add BASH_TOOL and SKILL_TOOL to specialToolNames set so they don't go
  through the generic loadTools path (bash is created here, skill is
  intercepted in handler before tool.invoke)
- Remove file name listing from skill content text — it's the skill
  author's responsibility to disclose files in SKILL.md, not the framework

* feat: batch upload for skill files, replace sequential uploads

- Add batchUploadCodeEnvFiles() to crud.js: single POST to /upload/batch
  with all files in one multipart request, returns shared session_id
- Rewrite primeSkillFiles to collect all streams (SKILL.md + bundled files)
  then do one batch upload instead of N sequential uploads
- Replace uploadCodeEnvFile with batchUploadCodeEnvFiles across all callers
  (handlers.ts, initialize.js, openai.js, responses.js)

* refactor: remove invokedSkillIds from conversation schema

Skills aren't re-loaded between runs, so conversation-level state for
invoked skills doesn't help. Skill state will live on messages instead
(like tool_search discoveredTools and summaries), enabling in-place
re-injection on follow-up runs.

Removes invokedSkillIds from: convo Mongoose schema, IConversation
interface, Zod schema, ToolExecuteOptions.updateConversation, and
all three caller wiring points.

* feat: smart skill file re-priming with session freshness checking

Schema:
- Add codeEnvIdentifier field to ISkillFile (type + Mongoose schema)
- Add updateSkillFileCodeEnvIds batch method (uses tenantSafeBulkWrite)
- Export checkIfActive from Code/process.js

Extraction:
- Add extractInvokedSkillsFromHistory() to run.ts — scans message
  history for AIMessage tool_calls where name === 'skill', extracts
  skillName args. Follows same pattern as extractDiscoveredToolsFromHistory.

Smart re-priming in primeSkillFiles:
- Before batch uploading, checks if existing codeEnvIdentifiers are
  still active via getSessionInfo + checkIfActive (23h threshold)
- If session is still active, returns cached references (zero uploads)
- If stale or missing, batch-uploads everything and persists new
  identifiers on SkillFile documents (fire-and-forget)
- Single session check covers all files (batch shares one session_id)

Wiring:
- Pass getSessionInfo, checkIfActive, updateSkillFileCodeEnvIds
  through ToolExecuteOptions and all three controller entry points

* feat: wire skill file re-priming at run start via initialSessions

Flow:
1. initialize.js creates primeInvokedSkills callback with all deps
2. client.js calls it with message history before createRun
3. extractInvokedSkillsFromHistory scans for skill tool calls
4. For each invoked skill with files, primeSkillFiles uploads/checks
5. Returns initialSessions map passed to createRun
6. createRun passes initialSessions to Run.create (via RunConfig)
7. Run constructor seeds Graph.sessions, making skill files available
   to subsequent bash/code tool calls via ToolNode session injection

Requires @librechat/agents with initialSessions on RunConfig (PR #94).

* refactor: use CODE_EXECUTION_TOOLS set for code tool checks

Import CODE_EXECUTION_TOOLS from @librechat/agents and replace inline
constant checks in handlers.ts and callbacks.js. Fixes missing bash
tool coverage in the session context injection (handlers.ts) and code
output processing (callbacks.js).

* refactor: move primeInvokedSkills to packages/api, add skill body re-injection

Moves primeInvokedSkills from an inline closure in initialize.js (with
dynamic requires) to a proper exported function in packages/api
skillFiles.ts with explicit typed dependencies.

Key changes:
- primeInvokedSkills now returns both initialSessions (for file priming)
  AND injectedMessages (skill bodies for context continuity)
- createRun accepts invokedSkillMessages and appends skill bodies to
  systemContent so the model retains skill instructions across runs
- initialize.js calls the packaged function with all deps passed explicitly
- client.js passes both initialSessions and injectedMessages to createRun

* fix: move dynamic requires to top-level module imports

Move primeInvokedSkills, getStrategyFunctions, batchUploadCodeEnvFiles,
getSessionInfo, and checkIfActive from inline requires to top-level
module requires where they belong.

* refactor: skill body reconstruction via formatAgentMessages, not systemContent

Replaces the lazy systemContent approach with proper message-level
reconstruction:

SDK (formatAgentMessages):
- New invokedSkillBodies param (Map<string, string>)
- Reconstructs HumanMessages after skill ToolMessages at the correct
  position in the message sequence, matching where ToolNode originally
  injected them

LibreChat:
- extractInvokedSkillsFromPayload replaces extractInvokedSkillsFromHistory
  (works with raw TPayload before formatAgentMessages, not BaseMessage[])
- primeInvokedSkills now takes payload instead of messages, returns
  skillBodies Map instead of injectedMessages
- client.js calls primeInvokedSkills BEFORE formatAgentMessages, passes
  skillBodies through as the 4th param
- Removed invokedSkillMessages from createRun (no more systemContent hack)
- Single-pass: skill detection happens inside formatAgentMessages' existing
  tool_call processing loop, zero extra message iterations

* refactor: rename skillBodies to skills for consistency with SDK param

* refactor: move auth loading into primeInvokedSkills, pass loadAuthValues as dep

The payload/accessibleSkillIds guard and CODE_API_KEY loading now live
inside primeInvokedSkills (packages/api) rather than in the CJS caller.
initialize.js passes loadAuthValues as a dependency and the callback
is only created when skillsCapabilityEnabled.

* feat: ReadFile tool + conditional bash registration + skill path namespacing

ReadFile tool (read_file):
- General-purpose file reader, event-driven (ON_TOOL_EXECUTE)
- Schema: { file_path: string } — "{skillName}/{path}" convention
- handleReadFileCall: resolves skill name from path, ACL check, reads
  from DB cache or storage, binary detection, size limits (256KB),
  lazy caching (512KB), line numbers in output
- SKILL.md special case: reads skill.body directly
- Dispatched alongside SKILL_TOOL in createToolExecuteHandler
- Added to specialToolNames in ToolService

Conditional tool registration:
- ReadFile + SkillTool: always registered when skills enabled
- BashTool: only registered when codeEnvAvailable === true
- codeEnvAvailable passed through InitializeAgentParams from caller

Skill file path namespacing:
- primeSkillFiles now uploads as "{skillName}/SKILL.md" and
  "{skillName}/{relativePath}" instead of flat names
- Prevents file collisions when multiple skills are invoked

Wiring:
- getSkillFileByPath + updateSkillFileContent passed through
  ToolExecuteOptions in all three callers

* feat: return images/PDFs as artifacts from read_file, tighten caching

Binary artifact support:
- Images (png, jpeg, gif, webp) returned as base64 in artifact.content
  with type: 'image_url', processed by existing callback attachment flow
- PDFs returned as base64 artifact similarly
- Binary size limit: 10MB (MAX_BINARY_BYTES)
- Other binary files still return metadata + bash fallback

Caching:
- Text cached only on first read (file.content == null check)
- Binary flag cached only on first detection (file.isBinary == null)
- Skill files are immutable; no redundant cache writes

Registration:
- ReadFileToolDefinition now includes responseFormat: 'content_and_artifact'

* chore: update @librechat/agents to version 3.1.66-dev.0 and add peer dependencies in package-lock.json and package.json files

* fix: resolve review findings #1,#2,#4,#5,#6,#10,#13

Critical:
- #1: primeInvokedSkills now accumulates files across all skills into
  one session entry instead of overwriting. Parallel processing via
  Promise.allSettled.
- #2: codeEnvAvailable now computed and passed in openai.js and
  responses.js (was missing, bash tool never registered in those flows)

Major:
- #4: relativePath in updateSkillFileCodeEnvIds now strips the
  {skillName}/ prefix to match SkillFile documents. SKILL.md filter
  uses endsWith instead of exact match.
- #5: File priming guarded on apiKey being non-empty (skip when not
  configured instead of failing with auth error)
- #6: Skills processed in parallel via Promise.allSettled instead of
  sequential for-of loop

Minor:
- #10: Use top-level imports in initialize.js instead of inline requires
- #13: Log warning when skill catalog reaches the 100-skill limit

* fix: resolve followup review findings N1,N2,N4

N1 (CRITICAL): Wire skill deps into responses.js non-streaming path.
Was completely missing getSkillByName, file strategy functions, etc.

N2 (MAJOR): Single batch upload for ALL skills' files. Resolves skills
in parallel (Phase 1), then collects all file streams across skills
and does ONE batchUploadCodeEnvFiles call (Phase 2). All files share
one session_id, eliminating cross-session isolation issues.

N4 (MINOR): Move inline require() to top-level in openai.js and
responses.js, consistent with initialize.js.

* fix: add mocks for new file strategy imports in controller tests

* fix: restore session freshness check, parallelize file lookups, add warnings

R1: Re-add session freshness check before batch upload. Checks any
existing codeEnvIdentifier via getSessionInfo + checkIfActive. If the
session is still active (23h window), returns cached file references
with zero re-uploads.

R2: listSkillFiles calls parallelized via Promise.all (were sequential
in the for-of loop).

R3: Log warning when skill record lookup fails during identifier
persistence (was a silent empty-string fallback).

* fix: guard freshness cache on single-session consistency

* fix: multi-session freshness check (code env handles mixed sessions natively)

The code execution environment fetches each file by its own
{session_id, fileId} pair independently — no single-session
requirement. Removed the sessionIds.size === 1 guard.

Now checks ALL distinct sessions for freshness. If every session
is still active (23h window), returns cached references with per-file
session_ids preserved. If any session expired, falls through to
re-upload everything in a single batch.

* perf: parallelize session freshness checks via Promise.all

* fix: add optional chaining for session info retrieval in primeInvokedSkills

Updated the primeInvokedSkills function to use optional chaining for getSessionInfo and checkIfActive methods, ensuring safer access and preventing potential runtime errors when these methods are undefined.

* fix: address review findings #1-#9 + Codex P1/P2 + session probe

Critical:
- #1/Codex P1: Add codeApiKey loading to openai.js and responses.js
  loadTools configurable (was missing, file priming broken in 2/3 paths)
- Codex P1: Fix cached file name prefix in primeSkillFiles cache path
  (was sf.relativePath, now ${skill.name}/${sf.relativePath})

Major:
- Codex P2: Honor ephemeral skills toggle in agents endpoint
  (check ephemeralAgent?.skills !== false alongside admin capability)
- #4: Early size check using file.bytes from DB before streaming
  (prevents full-file buffer for oversized files)

Minor:
- #5: Replace Record<string, any> with Record<string, boolean | string>
- #6: Localize Pin/Unpin aria-labels with com_ui_pin/com_ui_unpin
- #8: Parallelize stream acquisition in primeSkillFiles via
  Promise.allSettled
- #9: Log warning for partial batch upload failures with filenames

Performance:
- Session probe optimization: getSessionInfo now hits per-object
  endpoint (GET /sessions/{sid}/objects/{fid}) instead of listing
  entire session (GET /files/{sid}?detail=summary). O(1) stat vs
  O(N) list + linear scan.

* refactor: extract shared skill wiring helper + add unit tests

DRY (#3):
- New skillDeps.js exports getSkillToolDeps() with all 9 skill-related
  deps (getSkillByName, listSkillFiles, getStrategyFunctions, etc.)
- Replaces 5 identical copy-paste blocks across initialize.js, openai.js,
  responses.js (streaming + non-streaming paths)
- One place to maintain when skill deps change

Tests (#2):
- 8 unit tests for extractInvokedSkillsFromPayload covering:
  string args, object args, missing skill tool_calls, non-assistant
  messages, malformed JSON, empty skillName, empty payload, dedup

* fix: remove @jest/globals import, use global jest env

* fix: resolve round 2 review findings R2-1 through R2-7

R2-1 (toggle semantics): openai.js + responses.js now check admin
  capability (AgentCapabilities.skills) alongside ephemeral toggle.
  Aligns with initialize.js.

R2-2 (swallowed error): primeInvokedSkills now logs
  updateSkillFileCodeEnvIds failures (was .catch(() => {}))

R2-4 (test cast): Record<string, string> → Record<string, unknown>

R2-5 (DRY regression): Extract enrichWithSkillConfigurable() into
  skillDeps.js. Replaces 4 identical loadAuthValues blocks.
  Each loadTools callback is now a one-liner. JSDoc added (R2-6).

R2-7 (sequential streams): primeInvokedSkills now uses
  Promise.allSettled for parallel stream acquisition.

* fix: require explicit skills toggle + treat partial cache as miss

- initialize.js: change ephemeralSkillsToggle !== false to === true
  (unset toggle no longer enables skills)
- primeSkillFiles cache: require ALL files to have codeEnvIdentifier
  before using cache (partial persistence = cache miss = re-upload)
- primeInvokedSkills cache: same check (allFilesWithIds.length must
  equal total file count)

* fix: pass entity_id=skillId on batch upload, eliminates per-user cache thrashing

primeSkillFiles now passes entity_id: skill._id.toString() to
batchUploadCodeEnvFiles. This scopes the code env session to the
skill, not the user. All users sharing a skill share the same
uploaded files — no more cache thrashing from overwriting each
other's codeEnvIdentifier.

The stored codeEnvIdentifier now includes ?entity_id= suffix so
freshness checks pass the entity_id through to the per-object
stat endpoint. Both primeSkillFiles and primeInvokedSkills
store consistent identifier formats.

* fix: pass entity_id on multi-skill batch upload, consistent identifier format

* Revert "fix: pass entity_id on multi-skill batch upload, consistent identifier format"

This reverts commit c85ce2161e.

* refactor: per-skill upload in primeInvokedSkills, eliminate multi-skill batch

Replace the monolithic multi-skill batch upload with per-skill
primeSkillFiles calls. Each skill gets its own session with
entity_id=skillId, ensuring:

- Correct session auth (entity_id matches on freshness checks)
- Per-skill freshness caching (only expired skills re-upload)
- Shared skill sessions work across users (same entity_id=skillId)
- Code env handles mixed session_ids natively

The big batch block (stream collection, single upload, identifier
mapping) is replaced by a simple loop over primeSkillFiles, which
already handles freshness caching, batch upload, and identifier
persistence per-skill.

* fix: resolve review findings #1,#3-5,#7,#9-11

Critical:
- #1: Strip ?entity_id= query string before splitting codeEnvIdentifier
  into session_id/fileId (was corrupting cached file IDs in 4 locations)

Major:
- #4: Parallelize per-skill primeSkillFiles via Promise.allSettled
- #5: Add logger.warn to all empty .catch(() => {}) on cache writes

Minor:
- #7: Add logger.debug to enrichWithSkillConfigurable catch block
- #9: Use error instanceof Error guard in batchUploadCodeEnvFiles
- #10: Move enrichWithSkillConfigurable to TypeScript in packages/api
  (skillConfigurable.ts), skillDeps.js wraps with loadAuthValues dep
- #11: Reduce MAX_BINARY_BYTES from 10MB to 5MB (~11.5MB peak with b64)

* fix: forward entity_id in session probe + always register bash tool

Codex P2 (entity_id in probe): getSessionInfo now preserves and
forwards query params (including entity_id) to the per-object stat
endpoint. Without this, identifiers stored as ...?entity_id=... would
fail auth checks because the entity_id scope was dropped.

Codex P2 (bash tool availability): Remove codeEnvAvailable gate from
injectSkillCatalog. Bash tool definition is now always registered when
skills are enabled. Actual tool instance creation still happens at
execution time in loadToolsForExecution (which loads per-user
credentials). This ensures users with per-user CODE_API_KEY get
bash without requiring a global env var at init time.

Removes codeEnvAvailable from InjectSkillCatalogParams,
InitializeAgentParams, and all three controller entry points.

* fix: add debug logging to primeInvokedSkills catch, rename export alias

* fix: stub bash tool when no key + remove PDF artifact path

Codex P1 (bash tool): When CODE_API_KEY is unavailable, create a stub
tool that returns "Code execution is not available. Use read_file
instead." This prevents "tool not found" errors from the model
repeatedly calling bash_tool in no-code-env deployments while still
registering the definition for per-user credential users.

Codex P2 (PDF artifacts): Remove PDF image_url artifact path. The
host artifact pipeline processes image_url via saveBase64Image which
fails for PDFs. PDFs now fall through to the generic binary handler
("Use bash to process"). TODO comment for future document artifact
support.

Also: isImageOrPdf → isImage in early size checks (PDFs are no
longer treated as artifact candidates).

* fix: remove dead PDF_MIME constant, hoist skillToolDeps, document session_id

- #7: Remove unused PDF_MIME constant (dead code after PDF artifact removal)
- #11: Hoist skillToolDeps to module-level constant (avoid per-call allocation)
- #6: Document that CodeSessionContext.session_id is a representative value;
  ToolNode uses per-file session_id from the files array

* fix: call toolEndCallback for skill/read_file artifacts + clear codeEnvIdentifier on re-upload

Codex P1 (toolEndCallback bypass): skill and read_file handler branches
returned early, bypassing the toolEndCallback that processes artifacts
(image attachments). Now calls toolEndCallback when the result has an
artifact, using the same metadata pattern as the normal tool.invoke path.

Codex P1 (stale identifiers): upsertSkillFile now $unset's
codeEnvIdentifier alongside content and isBinary when a file is
re-uploaded. Prevents the freshness cache from returning references
to old file content after a skill file is replaced.

* fix: add session_id comment at cached path, rename skillResult to handlerResult

* fix: return content_and_artifact from bash stub so result.content is populated

* fix: deterministic skill lookup, dedup warning, and multi-session freshness check

- getSkillByName: add sort({updatedAt:-1}) so name collisions resolve
  deterministically to the most recently updated skill
- injectSkillCatalog: warn when multiple accessible skills share a name
- primeSkillFiles: check ALL distinct sessions for freshness, not just
  the first file's session, preventing stale refs after partial bulkWrite

* refactor: update icon import in Skills component

- Replaced the Scroll icon with ScrollText in the Skills component for improved clarity and consistency in the UI.

* fix: SKILL.md cache parity, gate bash_tool on code env, fix read_file too-large message

- primeSkillFiles: filter SKILL.md from returned files array on fresh
  upload so cached and non-cached paths return identical file sets
  (SKILL.md is still on disk in the session for bash access)
- injectSkillCatalog: only register bash_tool when codeEnvAvailable is
  true; thread the flag from all three CJS callers via execute_code
  capability check
- handleReadFileCall: tell the model to invoke the skill first before
  suggesting /mnt/data paths for oversized files

* fix: use EnvVar constant, deduplicate auth lookup, validate batch upload, stream byte limit

- Replace hardcoded 'LIBRECHAT_CODE_API_KEY' with EnvVar.CODE_API_KEY
  in skillConfigurable.ts and skillFiles.ts
- Resolve code API key once at run start in initialize.js and pass to
  both primeInvokedSkills and enrichWithSkillConfigurable via optional
  preResolvedCodeApiKey param, eliminating redundant loadAuthValues calls
- Add response structure validation in batchUploadCodeEnvFiles before
  accessing session_id/files to surface unexpected responses early
- Add streaming byte counter in handleReadFileCall that aborts and
  destroys the stream when accumulated bytes exceed MAX_BINARY_BYTES,
  preventing full file buffering when DB metadata is inaccurate

* refactor: update icon import in ToolsDropdown component

- Replaced the Scroll icon with ScrollText in the ToolsDropdown component for improved clarity and consistency in the UI.

* fix: partial upload failure detection, EnvVar in initialize.js, declaration ordering

- primeSkillFiles: return null (failure) when batch upload partially
  succeeds — missing bundled files would cause runtime bash/read
  failures with missing paths in code env
- initialize.js: replace hardcoded 'LIBRECHAT_CODE_API_KEY' with
  EnvVar.CODE_API_KEY imported from @librechat/agents
- initialize.js: move enabledCapabilities, accessibleSkillIds, and
  codeApiKey declarations before the toolExecuteOptions closure that
  references them (eliminates reliance on temporal dead zone hoisting)
2026-04-25 04:02:00 -04:00
Marco Beretta
f6ee2ea0ee 📜 feat: Skills UI + Initial E2E CRUD / Sharing (#12580)
* 🎨 feat: Skills UI — Create/Edit/Share/List with Conditional File Tree

First-pass UI on top of the CRUD API scaffolding (#12613). Ships the full
user-facing flow for inline, single-SKILL.md skills and leaves a clean
drop-in for phase-2 multi-file support.

- Create a skill from /skills/new with name (kebab-case, validated),
  description, and SKILL.md body — wired to the real `useCreateSkillMutation`
  and `TCreateSkill` payload.
- List skills in a sidebar (SkillsSidePanel) via `useListSkillsQuery` with
  live search filtering.
- Edit any skill the caller has EDIT permission on — `useUpdateSkillMutation`
  passes `expectedVersion` for optimistic concurrency and surfaces 409
  conflicts as a warning toast + cache refetch.
- Non-blocking `TSkillWarning[]` (e.g. "description too short") are shown
  inline above the form after a successful create/patch.
- Read-only mode when the current user lacks EDIT — the form still renders
  but inputs are marked `readOnly` and the save/reset buttons are hidden.
- Share via ACL using the existing `GenericGrantAccessDialog` — the
  `ShareSkill` button is gated on the SHARE permission.
- Delete with confirmation, driven by `useDeleteSkillMutation({ id })`.
- Conditional file tree: only rendered when `useListSkillFilesQuery`
  returns > 0 files. The tree groups flat `relativePath` strings into a
  nested view (no `react-arborist` dependency) and supports per-file
  deletion via `useDeleteSkillFileMutation`. Upload is intentionally
  deferred — the backend stubs it at 501 in phase 1.

- New routes: `/skills`, `/skills/new`, `/skills/:skillId`.
- Sidebar accordion (`SkillsAccordion` wrapping `SkillsSidePanel`) added
  to `useSideNavLinks` gated on `PermissionTypes.SKILLS` USE.

The initial UI branch (#12580) shipped a lot of exploration code on top of
a now-superseded placeholder backend. Kept as complementary: the `Skills/`
component tree, translation keys, role descriptions, `PublicSharingToggle`
SKILL mapping, `resources.ts` SKILL config, `useCanSharePublic` SKILL
mapping, and `data-provider/roles.ts` `useUpdateSkillPermissionsMutation`.

Deferred out of this first pass:
- Skill favorites (`useSkillFavorites`, `getSkillFavorites` endpoint) —
  the backend route doesn't exist yet; saving for a follow-up.
- AgentConfig `SkillSelectDialog` integration — the UI branch had this
  gated behind `false &&`; rolled back with the config.
- `InvocationMode` / `CategorySelector` / `parseSkillMd` / tree-node
  mutations — not in the Anthropic skill spec and not in the CRUD API.
- `react-arborist` dependency — replaced with a hand-rolled recursive
  tree built from flat `TSkillFile[]`.

- 38 data-schemas skill model tests: pass
- 25 api skill route tests: pass
- 16 user-controller cleanup tests: pass

* 🔐 feat: Default-On Skills in Interface Config and Role Seeder

The skills accordion was registered in the side nav gated on
`PermissionTypes.SKILLS` USE, but no one was actually seeding that
permission on startup, so a fresh install had the USER role with
zero skill permissions and the accordion never rendered.

Fixes three gaps:

1. `interfaceSchema` in data-provider's `config.ts` had no `skills`
   field at all. Added it alongside the existing agents/prompts shape
   (boolean | { use, create, share, public }) and a default of
   `{ use: true, create: true, share: false, public: false }`.

2. `loadDefaultInterface` in data-schemas passed every interface key
   through to the loaded config EXCEPT `skills`. Added the one-line
   passthrough so `appConfig.interfaceConfig.skills` is actually
   populated on boot.

3. `updateInterfacePermissions` in packages/api/src/app/permissions.ts
   seeds role permissions from the interface config on every restart.
   Added:
   - `SKILLS` case to `hasExplicitConfig`
   - `skillsDefaultUse/Create/Share/Public` extraction (mirrors
     prompts/agents)
   - `PermissionTypes.SKILLS` block in `allPermissions` that falls
     through config → roleDefaults → schema default, same pattern as
     AGENTS and PROMPTS
   - `SKILLS` entry in the share-backfill array so that pre-existing
     SKILL role docs missing SHARE/SHARE_PUBLIC get them filled on the
     next restart

Test expectations updated: seven `expectedPermissionsFor(User|Admin)`
blocks in `permissions.spec.ts` now include SKILLS, matching the
role-default values (USER: use+create true, share/public false;
ADMIN: all true).

Result: on a fresh install, a regular USER gets skill USE/CREATE
and the "Skills" accordion shows up in the chat side panel without
any yaml config. Admins can lock it down per role or per tenant via
`interface.skills` in librechat.yaml.

Tests:
- 34 packages/api permissions.spec.ts: pass
- 151 packages/api app tests: pass
- 38 data-schemas skill.spec.ts: pass
- 928 data-provider tests: pass
- 25 api skills.test.js: pass

* ♻️ fix: Resolve Skills UI Review Findings

Addresses the 13 findings from the PR review against the prior commit.

1. **canEdit consistency** — extracted `useSkillPermissions(skill)` as the
   single source of truth for owner/admin/ACL gating. `SkillsView`,
   `SkillForm`, `ShareSkill` all consume it; `SkillFileTree`'s per-file
   delete button now honors admin + EDIT-bit permissions instead of just
   ownership. Unit tests cover owner, admin, editor-ACL, viewer-ACL,
   owner-ACL, loading, and undefined-skill cases.

2. **Disabled submit buttons** — create/edit form submit buttons now set
   native `disabled` (not just `aria-disabled`) during `isLoading`.
   `onSubmit` also guards with an early return when the mutation is still
   in-flight so a duplicate enter-key submit can't create two skills.

3. **Wrong maxLength error message** — description/name `maxLength` rules
   no longer re-use `com_ui_skill_*_required`. Added dedicated
   `com_ui_skill_name_too_long` and `com_ui_skill_description_too_long`
   keys with the literal limit interpolated (`{{0}}`).

4. **Search debouncing** — `SkillsSidePanel` now threads the filter input
   through the existing `useDebounce` hook (250ms) so typing "skills" no
   longer fires six separate list queries.

5. **Frontend test coverage** — added:
   - `tree.test.ts` (9 tests) covering `buildTree` / `nodeKey` edge cases:
     empty input, single root file, multiple roots, nested folders,
     deeply-nested trees, lexicographic sort, empty paths, stable keys
   - `useSkillPermissions.test.ts` (7 tests) covering every precedence
     branch (owner / admin / EDIT / VIEW / owner-ACL / loading / undef)

   Form integration tests proved flaky against react-hook-form's async
   `isValid` with our jest-dom mock setup; deferred to a follow-up PR
   with a proper `@librechat/client` test harness.

6. **Shared `SKILL_NAME_PATTERN`** — promoted the regex plus the four
   length constants (`SKILL_NAME_MAX_LENGTH`,
   `SKILL_DESCRIPTION_MAX_LENGTH`, `SKILL_DESCRIPTION_SHORT_THRESHOLD`,
   `SKILL_DISPLAY_TITLE_MAX_LENGTH`, `SKILL_BODY_MAX_LENGTH`) out of
   `packages/data-schemas/src/methods/skill.ts` and into
   `packages/data-provider/src/types/skills.ts`. The data-schemas
   module now aliases the shared exports so the backend validator and
   the frontend form share one source of truth. Also fixed a latent bug:
   the client regex was stricter than the backend
   (`^[a-z0-9]+(?:-[a-z0-9]+)*$` vs. the real `^[a-z0-9][a-z0-9-]*$`),
   which would have rejected valid names like `foo--bar` client-side.

7. **Removed hardcoded "Claude"** — replaced `com_ui_skill_description_help`
   ("Claude uses this to...") with a new `com_ui_skill_create_subtitle`
   for the form header and `com_ui_skill_description_field_hint`
   ("This is what the model reads to decide...") for the inline hint.
   LibreChat is LLM-agnostic; the old copy misled GPT/Gemini users.

8. **Lifted tree mutation hook** — `useDeleteSkillFileMutation` is now
   instantiated once in `SkillFileTree` (not per `TreeRow`). A
   `TreeContext` provides `onDeleteFile` + `isDeleting` + `canEdit` to
   rows. A 60-node tree used to instantiate 60 mutation hooks; it now
   instantiates one.

9. **List O(n) re-render** — `SkillListItem` no longer reads
   `useParams()` directly. `SkillList` reads the active id once and
   passes `isActive` as a prop, so navigation only re-renders the two
   items whose `isActive` flipped (memo'd), not all N items.

10. **Deduped help text** — the field-level hint and form-level subtitle
    now use different translation keys with distinct copy instead of
    showing the same sentence twice on the same page.

11. **Removed ineffective `useCallback`** — `DeleteSkill.handleDelete`,
    `CreateSkillForm.onSubmit` / `.handleCancel`, `SkillForm.onSubmit`,
    and `SkillFileTree.handleDeleteFile` all wrapped closures around
    React Query `mutation` refs, whose identities change every render.
    Their dep arrays invalidated every render, making the memo a no-op
    with extra overhead. `SkillFileTree` now destructures the stable
    `mutate` function and inlines the arrow inside the memoized
    `contextValue` — one stable reference per deps change.

12. **Import order** — fixed shortest→longest package ordering and
    longest→shortest local ordering across all touched skill files per
    AGENTS.md. `react` always first where imported.

13. **Memoization principle** — documented the rule with inline comments:
    `memo` on components that appear in repeated contexts (`TreeRow`,
    `SkillListItem`) or as children of frequently-re-rendering parents
    (`ShareSkill` / `DeleteSkill` under `SkillForm`'s per-keystroke
    form-state updates). Removed `memo` from `SkillFileTree` since its
    parent `SkillDetailPanel` only re-renders on query-data changes.

- 38 data-schemas skill.spec.ts
- 34 packages/api permissions.spec.ts
- 25 api skills.test.js
- 16 client unit tests (9 buildTree + 7 useSkillPermissions)
- All type-checks + eslint clean on touched files

* 🧹 fix: Skills Duplication, Input Styling, Remove LLM-specific Copy

Three UI fixes from an in-chat review pass:

1. **Sidebar duplication** — `SkillsView` was rendering its own
   `SkillsSidePanel` aside alongside the chat side panel's
   `SkillsAccordion`, so on `/skills` the user saw the skill list
   twice. Fixed by mirroring the `InlinePromptsView` pattern: the
   route content is now just the detail / create panel and the
   chat side panel is the sole list. Added `/skills → /skills/new`
   redirect and a `/skills/new` literal route so `useParams().skillId`
   is `undefined` for "new" (matches prompts).

2. **Name Input styling** — the big floating-label pattern used by
   prompts/agents for the primary name field was replaced with a
   conventional `<Label>` + `<Input>` above it, diverging from the
   rest of the app. Restored the prompts-style `text-2xl` input with
   the peer-focus animated label on both `CreateSkillForm` and
   `SkillForm`. Kept the conventional pattern for description and
   body since they're textareas.

3. **Remove LLM-specific copy from skill translations** — dropped
   `com_ui_skill_description_help` ("Claude uses this to...") and
   the transitional "This is what the model reads..." phrasing.
   Field hint is now a neutral "Be specific about when this skill
   should apply." and the create-page subtitle is a neutral "Author
   a new skill your agents can invoke." LibreChat is LLM-agnostic;
   baking product names into user-facing copy is wrong outside the
   `com_endpoint_anthropic_*` keys where the setting actually only
   applies to Claude models.

Side-effect: the `SkillDetailView` wrapper in `SkillsView` now only
renders the file-tree aside when the skill has > 0 files — same
conditional-tree behavior as before, just scoped to this route
instead of also trying to also render a list sidebar.

- 16 client skill tests still pass
- Type-check + eslint clean on touched files

* 🎁 feat: Restore Skills UI from PR #12580

Brings back everything the original UI PR (#12580, commit da039917c)
shipped that my earlier rebase dropped. Verbatim restores where possible;
adapts the new hooks/types where the backend contract has shifted.

**Scoped-out / gated-off (now restored as inert UI scaffolding):**
- `hooks/useSkillFavorites.ts` + `utils/favoritesError.ts` + the
  `useGetSkillFavoritesQuery` / `useUpdateSkillFavoritesMutation` additions
  in `data-provider/Favorites.ts`. The backend route doesn't exist yet —
  the data-service functions resolve with empty arrays so the Star UI is a
  visual-only no-op until phase 2.
- `dialogs/SkillSelectDialog.tsx` + the "Add Skills" section in
  `SidePanel/Agents/AgentConfig.tsx` (still gated behind the original
  `false &&`) + `skills?: string[]` on `AgentForm` / `Agent` /
  `AgentCreateParams` / `AgentUpdateParams` + the `skills: []` entry in
  `defaultAgentFormValues`.
- `TUserFavorite.skillId` reserved on the shared favorites type.

**Concept-is-gone / deleted-types (restored as UI-only types + stubs):**
- `InvocationMode` enum and `TSkillNode`, `TSkillTreeResponse`,
  `TCreateSkillNodeRequest`, `TUpdateSkillNodeRequest` types in
  `packages/data-provider/src/types.ts`. UI-facing only; the backend flat
  `TSkillFile[]` contract is unchanged.
- `TSkill.invocationMode?: InvocationMode` as an optional field. Forms
  read/write it in local state and deliberately drop it from the PATCH
  payload until the backend column lands.
- `tree/SkillFileTree.tsx` (`react-arborist`-based), `SkillTreeNode.tsx`,
  `TreeToolbar.tsx`, `SkillFileEditor.tsx`, `SkillFilePreview.tsx` — full
  filesystem-style browser UI restored verbatim.
- `data-provider/Skills/tree-queries.ts` + `tree-mutations.ts` hooks
  (`useGetSkillTreeQuery`, `useCreateSkillNodeMutation`, etc.). The
  `data-service` stubs them: `getSkillTree` returns `{ nodes: [] }`,
  `createSkillNode` / `updateSkillNode` / `updateSkillNodeContent` return
  synthetic node shapes, `deleteSkillNode` resolves void. Hooks compile
  and run; tree is empty until phase 2 wires a real backend.
- `MutationKeys.createSkillNode` / `updateSkillNode` / `deleteSkillNode` /
  `updateSkillNodeContent` + `CreateSkillNodeBody` /
  `UpdateSkillNodeVariables` / `DeleteSkillNodeBody` /
  `UpdateSkillNodeContentVariables` types.
- `QueryKeys.skillTree` / `skillNodeContent` / `skillFavorites` /
  `favorites` and the `skillTree()` endpoint helper.

**Scope-simplified (restored with minimal adaptation):**
- `display/SkillDetailHeader.tsx` + `display/SkillDetail.tsx`. Header now
  falls back to `InvocationMode.auto` when `skill.invocationMode` is
  undefined.
- `forms/SkillContentEditor.tsx` — click-to-edit markdown preview toggle
  for the SKILL.md body field. Wired into both `CreateSkillForm` and
  `SkillForm` replacing the plain `<TextareaAutosize>`.
  (Needed `@ts-ignore` on `remarkPlugins` / `rehypePlugins` for the same
  `PluggableList` vs `Pluggable[]` shape drift `MarkdownLite.tsx` already
  works around.)
- `forms/InvocationModePicker.tsx` + `forms/CategorySelector.tsx` — the
  auto/manual/both dropdown and the skill category selector. Wired into
  both forms inside a `FormProvider` so the Controller-based widgets can
  read `useFormContext`. `category` flows to the PATCH / POST payload as
  before; `invocationMode` is UI-only per the type note above.
- `buttons/CreateSkillMenu.tsx` + `utils/parseSkillMd.ts` — dropdown with
  AI / Manual / Upload SKILL.md entries + the YAML frontmatter parser for
  the upload path. `CreateSkillForm.defaultValues` now accepts the parsed
  shape, so the upload → redirect → pre-populated form flow works again.
- `buttons/AdminSettings.tsx` — admin permissions dialog. Uses the
  existing `useUpdateSkillPermissionsMutation` which was already wired.
- `sidebar/FilterSkills.tsx` — restored filter + AdminSettings +
  CreateSkillMenu wrapper. `SkillsSidePanel.tsx` is back to the original
  `FilterSkills`-based layout.
- `lists/SkillList.tsx` + `lists/SkillListItem.tsx` — restored verbatim.
- `layouts/SkillsView.tsx` — restored the full tree + file editor + file
  preview layout. The chat side panel keeps its own accordion list; this
  view is the inline detail experience.
- `hooks/Generic/useUnsavedChangesPrompt.ts` — route-leave guard hook.

- `useGetSkillByIdQuery` is aliased to `useGetSkillQuery` so restored
  components (`SkillsView`, `SkillForm`) that import the old name resolve
  to the new hook.
- `SkillSelectDialog` + `AgentConfig` coerce `skillsData?.skills` instead
  of `.data` (list response shape drift from the CRUD PR).
- `CreateSkillForm` / `SkillForm` wrap their JSX in `FormProvider` so the
  restored `CategorySelector` and `SkillContentEditor` components —
  which read `useFormContext` — work inside the existing forms without
  another refactor.
- `CreateSkillForm.defaultValues` prop accepts `Partial<Values> &
  { invocationMode?: unknown }` so the upload flow's
  `{ name, description, invocationMode }` shape passes through cleanly.
- `SkillsView` route map gains `/skills/:skillId/edit` and
  `/skills/:skillId/file/:nodeId` so the tree-navigation URLs the original
  view produces actually resolve.
- `client/package.json` gains `react-arborist@^3.4.3`.
- ~60 translation keys the restored files reference — invocation labels,
  edit/create page titles, file editor chrome, tree toolbar tooltips,
  favorites, admin allow-settings, unknown-file-type, sr_public_skill,
  delete/rename _var variants — all added to `en/translation.json`.

- Prompts-style floating-label name input — kept from my earlier commit
  so it matches the rest of the app (user reviewed and approved that
  styling). Hidden skill-body textarea is replaced by `SkillContentEditor`
  in both forms.

- 38 data-schemas skill.spec.ts
- 34 packages/api permissions.spec.ts
- 25 api skills.test.js
- 7 client useSkillPermissions.test.ts
- Type-check: pre-existing error count (188) dropped to 120 because my
  restorations fixed some previously-broken field types.

* chore: Update package-lock.json to include react-arborist and memoize-one

* feat: Add support for react-arborist in Vite configuration

This update introduces a new condition in the Vite configuration to handle the 'react-arborist' package, ensuring it is properly recognized during the build process. This change enhances compatibility with the recently added 'react-arborist' dependency in the project.

* 🩹 fix: Hide InvocationMode, Fix SkillContentEditor Click-to-Edit

1. Hide InvocationModePicker from both CreateSkillForm and SkillForm.
   Component stays on disk for when the backend lands the column.

2. Fix "Click to edit" doing nothing on SkillContentEditor. The
   `onBlur={() => setIsEditing(false)}` on the TextareaAutosize was
   racing with `autoFocus` — React renders the textarea, autoFocus
   fires, then a layout/reconciliation blur fires immediately,
   bouncing back to preview mode before the user can interact.
   Removed onBlur; users toggle via the header button or Escape key.

* 🎨 feat: Reader-First Skills UI — Match Claude.ai Layout

Reworks the Skills UI from form-first to reader-first, matching
Claude.ai's skill detail pattern.

**Default view is now read-only.** Clicking a skill in the sidebar
navigates to `/skills/:id` which renders `SkillDetail` — a clean
content view with:
- Skill name as the primary heading
- Metadata row: "Added by" + "Last updated" (formatted date)
- Description block
- Rendered SKILL.md body in a bordered card with a source/rendered
  toggle (eye + code icons, matching Claude.ai's segmented control)

No form fields, no save/cancel buttons. The user reads the skill
first and takes action deliberately.

**Create is now a dialog.** The `/skills/new` route is gone.
`CreateSkillMenu` (the + dropdown in the sidebar) now opens
`CreateSkillDialog` — a minimal modal with name, description, and
instructions fields. Upload-from-file still works: parse → populate
dialog → create. Matches Claude.ai's "Write skill instructions"
modal.

**Edit is behind an action.** The detail view shows an "Edit" button
(permission-gated) that navigates to `/skills/:id/edit`, rendering
the existing `SkillForm`. The edit route is preserved for direct
linking.

**Navigation goes to detail, not edit.** `SkillListItem` now
navigates to `/skills/:id` (detail) instead of `/skills/:id/edit`.

- `display/SkillMarkdownRenderer.tsx` — shared ReactMarkdown
  component extracted from `SkillContentEditor`. Same remark/rehype
  plugins, no form dependency.
- `display/SkillDetail.tsx` — the reader-first view (replaces the
  old thin wrapper).
- `dialogs/CreateSkillDialog.tsx` — OGDialog modal for skill
  creation.

- `layouts/SkillsView.tsx` — gutted and rebuilt. Three states:
  no-skill (empty state), skillId (SkillDetail), skillId+edit
  (SkillForm). Removed full-page CreateSkillForm, removed TreeView.
- `buttons/CreateSkillMenu.tsx` — opens dialog instead of navigating
  to `/skills/new`. Upload flow: parse → set dialog defaults → open.
- `lists/SkillListItem.tsx` — navigate to detail, not edit.
- `routes/index.tsx` — removed `/skills/new` and file/nodeId routes;
  `/skills` renders SkillsView directly (empty state).
- `display/index.ts`, `dialogs/index.ts` — added new exports.
- `locales/en/translation.json` — added ~10 new keys for metadata,
  toggle labels, dialog title, empty state.

* 🩹 fix: SkillContentEditor click-to-edit z-index — button was z-0 behind rendered content

* 🩹 fix: Align Edit button size with Share/Delete (size-9)

* 🎨 feat: Claude.ai-Style Skill List Panel

Rewrites the skills sidebar to match Claude.ai's panel layout:

- Header: "Skills" title + search icon (toggles input) + add icon
  (opens CreateSkillDialog directly, no dropdown menu)
- Collapsible "Skills" section with chevron toggle
- Skill items: 24px icon badge (rounded square with ScrollText icon)
  + name only. No description text in the list — that lives in the
  detail view. Active item gets highlighted bg + bold font.
- Removed AdminSettings button from sidebar header — admin config
  is accessible via the admin dashboard, not cluttering every user's
  skill list.
- Removed FilterSkills wrapper (was Filter + AdminSettings +
  CreateSkillMenu). The search + create are now inline in the panel
  header.

Files changed:
- sidebar/SkillsSidePanel.tsx — full rewrite
- sidebar/SkillsAccordion.tsx — simplified wrapper
- lists/SkillList.tsx — collapsible section, no description
- lists/SkillListItem.tsx — icon badge + name, memo'd

* 🎨 fix: Align Skills UI Styling with Prompts Patterns

Style alignment pass based on direct comparison with claude.ai and
the existing prompts preview dialog.

SkillsSidePanel search now replaces the title in the header row when
toggled (search icon + input + X close), matching Claude.ai's pattern.
Previously it pushed a separate input below the header, wasting
vertical space. Close button clears the search term.

Replaced `text-text-tertiary` with `text-text-secondary` across
SkillDetail, SkillList, SkillForm, CreateSkillForm, CreateSkillDialog,
SkillContentEditor. Tertiary was too dark / low contrast.

SkillList section chevron label now reads "Personal skills" (matching
Claude.ai) via the existing `com_ui_my_skills` key, instead of the
generic "Skills" which duplicated the header.

Aligned with `PromptDetailHeader` styling:
- 48px round icon (ScrollText in bg-surface-secondary circle)
- Name + public badge in the icon row
- Metadata below the icon: User icon + author, Calendar icon + date
  (text-xs text-text-secondary with gap-3, matching prompts exactly)
- Description uses the same label-above-text pattern as prompts
- Content card uses `bg-transparent` border (not bg-surface-primary-alt)
- Toggle buttons use size-5 icons and text-text-secondary for inactive

Changed from `max-w-lg p-0` to `max-w-5xl` with the same max-height
and padding pattern as the prompts PreviewPrompt dialog:
`max-h-[80vh] p-1 sm:p-2 gap-3 sm:gap-4`. Close button now renders
via default OGDialogContent behavior (removed showCloseButton=false).

* 🩹 fix: SkillDetail fills parent height, tighter spacing (px-6 pb-6 gap-2)

* 🩹 fix: Align Skills panel header padding (px-4) with list content below

* 🩹 fix: Reduce Skills header top padding (pt-2) to align with sidebar icon strip

* 🩹 fix: Tighten Skills header (py-2) and detail top (py-2) to align with sidebar icons and match edit view

* 🩹 fix: Offset SidePanel Nav pt-2 with -mt-2 on SkillsAccordion so Skills header aligns with icon strip

* 🛠️ fix: Increase Node memory limit for production build in package.json

* 🩹 fix: Remove top padding from SkillDetail header row (py-2 → pb-2)

* 🏗️ refactor: Move pt-2 from SidePanel/Nav wrapper to each panel

Removed the global `pt-2` from `SidePanel/Nav.tsx` and pushed it
into each panel's own top-level wrapper. This lets each panel own
its vertical alignment independently — Skills can sit flush at the
top to align with the sidebar icon strip, while other panels keep
their original spacing.

Panels updated with `pt-2`:
- PromptsAccordion (via className on PromptSidePanel)
- BookmarkPanel
- FilesPanel
- MemoryPanel
- MCPBuilderPanel
- AgentPanel (form wrapper)
- AssistantPanel (form wrapper)
- ParametersPanel (already had pt-2)

SkillsAccordion: removed the -mt-2 hack, now naturally flush.

* 🧹 fix: Align CreateSkillDialog field styling + remove 19 unused i18n keys

Dialog fields: all three inputs now use consistent `rounded-xl
border-border-medium px-3 py-2 text-sm` styling. Replaced the
`<Input>` component with a plain `<input>` to avoid the component's
built-in `rounded-lg border-border-light` overriding the dialog's
border style. Labels use `font-medium` for consistency.

Removed 19 unused translation keys from translation.json:
com_ui_skill_body, com_ui_skill_body_placeholder,
com_ui_skill_create_subtitle, com_ui_skill_file_delete_confirm,
com_ui_skill_file_delete_error, com_ui_skill_file_deleted,
com_ui_skill_files_empty, com_ui_skill_files_multi_hint,
com_ui_skill_list, com_ui_skill_load_error,
com_ui_skill_resize_file_tree, com_ui_skill_select_file,
com_ui_skill_select_file_desc, com_ui_skills_load_error,
com_ui_add_first_skill, com_ui_create_skill_page,
com_ui_edit_skill_page, com_ui_save_skill, com_ui_no_skills_title

* 🎁 feat: Upload Skill Dialog + Simplified Create Menu

New `UploadSkillDialog` matching Claude.ai's upload modal:
- Dashed drop zone with drag-and-drop support
- Accepts .md, .zip, .skill files
- Phase 1: processes .md files (parses YAML frontmatter → creates
  skill with body as the full file content)
- Shows file requirements below the drop zone
- On success: navigates to the new skill's detail view

`CreateSkillMenu` now has two flat options (no sub-menu):
- "Write skill instructions" → opens `CreateSkillDialog`
- "Upload a skill" → opens `UploadSkillDialog`

Removed the disabled "Create with AI" option and the old file input
hidden-element approach. The sidebar `+` button now renders
`CreateSkillMenu` directly instead of a standalone create dialog.

- Removed 5 unused i18n keys (com_ui_skill_added_by,
  com_ui_skill_last_updated, com_ui_skills_add_first,
  com_ui_skills_filter_placeholder, com_ui_skills_new)
- Tightened metadata gap in SkillDetail (mt-1 → mt-0.5)
- Added 7 new upload-related i18n keys

* 🔒 feat: Zip/Skill File Upload Support with Safety Limits

Rewrites UploadSkillDialog to properly handle all three accepted
file types:

- `.md` — reads as text, parses YAML frontmatter, creates skill
- `.zip` / `.skill` — reads as ArrayBuffer, extracts with JSZip,
  finds SKILL.md (at root or one level deep), parses its content,
  creates skill. Shows spinner during processing.

Security guards against zip bombs:
- MAX_ZIP_SIZE: 50MB compressed file limit
- MAX_ENTRIES: 500 file limit inside the archive
- Path traversal rejection: skips entries with `..` or leading `/`
- SKILL.md search limited to depth ≤ 2 segments

Added `jszip@^3.10.1` to client dependencies (already in the
monorepo's node_modules from backend usage).

The name is inferred from the zip filename if SKILL.md frontmatter
doesn't have one (e.g. `skills-autofix.zip` → `skills-autofix`).

* 🚀 feat: Backend Skill Import + Live File Upload Endpoints

New endpoint that accepts a single multipart file (.md, .zip, .skill)
and creates a skill with all its files in one request:

- **.md**: parse YAML frontmatter → create skill with body
- **.zip / .skill**: extract with JSZip, find SKILL.md (root or one
  level deep), create skill from its content, then persist every
  additional file via `upsertSkillFile` + local file storage strategy.
  Returns the created skill + an `_importSummary` with per-file
  results.

Security:
- 50MB compressed file size limit (multer)
- 500 max entries in archive
- 10MB per individual file
- Path traversal rejection (no `..`, no absolute, validated charset)
- File type filter: only .md/.zip/.skill accepted
- Rate limited via existing `fileUploadIpLimiter` + `fileUploadUserLimiter`

Handler lives in `packages/api/src/skills/import.ts` with injectable
deps (`createSkill`, `upsertSkillFile`, `saveBuffer`) for testability.

Replaced the 501 stub with a real handler:
- Accepts multipart FormData with `file` + `relativePath`
- Saves file via local storage strategy
- Calls `upsertSkillFile` to persist the SkillFile record
- Returns the upserted document
- Rate limited, ACL-gated (EDIT permission required)
- 10MB per file limit

`UploadSkillDialog` now sends the file to `/api/skills/import` via
`dataService.importSkill(formData)` — no more client-side JSZip.
Removed `jszip` from client dependencies (only backend needs it).

Added `importSkill()` in data-service + `importSkill()` endpoint
builder in api-endpoints.

Updated the file upload test from expecting 501 stub to expecting
400 "no file provided" (live validation). All 25 skill route tests
pass.

* 🔒 fix: Complete Import Handler — Validation, Ownership, Error Surfacing

Fixes several gaps in the skill import flow:

1. **Skill validation now runs and surfaces properly.** The import
   handler calls the real `createSkill(CreateSkillInput)` which runs
   `validateSkillName`, `validateSkillDescription`, `validateSkillBody`.
   Validation errors (SKILL_VALIDATION_FAILED) are caught and returned
   as 400 with the issue messages. Duplicate-key errors return 409.
   Previously all errors were swallowed into a generic 500.

2. **`authorName` is now populated.** The `CreateSkillInput` requires
   `authorName` which was missing — resolved from `req.user.name ??
   req.user.username ?? 'Unknown'`, matching the existing create handler.

3. **SKILL_OWNER permission is granted after import.** Calls
   `grantPermission` with `AccessRoleIds.SKILL_OWNER` so the uploader
   can edit/delete/share the imported skill. This was entirely missing —
   imported skills would have been ownerless.

4. **`tenantId` propagated.** Both the skill and each SkillFile record
   receive `req.user.tenantId` for multi-tenant deployments.

5. **SkillFile records are created in the DB.** Each non-SKILL.md file
   in the zip is saved to file storage via `saveBuffer` and recorded
   via `upsertSkillFile`, which validates the relativePath, infers the
   category from the path prefix, and atomically bumps the skill's
   `fileCount` and `version`.

Import deps now include `grantPermission` from PermissionService,
injected in `api/server/routes/skills.js`.

* 🐛 fix: Import grant uses accessRoleId (not roleId) — fixes skill not appearing in list

* 🎨 fix: Cache invalidation, file tree, frontmatter rendering

Three fixes for the skill detail view:

1. **Cache invalidation after import.** UploadSkillDialog now calls
   `queryClient.invalidateQueries([QueryKeys.skills])` after a
   successful import so the sidebar list picks up the new skill
   without requiring a page refresh.

2. **File tree in detail view.** When a skill has `fileCount > 0`,
   the detail view now queries `useListSkillFilesQuery` and renders
   a file list below the body card — SKILL.md first, then folders
   and root files. Icons: Folder for directories, FileText for files.

3. **Frontmatter stripped and rendered as metadata.** YAML frontmatter
   (`---\nversion: 0.1.0\ntriggers: ...\n---`) is now parsed out of
   the body before markdown rendering. The `name` and `description`
   fields are skipped (already shown in the header). Remaining fields
   (version, triggers, dependencies, etc.) are displayed in a
   Claude.ai–style grid: label on the left, value on the right,
   above the rendered markdown content. Source view still shows the
   full raw body including frontmatter.

* 🩹 fix: Always fetch skill files — fileCount may be stale in cached skill object

* 🌳 feat: Inline File Tree in Sidebar Skill List

Moves the file tree from the bottom of SkillDetail into the sidebar
list, matching Claude.ai's pattern:

- Multi-file skills show a chevron toggle on the right side of the
  skill list item
- Clicking the chevron expands an inline file tree below the skill
  name: SKILL.md first, then folders (with folder icon + right
  chevron) and root files
- File list is fetched lazily (only when expanded) via
  useListSkillFilesQuery
- Clicking a file navigates to the skill detail view
- Files section removed from SkillDetail — the sidebar is now the
  sole file tree location, keeping the detail panel clean

SkillDetail cleaned up: removed groupFiles helper, file-related
state, useListSkillFilesQuery import, FileText/Folder icon imports.

* 🌲 feat: Virtualized inline file tree with react-vtree

Replace hand-rolled recursive FolderRow/FileRow buttons with a proper
virtualized FixedSizeTree from react-vtree for the sidebar skill list.
Dynamic height tracks open folders; capped at 350px with smooth
expand/collapse transitions.

* chore: Remove no longer used SkillFileTree and SkillTreeNode components

* chore: Update Vite config to replace 'react-arborist' with 'react-vtree' for module resolution

* feat: Skill file content viewing with lazy DB caching

- Add `skills` field to `fileStrategiesSchema` so operators can
  configure a dedicated storage backend for skill files. Falls back
  by type (image/document) when unset.
- Fix hardcoded `FileSources.local` in skill save/import — now uses
  the resolved strategy via `getFileStrategy(req.config, { context })`.
- Replace 501 download stub with real handler that streams from any
  storage backend and returns JSON `{ content, mimeType, isBinary }`.
- Binary detection (null-byte + non-printable ratio on first 8 KB)
  flags files on first read so they're never re-fetched.
- Text content ≤ 512 KB is cached in the SkillFile MongoDB document;
  subsequent reads skip storage entirely.
- Clicking a skill row now expands inline files (not just chevron).
- Clicking a file navigates to `?file=<path>` and renders content
  in a new SkillFileViewer (markdown, code, images, binary placeholder).

* chore: Remove react-window and its type definitions from package.json and package-lock.json

- Deleted `react-window` and `@types/react-window` dependencies from both `package.json` and `package-lock.json` to streamline the project and reduce unnecessary bloat.

* fix: Build errors — remove endpoints import, fix Uint8Array cast

- Replace `import { endpoints }` (not public) with inline URL in
  SkillFileViewer
- Remove `as Uint8Array` cast in stream chunk handling
- Extend getSkillFileByPath return type with content/isBinary to
  decouple from data-schemas build artifact resolution

* chore: Remove 8 unused i18next keys

com_ui_create_skill_ai, com_ui_create_skill_manual,
com_ui_delete_folder_confirm_var, com_ui_delete_skill,
com_ui_delete_skill_confirm_var, com_ui_delete_var,
com_ui_rename_var, com_ui_skill_files

* fix: Add configMiddleware to skills router, handle SKILL.md in viewer

- Add configMiddleware to skills router so req.config is populated
  when getLocalFileStream (or any strategy) reads file paths.
- Handle SKILL.md in download handler — serves skill.body directly
  from the Skill document instead of looking for a SkillFile record.
- Clicking SKILL.md in sidebar tree now opens the file viewer
  (matching Claude.ai behavior: file view vs default detail view).

* ci: Run unit tests on PRs to any branch

Remove the branches filter from both test workflows so contributor
PRs targeting feature branches (not just main/dev) get CI coverage.
Path filters are kept so tests only run when relevant files change.

* fix: Update skills route tests for download handler changes

- Mock configMiddleware (sets req.config for file storage access)
- Mock getStrategyFunctions and getFileStrategy (storage strategy deps)
- Replace 501 stub test with SKILL.md content test + 404 test

* fix: Auto-expand files, frontmatter parsing, select-none, prefetch

- Auto-expand file tree when navigating directly to a skill URL
- Prefetch files for the active skill (eliminates first-expand lag)
- Fix frontmatter parser to handle multi-line YAML list values
  (triggers field was missing because it uses list syntax)
- SkillFileViewer now parses frontmatter for .md files — shows
  structured grid + rendered body (matching SkillDetail's display)
  with source/rendered toggle
- Add select-none to all sidebar skill and file tree buttons

* refactor: Derive expanded state from isActive instead of useEffect

Replace useEffect sync with deterministic derivation:
expanded = hasFiles && (isActive || !collapsed)

Active skill is always open. collapsed is a manual toggle that
only takes effect on non-active items.

* fix: Remove empty space above body card — overlay view toggle

Move the rendered/source toggle from a dedicated row (40px of empty
space) to an absolute-positioned overlay in the card's top-right
corner, matching Claude.ai's layout.

* fix: Remove header bars from content editors — overlay action buttons

Collapse the full-width header bars ("Skill Content", "Text") in
SkillContentEditor, PromptTextCard, and PromptEditor. Action buttons
(edit/save toggle, copy, variables) are now absolute-positioned in
the card's top-right corner, reclaiming ~46px of vertical space.

* fix: Spinner visibility in file viewer — use text-text-secondary

* fix: Address review findings — security, correctness, code quality

Codex P1: Use $unset instead of undefined to clear cached content
and isBinary fields on file re-upload (Mongoose strips undefined).

Codex P2: Match skill-file validation errors by error.code instead
of error.message substring.

F1: Zip bomb defense — track cumulative decompressed bytes (500 MB
cap), check declared uncompressed size before buffering each entry.

F2: Remove misleading "atomically" from import handler JSDoc.

F3: Static import for isBinaryBuffer instead of dynamic import().

F4: Replace console.error with logger in upload handler.

F6: Add multer error handler middleware to skills router.

F7: Move React import to top of SkillDetail.tsx.

F9: Fix variable shadowing (trimmed → item) in parseFrontmatter.

F11: Replace JSON.parse(JSON.stringify()) with toJSON() for
Mongoose document serialization.

F12: Remove dead dynamic import('fs') fallback (memoryStorage
always provides file.buffer).

F13: Hoist MIME_MAP to module scope to avoid per-call allocation.

F16: Share single multer.memoryStorage() instance.

* fix: Follow-up review — close zip bomb gap, fix error handler

F1: Add post-decompression cumulative byte check with break (the
pre-decompression check relies on undocumented JSZip internals
that may be absent; this closes the gap unconditionally).

F2+F3: Multer error handler now forwards non-multer errors via
next(err) instead of swallowing them. Also catches file filter
rejections (plain Error, not MulterError) by message prefix.

F4: Move isBinaryBuffer import to local imports section per
CLAUDE.md import order rules.

F5: Simplify dead toJSON branch — createSkill returns a POJO.

* nit: Link filter error message to handler prefix check

* feat: Accordion expansion + active file highlight in sidebar

- Only one skill's file tree can be expanded at a time (accordion).
  Expansion state lifted from SkillListItem to SkillList.
- Selected file gets bg-surface-active highlight in the tree.
  Skill row uses subtle style (no background) when a file is active,
  matching Claude.ai's pattern where the file — not the skill —
  carries the selection state.

* style: Adjust margin for file tree in SkillListItem component

- Reduced left margin from 10 to 5 for improved layout consistency in the file tree display.

* fix: TS error on FileTreeNode, nested ternary, chevron collapse

- Make style prop optional to match react-vtree's NodeComponentProps
- Flatten nested ternary for skill row active styles
- Skill row click expands (but doesn't collapse) files + navigates
- Chevron click explicitly toggles collapse (matching Claude.ai
  where clicking the chevron is how you collapse files)

* fix: Upload basePath, reject SKILL.md uploads, add skills permission route

- Pass basePath: 'uploads' in per-file upload handler (was defaulting
  to 'images' path, inconsistent with the import flow).
- Reject uploads targeting SKILL.md (reserved path — download handler
  special-cases it to return skill.body, making an uploaded file
  unreachable via the API).
- Add skills entry to roles router permissionConfigs so PUT
  /api/roles/:roleName/skills actually reaches a handler instead
  of returning 404.

* feat: Expand content area, move controls to header, reduce padding

Default detail view:
- Remove rounded-xl bordered card wrapper — content flows directly
  into the article, capitalizing on full screen width
- Move eye/code toggle inline with the divider row
- Reduce px-6/pb-6 to px-4/pb-4

File viewer:
- Move eye/code toggle from card overlay to the header bar
- Add copy-to-clipboard button for text files in the header bar
- Remove rounded-xl bordered card wrapper for markdown content
- Remove bordered pre wrapper for non-markdown text
- Reduce px-6/py-4 to px-4/py-3

Both views maximize content space over decorative chrome.

* fix: Stable header height, restore some padding

- Fix layout shift in file viewer header: use fixed h-10 so the
  bar height stays constant whether the eye/code toggle renders
  (markdown) or not (plain text).
- Bump content padding from px-4/py-3 back to px-5/py-4 in both
  views — the previous reduction was too aggressive.

* fix: Grant rollback, path validation, error format, dead code cleanup

F2: grantOwnership now rolls back (compensating delete) on failure,
matching the create handler. Both markdown and zip import paths
check the result and return 500 on grant failure.

F4: Upload handler validates relativePath with regex + traversal
check before calling downstream upsertSkillFile.

F5: Document JSZip _data.uncompressedSize as best-effort; the
post-decompression cumulative check is the real safety net.

F10: Standardize all upload handler error responses to { error }
(was { message }, inconsistent with handlers.ts).

F13: Single-pass fileResults accumulation in import handler.

F1-5: Remove dead uploadFileStubHandler (no route references it).

Codex P2: Fix delete nav from /skills/new to /skills.

F12: Use cn() in UploadSkillDialog instead of template literals.

* perf: Stream-first binary detection + O(1) public skill check

F1: Download handler now reads only the first 8 KB for binary
detection. If binary, the stream is destroyed immediately without
buffering the remaining file. Text files continue reading for
caching. Eliminates buffering up to 10 MB per request for binary
files under concurrent load.

F7: Single-skill GET and PATCH now use hasPublicPermission (O(1)
ACL lookup) instead of getPublicSkillIdSet (queries ALL public
skill IDs). The list handler still uses the Set approach since it
serializes multiple skills. serializeSkill/serializeSkillSummary
now accept boolean | Set for flexibility.

* fix: Update test to match { error } response format

* fix: Critical stream truncation bug, grantedBy, error format

NF-1 (CRITICAL): Rewrite binary detection to single for-await loop.
Breaking out of for-await-of destroys the stream via iterator.return(),
so the previous two-loop approach silently truncated text files > 8KB.
Now: one loop collects chunks, checks binary after 8KB accumulated,
and either destroys+returns (binary) or continues reading (text).

NF-2: Add grantedBy to import handler's grantPermission call and
interface (was missing, inconsistent with create handler).

NF-3: Standardize all import handler error responses from { message }
to { error }, matching handlers.ts convention. Update client's
UploadSkillDialog to read response.data.error accordingly.

* fix: Prefer specific validation message over generic error field

* fix: YAML quote stripping, saveBuffer null guard, dot segment rejection

- Strip surrounding YAML quotes from frontmatter values so
  name: "my-skill" parses as my-skill (not "my-skill" with quotes
  that fails the name validator).
- Guard resolveSkillStorage against backends with saveBuffer: null
  (e.g. OpenAI/vector strategies) — throws a descriptive error
  caught by the handler's try/catch instead of a TypeError.
- Tighten upload path validation to reject . segments (e.g.
  docs/./a.md) matching the model-layer validator, preventing
  storage writes for paths the DB will reject.

* fix: Orphan cleanup, stream errors, malformed zip, cache latency

F1: Upload handler now deletes the stored blob if the subsequent
DB upsert fails, preventing orphaned files on disk/cloud.

F2: Multer error handler returns { error } (was { message }).

F3: Wrap JSZip.loadAsync in try/catch — malformed zip returns 400
instead of falling through to 500.

F4: Raw download stream gets an error handler — logs the error and
destroys the response if headers were already sent.

F8: Strip leading hyphens from inferred skill name so filenames
like _my-skill.zip don't produce -my-skill (invalid name pattern).

F9: Fire-and-forget all updateSkillFileContent cache writes so the
response is sent immediately. Cache failures are logged but don't
block or fail the read.

* fix: Import orphan cleanup + Content-Disposition sanitization

Finding A: Add deleteFile dep to ImportSkillDeps. The per-file loop
in handleZip now cleans up stored blobs when upsertSkillFile fails,
closing the second half of the F1 orphan fix (upload handler was
already fixed).

Finding B: Sanitize filename in Content-Disposition header for raw
downloads — strip quotes, backslashes, and newlines to prevent
header injection from user-uploaded filenames.

* security: Prevent stored XSS via raw file downloads

Non-image files served via ?raw=true now use Content-Disposition:
attachment (force download) instead of inline. An uploaded .html or
.svg file served inline from the LibreChat origin could execute
scripts with access to the user's session — this closes that vector.

Images stay inline (needed for <img> rendering in SkillFileViewer).
X-Content-Type-Options: nosniff added to prevent MIME sniffing.

* security: Block SVG XSS — allowlist safe raster MIME types for inline

SVG (image/svg+xml) passed the startsWith('image/') check and was
served inline, but SVG is a scriptable format — embedded <script>
tags execute in the LibreChat origin. Replace the prefix match with
a Set of safe raster-only MIME types (png, jpeg, gif, webp, avif,
bmp). SVGs and any future scriptable image/* subtypes now get
Content-Disposition: attachment (forced download).

* fix: Cap JSON text response at 1MB, consistent md name inference

F3: Text files > 1MB now return { isBinary: false } with no content
field, forcing the client to use ?raw=true for download. Prevents
buffering 10MB files into heap for JSON serialization. Frontend
shows a download fallback when content is absent.

F4: handleMarkdown now infers skill name from filename (same as
handleZip) when frontmatter has no name, instead of rejecting
with 400. Consistent behavior across import paths.

F1 (reviewer concern): upsertSkillFile is NOT affected — it uses
{ new: false } for insert-vs-replace detection but does a follow-up
findOne (lines 855-859) to return the post-upsert document.

* fix: deleteFile arg shape, raw URL base path, hoist SAFE_INLINE_MIMES

Codex P2: deleteFile expects { filepath } object, not a raw string.
Both upload handler cleanup and import handler cleanup now pass
{ filepath } to match the strategy contract (deleteLocalFile,
deleteFileFromS3 all expect a file object).

Codex P2: Raw download URL in SkillFileViewer now uses apiBaseUrl
prefix so subpath deployments (/chat, etc.) resolve correctly.

NIT: Hoist SAFE_INLINE_MIMES Set to factory scope — was re-allocated
per raw download request inside the if block.

* fix: Remove inert cache write for large text files, localize aria-label

N2: The { isBinary: false } cache write for text files > 1MB had no
effect — subsequent requests still fell through to stream read since
neither isBinary nor content provided a fast-path short-circuit.
Removed the pointless DB updateOne per request.

N4: Replace hardcoded "Back to skill" aria-label with localize().

* refactor: Extract shared parseFrontmatter, widen deleteFile type

N3: Extract parseFrontmatter into Skills/utils/frontmatter.ts —
single implementation shared by SkillDetail and SkillFileViewer.
Accepts optional skipKeys set so callers control which frontmatter
fields are excluded (SkillDetail skips name/description since
they're shown in the header; other .md files show all fields).

N5: Widen ImportSkillDeps.deleteFile file param from { filepath }
to { filepath; [key: string]: unknown } to signal extensibility
if strategies start accessing additional file properties.

* fix: Advance i past list items for skipped keys, DRY parseSkillMd

Finding A: parseFrontmatter now consumes multi-line YAML list items
before checking skipKeys — prevents list lines from leaking into
subsequent key parsing as spurious fields.

Finding B: parseSkillMd now delegates to the shared parseFrontmatter
instead of re-implementing the same frontmatter scanning loop.
Reduces client-side parseFrontmatter implementations from 3 to 1.

* fix: Call apiBaseUrl(), delete storage blob on file removal

- apiBaseUrl is a function, not a string — call it in the template
  literal so raw download URLs resolve correctly.
- deleteFileHandler now looks up the file record before deleting,
  then fire-and-forget deletes the storage blob via the strategy's
  deleteFile. Previously only the DB record was removed, leaving
  orphaned blobs in local/S3/Firebase/Azure storage.

* fix: Clean up storage blobs when deleting an entire skill

deleteHandler now lists all files for the skill before calling
deleteSkill, then fire-and-forget deletes each blob via the
storage strategy. Previously only per-file deletion cleaned up
blobs — deleting a whole skill left all associated files orphaned
in local/S3/Firebase/Azure storage.

* refactor: useImportSkillMutation hook, fix TSkill[] unsafe cast

- Create useImportSkillMutation in mutations.ts + ImportSkillOptions
  type. UploadSkillDialog now uses the mutation hook instead of
  calling dataService.importSkill directly with manual useState
  loading management. Eliminates unmounted-component state update
  risk and aligns with the React Query mutation pattern used by
  every other mutation in the codebase.

- SkillSelectDialog: replace as unknown as TSkill[] with proper
  TSkillSummary typing. SkillCard props updated to TSkillSummary.
  The dialog only uses summary-level fields (name, description,
  category, author) — the cast was hiding a type mismatch.

* fix: Use saved source for import cleanup, delete old blob on replace

Codex P2: Import cleanup now uses file.source (the backend the file
was actually saved to) instead of re-resolving from config. In mixed
strategy setups, the previous approach could target the wrong backend.

Codex P2: When re-uploading a file to an existing relativePath, the
old blob is now deleted after successful upsert. Previously only the
DB record was replaced, leaving the old storage object orphaned.

* fix: Register PUT /:roleName/skills route in roles router

* fix: Re-read skill after zip file processing for fresh metadata

The import response was built from the skill object created before
the file loop, but each upsertSkillFile bumps version and fileCount.
Clients caching the stale response would get 409 conflicts on first
edit and see incorrect file counts.

Now re-reads the skill via getSkillById after the loop so the
response reflects the current version, fileCount, and updatedAt.

* fix: Size-check SKILL.md before decompression, don't gate on fileCount

P1: SKILL.md was decompressed before any size accounting. A crafted
archive could expand SKILL.md past 10MB before validation ran. Now
checks declared size pre-decompression and actual size post, both
against MAX_SINGLE_FILE_BYTES.

P2: File list query was gated on cached fileCount which can be stale
after mutations. Now fetches files for the active skill regardless
of fileCount. hasFiles derived from fetched data with fileCount as
fallback, so newly uploaded files appear without hard refresh.

* fix: Move files declaration before hasFiles to avoid TDZ error

* security: Stream-decompress zip entries with enforced byte cap

Replace zipEntry.async('nodebuffer') (buffers entire entry before
checking limits) with zipEntry.nodeStream('nodebuffer') piped
through a byte counter that destroys the stream when the per-file
or cumulative limit is exceeded.

Previously, when JSZip's _data.uncompressedSize was absent (the
common case), a high-ratio entry could allocate hundreds of MB
before the post-decompression check caught it. Now decompression
is aborted mid-stream at the exact byte threshold — no entry can
exceed its limit regardless of compression ratio.

* refactor: Reorganize access check for prompts in useSideNavLinks hook

Moved the prompts access check to a new position in the code to improve readability and maintainability. This change ensures that the prompts link is added to the navigation only if the user has the appropriate access, without altering the existing functionality.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-04-25 04:02:00 -04:00
Danny Avila
963068b112 🧬 feat: Scaffold Skills CRUD with ACL Sharing and File Schema (#12613)
* 🧬 feat: Scaffold Skills CRUD with ACL Sharing and File Schema

Adds Skills as a new first-class resource modeled on Anthropic's Agent
Skills, reusing the existing Prompt ACL stack for sharing. Lays the
groundwork for multi-file skills (SkillFile schema + metadata routes)
without wiring upload processing — single-file skills (inline SKILL.md
body) work end-to-end, multi-file uploads are stubbed for phase 2.

* 🔬 fix: Wire Skill Cleanup, AccessRole Enum, and Express 5 Path Params

CI surfaced four follow-ups from the initial Skills scaffolding commit
that local builds missed:

- AccessRole's resourceType field had a hardcoded enum that didn't
  include `'skill'`, blocking SKILL_OWNER/EDITOR/VIEWER role creation
  in every test that hit the AccessRole model.
- The seedDefaultRoles assertion in accessRole.spec.ts hard-listed the
  expected role IDs and needed the new SKILL_* entries.
- deleteUserController had no cleanup for skills, and the
  deleteUserResourceCoverage guard test enforces every ResourceType has
  a documented handler — wired in db.deleteUserSkills(user._id) and
  added the entry to HANDLED_RESOURCE_TYPES.
- Express 5's path-to-regexp v6 rejects the legacy `(*)` named-group
  glob syntax. The two skill file routes now use a plain `:relativePath`
  param; the client already encodeURIComponents the path, so a single
  param is sufficient and decoded server-side.

* 🪡 fix: Make Skill Name Uniqueness Application-Level

Resolve three more CI failures from the Skills scaffolding PR:

- Mongoose creates indexes asynchronously and mongodb-memory-server
  tests can race ahead of the unique (name, author, tenantId) index
  being built, so the duplicate-name uniqueness test was flaky.
  Added an explicit findOne pre-check inside createSkill that throws
  with code 11000 (mimicking the index violation), giving deterministic
  behavior. The unique index stays as the persistent guarantee.
- The deleteUser.spec.js and UserController.spec.js suites mock the
  ~/models module directly and were missing deleteUserSkills, causing
  deleteUserController to throw and return 500 instead of 200.
- Removed two doc-comment claims that the SKILL_NAME_MAX_LENGTH and
  SKILL_DESCRIPTION_MAX_LENGTH constants "match Anthropic's API". The
  values themselves are reasonable but the comments were misleading
  about who enforces them.

* 🪢 fix: Address Code Review Findings on Skills Scaffolding

Resolve all 15 findings from the comprehensive PR review:

Critical:
- Rollback the created skill when grantPermission throws so a transient
  ACL failure cannot leave an orphaned, inaccessible skill in the DB.
- Fix infinite query cache corruption in useUpdateSkillMutation helpers.
  setQueriesData([QueryKeys.skills]) matches useSkillsInfiniteQuery's
  InfiniteData cache entries, which have { pages, pageParams } shape —
  spreading data.skills on those would throw. Added an isInfiniteSkillData
  guard and per-page transform so both flat and infinite caches update
  correctly.

Major:
- Fix TUpdateSkillContext type: the public type declared previousListData
  but onMutate actually returns previousListSnapshots (a [key, value]
  tuple array). Updated the type + added TSkillCacheEntry as a shared
  export from data-provider.
- Add cancelQueries calls before optimistic update in onMutate so
  in-flight refetches cannot clobber the optimistic state.
- Parallelize deleteUserSkills ACL removal via Promise.allSettled instead
  of a sequential await loop — O(1) round-trip vs O(n).
- Stub mockDeleteUserSkills in stubDeletionMocks() and assert it's called
  with user.id in the deleteUser.spec.js happy-path test.
- Add idResolver: getSkillById to the SKILL branch in accessPermissions.js
  so GET /api/permissions/skill/<missing-id> returns 404 instead of 403.

Minor:
- Reuse resolved skill from req.resourceAccess.resourceInfo in getHandler
  to eliminate a redundant getSkillById call per GET /api/skills/:id.
- Reject PATCH /api/skills/:id requests whose body contains only
  expectedVersion — previously they silently bumped version with no
  changes, triggering spurious 409s for collaborators.
- Make TSkill.frontmatter optional (wire type) and add serializeFrontmatter
  / serializeSourceMetadata helpers that return undefined for empty
  objects instead of casting incomplete data to SkillFrontmatter.
- Standardize deleteUserSkills to accept string | ObjectId and convert
  internally, matching deleteUserPrompts's signature; UserController now
  passes user.id consistently.
- Replace bumpSkillVersionAndRecount (read-then-write, racy) with
  bumpSkillVersionAndAdjustFileCount using atomic $inc. upsertSkillFile
  pre-checks existence to distinguish insert (+1) from replace (0).
- Add DELETE /api/skills/:id/files/:relativePath integration tests
  covering success, 404, and 403 paths.

Nits:
- Drop trivial resolveSkillId wrapper — pass getSkillById directly.
- Remove dead staleTime: 1000 * 10 from useListSkillsQuery since all
  refetch triggers are already disabled.

* 🧭 fix: Resolve Second Skills Review Pass — Cache, Gate, TOCTOU

Address 13 of 14 findings from the second code review; reject #13 as
misread of the AGENTS.md import-order rule (package types correctly
precede local types regardless of length).

Major:
- Fix addSkillToCachedLists closure bug: a hoisted `prepended` flag
  was shared across every cache entry matched by setQueriesData, so
  concurrent flat + infinite caches would silently drop the prepend
  on whichever was processed second. Replaced the shared helper with
  three per-entry inline updaters that handle InfiniteData at the
  page level (page 0 only for prepend, all pages for replace/remove).
- Tighten patchHandler's expectedVersion validation: NaN passes
  `typeof === 'number'` and would previously leak current skill state
  via a misleading 409. Now requires finite positive integer and
  returns 400 otherwise.
- Guard decodeURIComponent in deleteFileHandler with try/catch —
  malformed percent encoding now returns 400 instead of 500.
- Add PermissionTypes.SKILLS + skillPermissionsSchema +
  TSkillPermissions in data-provider; seed default SKILLS permissions
  for ADMIN (all true) and USER (use + create only); wire
  checkSkillAccess / checkSkillCreate via generateCheckAccess onto
  the skills router mirroring the prompts pattern. Skills route now
  enforces role-based capability gates alongside per-resource ACLs.
  Test suite adds a mocked getRoleByName returning permissive SKILLS.
- Fix upsertSkillFile TOCTOU: replaced the pre-check + upsert pair
  with a single `findOneAndUpdate({ new: false, upsert: true })` call
  that atomically returns the pre-update doc (null ⇒ insert) so
  fileCount delta can't double-count on concurrent same-path uploads.

Minor:
- Add `sourceMetadata` to listSkillsByAccess .select() so summaries
  no longer silently drop the field for GitHub/Notion-synced skills.
- Include `cursor` in useListSkillsQuery's query key so manual
  pagination doesn't alias across pages.
- Clean up TSkillSummary to `Omit<TSkill, 'body' | 'frontmatter'>`
  matching what serializeSkillSummary actually emits; drop the
  Omit-then-re-add noise.
- Skip getPublicSkillIdSet in createHandler; a newly-created skill
  cannot have a PUBLIC ACL entry, so pass an empty set directly
  instead of paying a DB round-trip.
- Trim SkillMethods public surface: drop internal helpers
  countSkillFiles / deleteSkillFilesBySkillId / getSkillFile from the
  return object; inline the file cascade into deleteSkill.
- Use TSkillConflictResponse at the PATCH 409 call site instead of
  an inline ad-hoc object literal.
- Drop the now-unused EXPECTED_VERSION_ERROR module constant.

* 🧩 fix: Extend Role Schema + Types with SKILLS PermissionType

CI type-check and unit test failures from the PermissionTypes.SKILLS
addition surfaced three unrelated places that all hardcode the
permission-type set:

- IRole.permissions in data-schemas/types/role.ts enumerates every
  PermissionTypes key as an optional field. Adding SKILLS to the enum
  without updating the interface caused TS7053 'expression of type
  PermissionTypes can't be used to index type' errors in
  role.methods.spec.ts (lines 407-408, 477-478) because
  Object.values(PermissionTypes) now yielded a value the interface
  didn't cover.
- schema/role.ts rolePermissionsSchema mirrors the interface at the
  Mongoose layer; also needed SKILLS added so the persisted role
  document can actually store skill permissions.
- data-provider/roles.spec.ts has a guard test that every permission
  type carrying CREATE/SHARE/SHARE_PUBLIC must be explicitly "tracked"
  either in RESOURCE_PERMISSION_TYPES or in the PROMPTS/AGENTS/MEMORIES
  exemption list. Added SKILLS to the exemption list since skills
  follow the same default model as prompts/agents (USE + CREATE on for
  USER, SHARE / SHARE_PUBLIC off).

All three are additive pass-throughs with no behavior change.

* 🏷️ refactor: Introduce ISkillSummary for Narrow List Projection

Follow-up NITs from the second review pass on the Skills PR:

- Define ISkillSummary = Omit<ISkill, 'body' | 'frontmatter'> and use
  it as the element type in ListSkillsByAccessResult. The list query's
  .select() intentionally omits body and frontmatter for payload size,
  but the previous type claimed both fields were present — a type lie
  that would mislead future readers even though serializeSkillSummary
  never touches those fields at runtime. handlers.ts's signature for
  serializeSkillSummary now accepts ISkillSummary too.
- Document the intentional second-round-trip `findOne` in
  upsertSkillFile. Switching to `findOneAndUpdate({ new: false })`
  was required for TOCTOU-safe insert-vs-replace detection, which
  means the handler needs a follow-up query to return the post-upsert
  document. A comment now explains the tradeoff so future readers
  don't silently "optimize" it away.

No behavior change.

* 🌐 fix: Wire SKILL into SHARE_PUBLIC Resource Maps

Address codex comment #1 — making a skill public was blocked on two
hardcoded resource→permission-type maps that didn't know about SKILL:

- api/server/middleware/checkSharePublicAccess.js's
  resourceToPermissionType map was missing ResourceType.SKILL, so
  PUT /api/permissions/skill/:id with { public: true } would fall
  through to the 400 "Unsupported resource type for public sharing"
  path even though PermissionTypes.SKILLS exists and ADMIN has
  SHARE_PUBLIC configured. Added the mapping.
- client/src/hooks/Sharing/useCanSharePublic.ts has an identical
  client-side map used to gate the "Make Public" UI toggle. Without
  the SKILL mapping the hook returned false for everyone, so the
  toggle wouldn't render for skills once the sharing UI lands in
  phase 2. Added the mapping.

Codex comment #2 (create/update cache writes inject skills into
unrelated filtered lists) is invalid — it flags a pattern that
mirrors useUpdatePromptGroup (which the PR description explicitly
cites as the model) and is a deliberate optimistic-update tradeoff.
Trying to match each cache key's embedded filter would couple the
mutation callback to query-key internals, which is exactly what
setQueriesData is designed to avoid. No change there.

* 🧪 feat: Frontmatter Validation, Reserved-Name Fixes, Coaching Warnings

Address the follow-up review notes on the Skills PR. This commit closes
the gap between the wire-type promise and what the backend actually
enforces, tightens the reserved-name rules, and adds a non-blocking
coaching tier for validators.

Frontmatter validation (new):
- Add `validateSkillFrontmatter` in data-schemas/methods/skill.ts with
  strict mode — unknown keys are rejected so expanding the allowed set
  is an intentional code change. Known keys are type-checked against a
  `FrontmatterKind` table derived from Anthropic's Agent Skills spec
  (name, description, when-to-use, allowed-tools, arguments,
  argument-hint, user-invocable, disable-model-invocation, model,
  effort, context, agent, paths, shell, hooks, version, metadata).
- `hooks` and `metadata` get a shallow JSON-safety check (max depth 4,
  max string 2000, max array 100) instead of a full schema, since their
  full shapes live outside this module.
- Wired into BOTH createSkill AND updateSkill so the PATCH path can't
  smuggle invalid frontmatter past the validator.

Validation warning tier (new):
- Add optional `severity: 'error' | 'warning'` to `ValidationIssue`
  (defaults to error). `partitionIssues` splits an issue list into
  blocking errors and non-blocking warnings.
- `createSkill` / `updateSkill` filter on errors for the throw check
  and return warnings in a new `warnings: ValidationIssue[]` field on
  their result objects (`CreateSkillResult` / `UpdateSkillResult`).
- `validateSkillDescription` now emits a `TOO_SHORT` warning for
  descriptions under 20 chars — the primary triggering field, so a
  little coaching goes a long way.
- `createHandler` / `patchHandler` in packages/api surface the warnings
  via a new `attachWarnings` helper that decorates the serialized
  response with a `warnings?: TSkillWarning[]` field.
- `TSkill` gains an optional `warnings?: TSkillWarning[]` field
  documented as "present on POST/PATCH, never on GET".

Reserved-name filter (tightened):
- Replace the substring match (`.includes('anthropic')`) with prefix
  matching on `anthropic-` and `claude-` plus exact-match rejection of
  CLI slash-command collisions (help, clear, compact, model, exit,
  quit, settings, plus the bare `anthropic` / `claude` words). Both
  the pure validator (`methods/skill.ts`) and the Mongoose schema
  validator (`schema/skill.ts`) updated in lockstep; comments on
  each reference the other to prevent drift.
- `research-anthropic-helper` and `about-claude` are now allowed;
  `anthropic-helper`, `claude-bot`, and `settings` are still rejected.

Documentation:
- Add docstrings on `ISkill`, `schema/skill.ts`, and `TSkill` explaining
  the semantics of `name` (Claude-visible identifier, kebab-case,
  stable), `displayTitle` (UI-only cosmetic label, NOT sent to Claude),
  `description` (highest-leverage trigger field), and `source` /
  `sourceMetadata` (reserved for phase 2+ external sync).
- Add a detailed consistency comment on `bumpSkillVersionAndAdjustFileCount`
  explaining that it runs as a separate MongoDB operation from
  upsertSkillFile/deleteSkillFile, so `fileCount` can drift if the
  second op fails — options listed, tradeoff documented, phase 1
  risk window noted as closed because upload is still stubbed.

Tests:
- data-schemas skill.spec.ts: destructure `{ skill, warnings }` from
  createSkill at every call site; add a TOO_SHORT warning test, a
  frontmatter strict-mode test, reserved-prefix tests (including
  positive cases for substring names that should pass), CLI reserved
  word tests, and a full `validateSkillFrontmatter` describe block
  covering unknown keys, type mismatches, and deep-nesting rejection.
- api/server/routes/skills.test.js: bump default test description
  above the 20-char threshold, add a warning-emission test, add
  reserved-prefix + reserved-CLI-word tests, add an unknown-frontmatter-
  key test asserting the 400 response carries `issues` with `UNKNOWN_KEY`.

* 📦 fix: Export CreateSkillResult from data-schemas Methods Index

`CreateSkillResult` was defined in `methods/skill.ts` and consumed by
`packages/api/src/skills/handlers.ts` but never re-exported from the
methods barrel, so the type-check job failed with TS2724
"'@librechat/data-schemas' has no exported member named 'CreateSkillResult'".

Rollup's bundle-mode build picked up the type via its internal resolver,
but the standalone `tsc --noEmit` type-check ran against the package's
public entrypoint and couldn't see it. Added the type import + export
alongside the existing `UpdateSkillResult` export, which fixes the
CI type-check without any runtime change.
2026-04-25 04:01:59 -04:00
Danny Avila
f7d59d3285
📦 chore: Update TypeScript Config for TS v7 (#12794)
- Enabled `esModuleInterop` in `client/tsconfig.json` for better module compatibility.
- Changed `moduleResolution` from `node` to `bundler` in `client/tsconfig.json`.
- Set `noEmit` to `true` in several `tsconfig.json` files to prevent output generation.
- Removed `baseUrl` from various `tsconfig.json` files to simplify path resolution.
- Updated path mappings in multiple packages to reflect new directory structures.

These changes aim to streamline TypeScript configurations and improve module resolution across the project.
2026-04-23 12:51:03 -04:00
Danny Avila
9ccc8d9bef
✨ v0.8.5 (#12727) 2026-04-22 13:10:19 -07:00
Danny Avila
181d705579
🧹 fix: Clean Up Orphaned Agent File Stubs After Deletion (#12781)
* 🧹 fix: Prune Orphaned File References on File Deletion

Deleting a file via the Manage Files tab left its file_id in every agent's
tool_resources.*.file_ids. Stubs accumulate until the frontend dedupe keys
them as duplicates and blocks all new uploads (issue #12776).

- Add removeAgentResourceFilesFromAllAgents in packages/data-schemas: a
  single updateMany/$pullAll across every EToolResources category.
- Invoke it from processDeleteRequest after db.deleteFiles so every
  referencing agent is cleaned up, not just the one passed in req.body.
- Wrap the cleanup in try/catch so a stale agent update cannot mask a
  successful file deletion.

* 🧼 fix: Prune Orphaned File References on Agent Update

Already-affected agents would stay broken even after the delete-time fix:
the stubs sit on the agent document until something strips them. Heal them
on the next save (issue #12776).

- Add collectToolResourceFileIds + stripFileIdsFromToolResources helpers
  in @librechat/api — centralizing the tool_resources traversal used by
  the controller and the follow-up migration script.
- In updateAgentHandler, check the effective tool_resources against the
  files collection. When orphans are found, either strip them from the
  incoming tool_resources (if the update sets them) or run the bulk
  cleanup (if the update leaves tool_resources untouched).

* 🧰 chore: Add Migration to Clean Up Orphaned Agent File References

Complements the delete-time and save-time fixes by healing agents that
already accumulated orphan stubs before the upgrade (issue #12776). The
script is idempotent — re-running it on a clean database is a no-op.

- Add config/migrate-orphaned-agent-files.js following the existing
  migrate-*.js convention: --dry-run by default omitted (writes by
  default) and --batch-size= tuning knob. Streams agents via cursor.
- Register migrate:orphaned-agent-files and :dry-run npm scripts.
- Reuse collectToolResourceFileIds from @librechat/api so migration and
  runtime share the same traversal logic.

* 🩹 fix: Address Codex/Copilot Review on Orphaned Agent File Cleanup

Refines the #12776 fix series based on automated review feedback.

- Scope save-time pruning to the current agent only. When a PATCH
  carries tool_resources, strip orphans from the incoming payload and
  pay the DB round-trip only then. Removes the collection-wide
  updateMany previously triggered when tool_resources was absent
  (Codex P2 / Copilot).
- Wrap the orphan check in try/catch so a transient db.getFiles
  failure can't turn a good save into a 500 (comprehensive review #1).
- Replace Object.values(EToolResources) casts with an explicit list of
  agent-side categories in both orphans.ts and agent.ts. code_interpreter
  belongs to the Assistants API and isn't a key of AgentToolResources —
  including it was a type lie and generated dead MongoDB clauses
  (comprehensive review #3, #8).
- Export TOOL_RESOURCE_KEYS from @librechat/api and consume it in the
  migration script, dropping one duplicated definition (#4).
- Cap migration results.details at 50 sample entries so the memory
  footprint stays bounded on deployments with thousands of corrupted
  agents (Codex P3).
- Add migrate:orphaned-agent-files:batch npm script to match the
  convention set by migrate-agent-permissions / migrate-prompt-permissions
  (#7).
- Add controller-level tests covering the three orphan-pruning paths:
  strip from incoming tool_resources, leave alone when tool_resources
  is absent, swallow db.getFiles errors and still save (#6).
- Back pre-existing "should validate tool_resources in updates" test's
  file_ids with real File docs — the new pruning would otherwise strip
  them, and that test is about OCR conversion / schema filtering, not
  file existence. Register the File model in beforeAll so the fixture
  works.

* 🩹 fix: Tighten TOOL_RESOURCE_KEYS Type and Align Migration Sample Output

Two follow-ups from the second review pass.

- Type data-schemas' TOOL_RESOURCE_KEYS as ReadonlyArray<keyof
  AgentToolResources> instead of readonly string[]. Data-schemas depends
  on data-provider, so the import is clean. Catches typos and aligns
  with the matching export in @librechat/api — doesn't guarantee
  exhaustiveness, but that's a TypeScript limitation, not a workspace
  one.
- Align the migration's console output with DETAIL_SAMPLE_LIMIT: print
  every collected detail (up to 50) and, when more agents were affected
  than the sample size allowed, show a truncation notice. The old hard
  cap of 25 meant affected agents in the 26-50 range were collected
  but never shown.

* ✅ test: Add Integration Coverage for Orphan Cleanup Paths (#12776)

Exercise the delete-time and migration paths end-to-end against a real
in-memory Mongo. Catches integration bugs the isolated unit tests on
each layer couldn't.

- api/server/services/Files/process.integration.spec.js — the primary
  repro: seed an Agent + File, call processDeleteRequest, assert the
  file_id disappears from every referencing agent's tool_resources
  while unrelated agents stay untouched. Also covers the no-op case
  and confirms a failure in the new cleanup step cannot roll back the
  file deletion itself.
- api/test/migrate-orphaned-agent-files.spec.js — drives the migration
  module: --dry-run reports without writing, apply mode prunes across
  every tool_resource category, re-running is idempotent, and
  DETAIL_SAMPLE_LIMIT caps the in-memory sample on wide corruption.
  Mocks only the connect helper (the spec owns the mongoose instance)
  so the real migration code path — cursor, $pullAll, reduce — runs.

* 🔒 fix: Run Orphan Cleanup Migration in System Tenant Context

Codex P2 catch: under TENANT_ISOLATION_STRICT=true, the migration
throws on the very first Agent.countDocuments() because the tenant
isolation plugin fail-closes on queries without tenant context — which
makes migrate:orphaned-agent-files unusable on the exact deployments
most likely to have accumulated corruption.

- Wrap the scan/prune body in runAsSystem so queries bypass the tenant
  filter (SYSTEM_TENANT_ID sentinel). The migration legitimately needs
  cross-tenant visibility — this is the same pattern seedDatabase and
  the S3 refresh job already use.
- Add a regression test that spies on Agent.countDocuments() and
  asserts the active tenantStorage context is SYSTEM_TENANT_ID during
  the call. Pins the wrap against future regressions without the
  brittleness of toggling the strict-mode env var (which caches on
  first read).

Note: the delete-time and save-time paths already run inside an
authenticated HTTP request where tenantStorage.run is set by auth
middleware, so the cleanup naturally scopes to the current tenant —
which is the correct behavior there since file ownership is
tenant-scoped.

* 🧹 chore: Drop Unused path Import From Process Integration Spec

Leftover from an earlier iteration that resolved the migration path
via path.resolve before I switched to a relative require. The import
does nothing now — removing it.
2026-04-22 11:35:48 -07:00
Danny Avila
ccf3a6c670
📐 fix: Align Summarization Trigger Schema with Documented and Runtime-Supported Types (#12735)
* 🐛 fix: accept documented `summarization.trigger.type` values

The Zod schema for `summarization.trigger.type` only accepted
`'token_count'`, but:

- the documentation lists `token_ratio`, `remaining_tokens`, and
  `messages_to_refine` as valid
- the `@librechat/agents` runtime only evaluates those three types and
  silently no-ops on anything else

The result was a double failure: any user following the docs hit a
startup Zod error, and anyone who matched the schema by using
`token_count` got a silent no-op at runtime where summarization never
fired.

Align the schema with the documented, runtime-supported trigger types.

Closes #12721

* 🧹 fix: bound `token_ratio` trigger value to (0, 1]

Per Codex review: the previous schema accepted `value: z.number().positive()`
for every trigger type. That meant `trigger: { type: 'token_ratio', value: 80 }`
(presumably meant as "80%") passed validation and then silently never fired —
because `usedRatio = 1 - remaining/max` is bounded at 1, so `>= 80` is always
false. That is exactly the silent-no-op pattern this PR is trying to eliminate.

Switch to a discriminated union so each trigger type has its own value
constraint:

- `token_ratio`: `(0, 1]` — documented as a fraction, so 80 is nonsense
- `remaining_tokens`: positive — token counts can be large
- `messages_to_refine`: positive — message counts can be > 1

Added tests for the upper-bound rejection and the inclusive upper bound
(`value: 1` still accepted as a valid "fire at 100%" extreme).

* 🧹 fix: accept `token_ratio: 0` per documented 0.0–1.0 inclusive range

Per Codex review: `.positive()` rejected `value: 0`, but the docs
describe the `token_ratio` range as `0.0–1.0` (both inclusive). Admins
who copy the documented lower bound into their YAML would fail schema
validation at startup.

Switch `token_ratio` to `.min(0).max(1)`. `0` is a valid (if extreme)
setting — the agents SDK's `usedRatio >= 0` check will fire as soon as
there is anything to refine, which is a legitimate "always summarize
when pruning happens" configuration.

`remaining_tokens` and `messages_to_refine` keep `.positive()`: both
are counts, and `0` there produces no meaningful behavior (the SDK
has an early return for `messagesToRefineCount <= 0`).

* 🐛 fix: preserve `token_ratio` trigger when `value: 0`

Per Codex review: now that the schema accepts `token_ratio: 0`,
`shapeSummarizationConfig` would silently drop it because of a truthy
check on `config?.trigger?.value`. The trigger would disappear and the
runtime would fall back to "no trigger configured" — which fires on any
pruning rather than honoring the explicit ratio.

Switch to `typeof value === 'number'`, which preserves `0` while still
rejecting `undefined`/`null`.

Added a regression test that asserts `{ type: 'token_ratio', value: 0 }`
survives the shaping function untouched.

* 🧹 fix: reject non-finite trigger values at schema level

Per Codex review: `z.number().positive()` still accepts `Infinity` and
`NaN` (via YAML `.inf`, `.nan`). Config validation would succeed, but
the agents SDK guards every trigger path with `Number.isFinite(...)` and
silently returns `false` — summarization never fires while the server
starts cleanly. That is the exact schema/runtime split this PR is trying
to eliminate.

Add `.finite()` to every trigger value. `token_ratio` already had an
implicit guard via `.max(1)`, but applying `.finite()` uniformly keeps
the intent obvious and catches `NaN` (which `.max(1)` does not).

* 🧹 fix: integer counts + targeted token_count migration warning

Two findings from the comprehensive review:

1. `remaining_tokens` and `messages_to_refine` are token/message counts
   and are always integers in the runtime (`Number.isFinite(...)` guards
   already assume integer semantics). `z.number().positive()` accepted
   fractional values like `2.5`, which was semantically confusing and
   would round oddly against the runtime's `>=` / `<=` comparisons. Add
   `.int()` to both count-based branches; `token_ratio` stays fractional.

2. Anyone upgrading with `trigger.type: 'token_count'` in their YAML got
   the generic "Invalid summarization config" warning plus a flattened
   Zod error. Detect that specific case in `loadSummarizationConfig` and
   emit a migration-friendly message that names the three valid
   replacements. Export the function so the behavior is unit-testable.

Also added a parameterized passthrough test covering `remaining_tokens`
and `messages_to_refine` shaping, complementing the existing
`token_ratio` coverage.

* 🧹 fix: accurate fallback wording + bare-string trigger test

Two nits from the follow-up audit:

1. The legacy-`token_count` warning claimed "Summarization will be
   disabled," but `shapeSummarizationConfig` treats a missing
   summarization config as self-summarize mode (fires on every pruning
   event using the agent's own provider/model). "Disabled" would
   mislead an admin into stopping investigation. Reword to describe the
   actual fallback and assert the new wording in the spec.

2. Add a regression test for the `trigger: 'bare-string'` YAML case, so
   the `typeof raw.trigger === 'object'` guard is exercised rather than
   implied.

3. Swap the en-dash in `(0–1)` for an ASCII hyphen so the log message
   is safe in every terminal/aggregator regardless of UTF-8 handling.

* 🔇 fix: cast `raw.trigger.type` to inspect legacy value past narrowed union

CI TS check failed: after the schema tightening, `raw.trigger.type` is
narrowed to `"token_ratio" | "remaining_tokens" | "messages_to_refine"
| undefined`, so the runtime comparison to `"token_count"` is a
TS2367 ("no overlap") error even though that's exactly the comparison
we want for the migration guard.

Widen just that one access via `as { type?: unknown }` so the migration
check reads runtime-shaped YAML input without the type system folding
it back into the narrowed union.
2026-04-19 19:33:52 -07:00
Danny Avila
3bd2681272
🪐 fix: Replace $bitsAllSet ACL Queries for Azure Cosmos DB Compatibility (#12736)
* 🐛 fix: replace `$bitsAllSet` ACL queries for Cosmos DB compatibility

Azure Cosmos DB for MongoDB API does not implement the `$bitsAllSet`
query operator, so every permission check against Cosmos DB threw. The
five read paths in `aclEntry.ts` (`hasPermission`, `findAccessibleResources`,
`findPublicResourceIds`, and two sites in `getSoleOwnedResourceIds`) now
fetch candidate entries and apply the bitwise mask in application code.
This matches the existing FerretDB-compatible pattern.

Fixes #12729.

* 🐛 fix: delegate `findPubliclyAccessibleResources` to fixed DB method

`AccessControlService.findPubliclyAccessibleResources` inlined the same
`$bitsAllSet` query as the data-schemas layer, which fails on Azure
Cosmos DB for MongoDB. Delegate to `_dbMethods.findPublicResourceIds`
so a single implementation carries the Cosmos-compatible bitwise logic.

Refs #12729.

* 🐛 fix: move `$bitsAllSet` filter out of remote-agent aggregation

`enrichRemoteAgentPrincipals` used `$bitsAllSet` inside an aggregation
`$match`, which Azure Cosmos DB for MongoDB does not implement. Project
`permBits` through the pipeline and filter for `PermissionBits.SHARE` in
application code. The extra documents fetched are bounded by ACL entries
on a single agent resource, so the cost is negligible.

Refs #12729.

* 🧪 test: rename misleading public-dedup test and add real dedup coverage

The test previously named "returns deduplicated IDs even if the public
principal has multiple entries" only set up a single ACL entry, so it
did not actually exercise deduplication. Split into two tests: one for
the happy path (single entry with required bits), and one that bypasses
`grantPermission`'s upsert via `AclEntry.create` to confirm the
application-layer dedup Map handles genuine duplicates.

Refs #12729.

* 🧪 test: cover SHARE-bit filter in `enrichRemoteAgentPrincipals`

The `$bitsAllSet` match stage in `enrichRemoteAgentPrincipals` previously
guaranteed every aggregation row had SHARE; the Cosmos DB fix moved that
check into a JS `continue` branch with no direct coverage. Add a
dependency-injected unit test that stubs the aggregation with mixed
SHARE / non-SHARE / zero-bit rows and asserts only SHARE holders are
enriched and queued for backfill. Also includes a regression guard that
the `$match` pipeline stage no longer contains a `permBits` filter.

Refs #12729.

* ♻️ refactor: extract `filterByBitsAndDedup` helper for ACL reads

`findAccessibleResources` and `findPublicResourceIds` each inlined the
same bitmask-filter + `Map`-based dedup loop. Lift it into a private
`filterByBitsAndDedup(entries, requiredBits)` helper so the Cosmos-DB
compatible pattern lives in one place. Pure rename/extract — no
behavior change.

Refs #12729.

* 📝 docs: fix stale `\$bitsAllSet` references in FerretDB spec

The describe block and header comment in the FerretDB parity spec still
referenced `\$bitsAllSet queries` after the Cosmos DB compatibility fix
moved the bit mask into application code. Update the title to
\"Bitwise permission queries\" and rewrite the header comment to
describe the application-layer behavior being validated.

Refs #12729.

* ⚡ perf: push permission-mask filter back to the database via `$in`

The original fix for #12729 moved `$bitsAllSet` filtering into application
code, which meant every ACL read fetched the full set of rows for a
principal/resource and filtered in JS. For tenants with large ACL
collections this inflates wire transfer and heap.

Replace the JS filter with `permBits: { $in: permissionBitSupersets(X) }`.
For the 4-bit `PermissionBits` enum the `$in` list is at most 16 values
(8 for a single-bit mask like SHARE). `$in` is indexable and supported by
Azure Cosmos DB for MongoDB, so the filter runs on the server again —
restoring `.distinct('resourceId')` and `findOne()` semantics.

`permissionBitSupersets(requiredBits)` is memoized and exported from
`@librechat/data-schemas`. Callers restored:
  - `hasPermission`: back to `findOne` short-circuit
  - `findAccessibleResources` / `findPublicResourceIds`: back to `.distinct()`
  - `getSoleOwnedResourceIds`: back to the `$match` + `$group` aggregation
  - `enrichRemoteAgentPrincipals`: bit filter back in `$match`, JS `continue` removed

Refs #12729.

* 🧪 test: add `\$bitsAllSet` vs `\$in` parity + perf spec

Introduces `aclEntry.parity.spec.ts` — a side-by-side spec that runs the
legacy `\$bitsAllSet` query and the current `\$in`-based query against the
same `mongodb-memory-server` fixture and asserts identical output sets for
every affected method (`hasPermission`, `findAccessibleResources`,
`findPublicResourceIds`, `getSoleOwnedResourceIds`) across all 7 meaningful
permBits combinations.

Also logs median wall-clock time for the two query paths over 20 runs on
an 800-entry fixture, with a loose 3x guard against catastrophic
regressions. Initial local numbers: 1.05 ms vs 1.07 ms
(findAccessibleResources), 1.10 ms vs 1.05 ms (findPublicResourceIds).

Refs #12729.

* 🔒 hardening: freeze `permissionBitSupersets` cache + enum-shape guard

Two defensive changes from the comprehensive audit:

* Cached superset arrays are now `Object.freeze`d and the return type is
  `readonly number[]`. Previously the cached arrays were returned by
  mutable reference, so a caller that mutated the result would silently
  corrupt the process-wide cache for every subsequent permission check.
  `Object.freeze` turns that into a loud `TypeError` at the mutation
  site. All existing call sites pass the result directly to Mongoose's
  `$in`, which does not mutate.
* Added a module-load guard `if (MAX_PERM_BITS === 0) throw`. If
  `PermissionBits` is ever refactored to a `const` object or string
  enum, `Object.values(...).filter(isNumber)` would return `[]` and
  `MAX_PERM_BITS` would silently become 0, making every query match no
  rows and breaking every permission check. The guard fails loudly
  instead.

Also collapsed four identical JSDoc lines across `hasPermission`,
`findAccessibleResources`, `findPublicResourceIds`, and
`getSoleOwnedResourceIds` into a single `{@link permissionBitSupersets}`
reference.

Refs #12729.

* 🧪 test: add focused unit tests for `permissionBitSupersets`

The helper is the single point of correctness for every ACL read path
(every query uses `permBits: { $in: permissionBitSupersets(X) }`), so it
warrants direct coverage independent of the higher-level parity and
behavior specs. Six cases added:

* `requiredBits=0` returns all 16 values
* `requiredBits=15` returns `[15]` only
* every returned value is a bitwise superset of `requiredBits`
* full parity against the `$bitsAllSet` definition for every
  `required` in 0..15
* memoization: repeat calls return the same frozen reference
* frozen result throws `TypeError` on mutation attempts

Refs #12729.

* 🧪 test: tighten parity perf guard and document fixture constants

The `expect(currentMs).toBeLessThan(legacyMs * 3 + 50)` form was
dominated by the `+ 50` additive term at typical sub-ms query
latencies — at legacy=1ms, a 50x regression would still pass. Replace
with `Math.max(legacyMs * 5, 50)` so the multiplicative ceiling is
intact once the new path climbs out of the fixed noise floor.

Also added inline rationale for the `FIXTURE_SIZE = 800` and
`PERF_ITERATIONS = 20` constants.

Refs #12729.

* 🧹 chore: remove stale perf-guard comment, hoist rationale to describe

Commit a75f122 left a "Sanity check: A 3x multiplier" comment block
above the first perf guard, and 21852ac layered a second
"Multiplicative-dominant guard" block directly below it. The stale
block described the `legacyMs * 3 + 50` formula that no longer exists,
so both blocks coexisted and contradicted each other.

Delete the stale block from the first test, remove the redundant copy
from the second test, and lift the (now-single) rationale to the
enclosing `describe('performance')` block. Each test is now one
`expect` line — the rationale lives once, at the scope it applies to.

Refs #12729.

* 📝 docs: sharpen MAX_PERM_BITS guard message + test-sort consistency

Two NITs from the follow-up audit:

* The `MAX_PERM_BITS === 0` guard now names the specific refactors that
  could cause it (const object / string enum / all-string shape) and
  gives two concrete remediation paths (rewrite the reducer, or give
  `permissionBitSupersets` an explicit bit-width parameter). Previously
  the message just said "update permissionBitSupersets before
  continuing", which was vague.
* The `requiredBits=15` unit test now applies the same
  `[...result].sort((a, b) => a - b)` normalization as the other tests
  so the set-equality assertions are uniform. The function happens to
  return values in ascending order, but the helper's JSDoc does not
  promise ordering, so sorting before comparing is the correct
  defensive pattern.

Refs #12729.

* 🔒 fix: enforce `permBits <= MAX_PERM_BITS` at the schema level

Codex flagged a real silent regression: `permissionBitSupersets`
enumerates `$in` candidates only in `[0, MAX_PERM_BITS]` (currently 15),
so any ACL row with bits above the enum — e.g. `permBits = 31` from a
future role or a manual DB write — would be excluded from reads that
`$bitsAllSet` would have matched. Current write paths only produce
values in `[0, 15]` via the `PermissionBits` / `RoleBits` enums, so
no live data is affected, but the schema did not prevent out-of-range
writes, so the divergence was reachable.

Fix: add `min: 0`, `max: MAX_PERM_BITS`, and an `isInteger` validator to
the `permBits` field in the AclEntry schema. `MAX_PERM_BITS` is derived
from `PermissionBits` the same way `permissionBitSupersets` computes it,
so when a new bit is added to the enum both the read-side enumeration
and the write-side bound expand together.

Tests: four new cases cover the upper bound, over-limit, negative, and
non-integer inputs, each asserting `Mongoose.Error.ValidationError`.
Plus updated JSDoc on `permissionBitSupersets` to document the
invariant that the schema now enforces.

Refs #12729.

* ♻️ refactor: hoist `MAX_PERM_BITS` + enum-shape/ceiling guards to shared util

Addresses one MINOR DRY finding and one NIT defensive-guard finding from
the latest audit. `MAX_PERM_BITS` was duplicated in `methods/aclEntry.ts`
and `schema/aclEntry.ts` with identical computations; they could silently
diverge. Move the constant plus both module-load guards to
`src/common/permissions.ts`:

* `MAX_PERM_BITS === 0` guard — fail loudly if `PermissionBits` is ever
  refactored to a `const` object or string enum (applied in both sites
  now, not just methods).
* `MAX_PERM_BITS > 255` guard — circuit-breaker for the `$in` enumeration
  strategy. At 4 bits the list tops out at 16; at 8 bits 256. Beyond
  that the approach degrades, so fail at module load rather than emit an
  unusably-large `$in` list.

Both the schema and the methods file now import from the single
`src/common/permissions.ts`. `readonly number[]` return type and
`Object.freeze` cache-value protection from the prior commits are
preserved.

Refs #12729.

* 🔒 fix: reject out-of-range `requiredBits` without caching (DoS fix)

Codex flagged a real unbounded-cache DoS vector:
`api/server/controllers/agents/v1.js` parses `req.query.requiredPermission`
via `parseInt` and forwards it unvalidated to `findAccessibleResources`,
which eventually calls `permissionBitSupersets(requiredBits)`. Because
the old code memoized *every* distinct input in a process-global Map,
an attacker could flood the cache with unique integers and grow memory
without bound.

Fix: when `requiredBits` is not an integer, is negative, or has any bits
set above `MAX_PERM_BITS`, return a single shared frozen empty array and
do NOT touch the cache. An empty `$in` list correctly matches zero rows
(rows cannot satisfy a bit we do not support), so the response is also
semantically correct — previously it "worked" only because invalid
inputs coincidentally expanded to `[]` too, but at the cost of a cache
entry each time.

Five new tests exercise the rejection path: upper-bound overflow, mixed
in-range+out-of-range bits, shared-instance identity across rejected
inputs, a 2000-unique-value cache-growth probe, and a regression check
that legitimate in-range inputs still get memoized normally.

Refs #12729.

* 🧪 test: probe `Map.prototype.set` to prove cache doesn't grow on rejection

Strengthens the DoS cache-growth test the review pass flagged: reference
identity alone allows a hypothetical regression like
`supersetCache.set(requiredBits, EMPTY_SUPERSETS); return EMPTY_SUPERSETS;`
to pass while still leaking one entry per attacker request.

Add a second test that spies on `Map.prototype.set`, records the global
call count before and after a burst of 1500 rejected inputs (500 each of
over-MAX integers, negatives, and non-integers), and asserts the delta
is zero. Manually verified the test has teeth: injecting the
hypothetical regression in the implementation produced exactly 1500
extra `Map.set` calls and the new test failed as expected; reverting
restored the clean state.

Renames the existing identity-based test to `(reference identity)` so
its scope is clear alongside the new `(Map-write probe)` companion.

Refs #12729.
2026-04-19 22:28:48 -04:00
Danny Avila
e2e3284713
🦉 feat: Claude Opus 4.7 Model Support (#12698)
* 🦉 feat: Claude Opus 4.7 Model Support

- Add `claude-opus-4-7` to shared Anthropic models and `anthropic.claude-opus-4-7` to Bedrock models
- Register 1M context window and 128K max output in anthropic token maps
- Add token pricing ($5/$25), cache rates (6.25/0.5), and premium tier ($10/$37.50 above 200K) in tx.ts
- Update `.env.example` with Opus 4.7 IDs in `ANTHROPIC_MODELS` and `BEDROCK_AWS_MODELS` examples
- Add parallel Opus 4.7 test cases for token/cache/premium rates, context length, max output, name-variation matching, and 1M-context qualification

* feat: Add `xhigh` Effort Level for Opus 4.7

- Add `xhigh` variant to `AnthropicEffort` enum between `high` and `max`
- Expose `xhigh` in `anthropicSettings.effort.options` and the UI slider `enumMappings`
- Reuse existing `com_ui_xhigh` translation key

* test: Cover `xhigh` Effort and Exact Opus 4.7 Premium Rates

- Assert `xhigh` position (between high and max), inclusion in
  `anthropicSettings.effort.options`, zod acceptance, and rejection of
  unknown values in schemas.spec.ts
- Verify bedrockInputParser emits `output_config: { effort: 'xhigh' }`
  for adaptive `anthropic.claude-opus-4-7`
- Verify getLLMConfig sets adaptive thinking and `output_config.effort =
  'xhigh'` for `claude-opus-4-7`
- Pin Opus 4.7 premium pricing to exact threshold/prompt/completion
  values (200000 / 10 / 37.5) so silent rate drift fails the test
2026-04-16 14:51:00 -04:00
Airam Hernández Hernández
277fdd2b43
🪪 feat: Optimized Entra ID Group Sync with Auto-Creation (#12606)
* feat: implement optimized Entra group sync with auto-creation

## Changes

### MUST FIX (Critical Issues) - RESOLVED

1. **BUG FIX: Prevent unintended user removal from existing groups**
   - ISSUE: db.syncUserEntraGroups() was called with only missing groups, causing removal
     from all existing Entra groups (full bidirectional sync behavior)
   - SOLUTION: Replaced with db.upsertGroupByExternalId() for each missing group followed
     by single bulkUpdateGroups() to add memberships (race-safe, idempotent)
   - BENEFIT: User memberships correctly maintained for mix of existing + new groups

2. **JSDoc @throws contradiction**
   - ISSUE: JSDoc declared function throws, but implementation catches all errors
   - SOLUTION: Removed @throws from JSDoc - function is best-effort
   - BENEFIT: Prevents unnecessary try/catch in caller code

3. **Missing test for group creation flow**
   - ISSUE: Auto-creating missing Entra groups had no test coverage
   - SOLUTION: Added regression test for mix of existing + new groups scenario
   - BENEFIT: Prevents future regressions on critical path

### SHOULD FIX (Important Improvements) - RESOLVED

4. **E11000 race condition handling**
   - SOLUTION: Upserts are idempotent and race-safe by design
   - BENEFIT: Concurrent logins no longer race each other

5. **Direct Mongoose access instead of db layer**
   - SOLUTION: Added findGroupsByExternalIds() helper to userGroup.ts
   - BENEFIT: Centralized data access, easier to add tenant scoping

6. **Serial DB round-trips on login path**
   - ISSUE: 40+ queries for user with 20 new groups
   - SOLUTION: Promise.all() for parallel upserts + single bulkUpdate
   - BENEFIT: ~10x performance improvement

7. **Graph API 429/503 throttling unhandled**
   - SOLUTION: Retry logic with exponential backoff (1s, 2s delays)
   - BENEFIT: Temporary API issues no longer cause permanent membership loss

8. **Sequential batch requests slow**
   - ISSUE: 200 groups = 10 batches × 200ms = ~2s sequential
   - SOLUTION: Promise.all() with concurrency limit (5 parallel batches)
   - BENEFIT: ~400ms total time

## Minor Fixes

- Removed dead code check
- PII removal: user._id instead of user.email in logs
- ES6 shorthand fixes
- Style consistency (blank lines)
- Projection optimization

## Verification

✅ npm run build - success
✅ npm run test:api - 61/61 passing (+ new regression test)
✅ npm run lint - no errors
✅ All feedback from danny-avila resolved

* docs: better JSDoc for the syncUserEntraGroupMemberships method

---------

Co-authored-by: Airam Hernández Hernández <airam.hernandez@intelequia.com>
2026-04-13 08:50:52 -04:00
Danny Avila
4f133f8955
✨ v0.8.5-rc1 (#12569) 2026-04-09 20:06:31 -04:00
Marco Beretta
1a83f36cda
📌 feat: Add Pin Support for Model Specs (#11219)
* feat: Enhance favorites functionality to support model specs

* refactor(FavoritesList): reorder imports based on lenght

* feat: improve favorite modelSpec controller; refactor: useIsActiveItem hook

* refactor: consolidate Favorite type, harden controller, add tests

- Add canonical TUserFavorite type in data-provider, replace three
  duplicate definitions (data-service, data-schemas, store/favorites)
- Consolidate FavoritesController spec validation into single block,
  add return on 500 paths, add maxlength to mongoose sub-schema
- Fix import order in FavoritesList.tsx, merge namespace type imports
  in FavoriteItem.tsx and FavoritesList.tsx
- Add focus-visible:ring-inset on pin buttons to prevent ring clipping
- Add explicit return type and JSDoc on useIsActiveItem hook
- Use props.type for narrowing consistency in FavoriteItem getTypeLabel
- Add 22 backend tests for FavoritesController (spec validation,
  typeCount exclusivity, persistence, GET path)
- Add 40 frontend tests: useFavorites spec methods, useIsActiveItem
  observer lifecycle, ModelSpecItem pin button, FavoriteItem all three
  type branches, FavoritesList spec rendering

* fix: address PR review findings for pin model specs

- Harden backend validation to reject partial cross-type fields
  (e.g. spec+endpoint, agentId+model without endpoint)
- Add stale-spec auto-cleanup in FavoritesList mirroring agent cleanup
- Add type="button" to pin buttons in ModelSpecItem/EndpointModelItem
- Fix import order violations in EndpointModelItem and ModelSpecItem
- Remove hollow test, dead key prop, inline trivial helpers
- Fix misleading test description, add onSelectSpec to test mock
- Add return to controller success responses for consistency
- Add 6 backend tests for partial cross-type field validation

* fix: guard stale-spec cleanup against unloaded startupConfig

Prevents race condition where spec favorites are incorrectly deleted
on cold start before startupConfig has loaded. Mirrors the existing
agentsMap === undefined guard pattern used for stale agent cleanup.

Also adds tests for stale-spec cleanup persistence and fixes namespace
import pattern in FavoritesList.spec.tsx.

* fix: replace nested ternaries with if/else in FavoriteItem

Resolves ESLint no-nested-ternary warnings for name and typeLabel
derivations.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-04-09 18:37:25 -04:00
Danny Avila
632ffbcb87
🧬 fix: Merge Custom Endpoints by Name Instead of Replacing Entire Array (#12586)
* fix: Merge Custom Endpoints by Name Instead of Replacing Entire Array

The DB base config's `endpoints.custom` array was wholesale-replacing
the YAML-derived array, causing YAML endpoint additions to be silently
lost after the first admin panel save. Add path-aware array merging
to `deepMerge` so keyed arrays (matched by `name`) are merged item-by-item
instead of replaced.

* fix: Harden mergeArrayByKey — deduplicate, sanitize, and prevent mutation

- Remove redundant sourceOrder array; iterate Map.keys() instead to
  prevent duplicate entries when source contains repeated names.
- Sanitize override-only appended items through deepMerge to enforce
  UNSAFE_KEYS prototype-pollution protection on all code paths.
- Shallow-copy unmatched base items to prevent mutation leak-back.
- Add post-OVERRIDE_KEY_MAP remapping note to ARRAY_MERGE_KEYS JSDoc.
- Add tests: duplicate source names, base mutation safety, multi-priority
  sequential merging of the same custom endpoint.

* fix: Add inline comments and test for keyless source items

- Document keyless item drop behavior in mergeArrayByKey with inline
  comment and matching test case.
- Add last-write-wins comment to deduplication test assertion.
- Clarify path semantics in mergeArrayByKey target-iteration comment.

* fix: Relocate keyless-item comment and test target-side preserve

- Move keyless-item comment to the if-guard where the skip happens and
  clarify that target-side keyless items are preserved, not dropped.
- Add test verifying base items without a name field are kept in output.
2026-04-09 09:38:26 -04:00
Danny Avila
33ee7dea1e
🔎 fix: Specify Explicit Primary Key for Meilisearch Document Operations (#12542)
* fix: pass explicit primaryKey to Meilisearch addDocuments/updateDocuments calls

Meilisearch v1.0+ refuses to auto-infer the primary key when a document
contains multiple fields ending with 'id'. The messages index has both
conversationId and messageId, causing addDocuments to silently fail with
index_primary_key_multiple_candidates_found, leaving message search empty.

Pass { primaryKey } to addDocumentsInBatches, addDocuments, and
updateDocuments — the variable was already in scope.

Also replace raw this.collection.updateMany with Mongoose Model.updateMany
to satisfy the no-restricted-syntax ESLint rule (tenant isolation guard).

Closes #12538

* fix: resolve additional Meilisearch plugin bugs found in review

Address review findings from PR #12542:

- Fix deleteObjectFromMeili using MongoDB _id instead of the Meilisearch
  primary key (conversationId/messageId), causing post-remove cleanup to
  silently no-op and leave orphaned documents in the index.

- Pass options.primaryKey explicitly to createMeiliMongooseModel factory
  instead of deriving it from attributesToIndex[0] (schema field order),
  eliminating a fragile implicit contract.

- Fix updateObjectToMeili skipping preprocessObjectForIndex, which meant
  updates bypassed content array-to-text conversion and conversationId
  pipe character escaping.

- Change collection.updateMany to collection.updateOne in addObjectToMeili
  since _id is unique (semantic correctness).

- Add primaryKey to validateOptions required keys.

- Strengthen test assertions to verify { primaryKey } argument is passed
  to addDocuments, addDocumentsInBatches, and updateDocuments. Add tests
  for the update path including preprocessObjectForIndex pipe escaping.

* fix: add regression tests for delete and message update paths

Address follow-up review findings:

- Add test for deleteObjectFromMeili verifying it uses messageId (not
  MongoDB _id) when calling index.deleteDocument, guarding against
  regression of the silent orphaned-document bug.

- Add test for message model update path asserting { primaryKey:
  'messageId' } is passed to updateDocuments (previously only the
  conversation model update path was tested).

- Add @param config.primaryKey to createMeiliMongooseModel JSDoc.
2026-04-03 18:01:06 -04:00
Danny Avila
b44ce264a4
📦 chore: Bump mongodb-memory-server to v11.0.1, mermaid to v11.14.0, npm audit (#12543)
* 🔧 chore: Update `mongodb-memory-server` to v11.0.1

- Bump `mongodb-memory-server` version in `package-lock.json`, `api/package.json`, and `packages/data-schemas/package.json` from 10.1.4 to 11.0.1.
- Update related dependencies in `mongodb-memory-server` and `mongodb-memory-server-core` to ensure compatibility with the new version.
- Adjust `tslib` version in `mongodb-memory-server` to 2.8.1 and `debug` to 4.4.3 for consistency.

* chore: npm audit fix

* chore: Update `mermaid` dependency to version 11.14.0 in `package-lock.json` and `client/package.json`

* fix: use deterministic timestamps in convoStructure test

MongoDB 8.x (from mongodb-memory-server v11) no longer guarantees
insertion-order return for documents with identical timestamps.
Use sequential timestamps with overrideTimestamp to ensure buildTree
processes parents before children.
2026-04-03 17:01:11 -04:00
Danny Avila
ea28dbfa89
🧹 chore: Clean Up Config Fields (#12537)
* chore: remove unused `interface.endpointsMenu` config field

* chore: address review — restore JSDoc UI-only example, add Zod strip test

* chore: remove unused `interface.sidePanel` config field

* chore: restrict fileStrategy/fileStrategies schema to valid storage backends

* fix: use valid FileStorage value in AppService test

* chore: address review — version bump, exhaustiveness guard, JSDoc, configSchema test

* chore: remove debug logger.log from MessageIcon render path

* fix: rewrite MessageIcon render tests to use render counting instead of logger spying

* chore: bump librechat-data-provider to 0.8.407

* chore: sync example YAML version to 1.3.7
2026-04-03 12:22:58 -04:00
Danny Avila
aa575b274b
🛡️ refactor: Self-Healing Tenant Isolation Update Guard (#12506)
* refactor: self-healing tenant isolation update guard

Replace the strict throw-on-any-tenantId guard with a
strip-or-throw approach:

- $set/$setOnInsert: strip when value matches current tenant
  or no context is active; throw only on cross-tenant mutations
- $unset/$rename: always strip (unsetting/renaming tenantId
  is never valid)
- Top-level tenantId: same logic as $set

This eliminates the entire class of "tenantId in update payload"
bugs at the plugin level while preserving the cross-tenant
security invariant.

* test: update mutation guard tests for self-healing behavior

- Convert same-tenant $set/$setOnInsert tests to expect silent
  stripping instead of throws
- Convert $unset test to expect silent stripping
- Add cross-tenant throw tests for $set, $setOnInsert, top-level
- Add same-tenant stripping tests for $set, $setOnInsert, top-level
- Add $rename stripping test
- Add no-context stripping test
- Update error message assertions to match new cross-tenant message

* revert: remove call-site tenantId stripping patches

Revert the per-call-site tenantId stripping from #12498 and
the excludedKeys patch from #12501. These are no longer needed
since the self-healing guard handles tenantId in update payloads
at the plugin level.

Reverted patches:
- conversation.ts: delete update.tenantId in saveConvo(),
  tenantId destructuring in bulkSaveConvos()
- message.ts: delete update.tenantId in saveMessage() and
  recordMessage(), tenantId destructuring in bulkSaveMessages()
  and updateMessage()
- config.ts: tenantId in excludedKeys Set
- config.spec.ts: tenantId in excludedKeys test assertion

* fix: strip tenantId from update documents in tenantSafeBulkWrite

Mongoose middleware does not fire for bulkWrite, so the plugin-level
guard never sees update payloads in bulk operations. Extend
injectTenantId() to strip tenantId from update documents for
updateOne/updateMany operations, preventing cross-tenant overwrites.

* refactor: rename guard, add empty-op cleanup and strict-mode warning

- Rename assertNoTenantIdMutation to sanitizeTenantIdMutation
- Remove empty operator objects after stripping to avoid MongoDB errors
- Log warning in strict mode when stripping tenantId without context
- Fix $setOnInsert test to use upsert:true with non-matching filter

* test: fix bulk-save tests and add negative excludedKeys assertion

- Wrap bulkSaveConvos/bulkSaveMessages tests in tenantStorage.run()
  to exercise the actual multi-tenant stripping path
- Assert tenantId equals the real tenant, not undefined
- Add negative assertion: excludedKeys must NOT contain tenantId

* fix: type-safe tenantId stripping in tenantSafeBulkWrite

- Fix TS2345 error: replace conditional type inference with
  UpdateQuery<Record<string, unknown>> for stripTenantIdFromUpdate
- Handle empty updates after stripping (e.g., $set: { tenantId } as
  sole field) by filtering null ops from the bulk array
- Add 4 tests for bulk update tenantId stripping: plain-object update,
  $set stripping, $unset stripping, and sole-field-in-$set edge case

* fix: resolve TS2345 in stripTenantIdFromUpdate parameter type

Use Record<string, unknown> instead of UpdateQuery<> to avoid
type incompatibility with Mongoose's AnyObject-based UpdateQuery
resolution in CI.

* fix: strip tenantId from bulk updates unconditionally

Separate sanitization from injection in tenantSafeBulkWrite:
tenantId is now stripped from all update documents before any
tenant-context checks, closing the gap where no-context and
system-context paths passed caller-supplied tenantId through
to MongoDB unmodified.

* refactor: address review findings in tenant isolation

- Fix early-return gap in stripTenantIdFromUpdate that skipped
  operator-level tenantId when top-level was also present
- Lazy-allocate copy in stripTenantIdFromUpdate (no allocation
  when no tenantId is present)
- Document behavioral asymmetry: plugin throws on cross-tenant,
  bulkWrite strips silently (intentional, documented in JSDoc)
- Remove double JSDoc on injectTenantId
- Remove redundant cast in stripTenantIdFromUpdate
- Use shared frozen EMPTY_BULK_RESULT constant
- Remove Record<string, unknown> annotation in recordMessage
- Isolate bulkSave* tests: pre-create docs then update with
  cross-tenant payload, read via runAsSystem to prove stripping
  is independent of filter injection

* fix: no-op empty updates after tenantId sanitization

When tenantId is the sole field in an update (e.g., { $set: { tenantId } }),
sanitization leaves an empty update object that would fail with
"Update document requires atomic operators." The updateGuard now
detects this and short-circuits the query by adding an unmatchable
filter condition and disabling upsert, matching the bulk-write
handling that filters out null ops.

* refactor: remove dead logger.warn branches, add mixed-case test

- Remove unreachable logger.warn calls in sanitizeTenantIdMutation:
  queryMiddleware throws before updateGuard in strict+no-context,
  and isStrict() is false in non-strict+no-context
- Add test for combined top-level + operator-level tenantId stripping
  to lock in the early-return fix

* feat: ESLint rule to ban raw bulkWrite and collection.* in data-schemas

Add no-restricted-syntax rules to the data-schemas ESLint config that
flag direct Model.bulkWrite() and Model.collection.* calls. These
bypass Mongoose middleware and the tenant isolation plugin — all bulk
writes must use tenantSafeBulkWrite() instead.

Test files are excluded since they intentionally use raw driver calls
for fixture setup.

Also migrate the one remaining raw bulkWrite in seedSystemGrants() to
use tenantSafeBulkWrite() for consistency.

* test: add findByIdAndUpdate coverage to mutation guard tests

* fix: keep tenantSafeBulkWrite in seedSystemGrants, fix ESLint config

- Revert to tenantSafeBulkWrite in seedSystemGrants (always runs
  under runAsSystem, so the wrapper passes through correctly)
- Split data-schemas ESLint config: shared TS rules for all files,
  no-restricted-syntax only for production non-wrapper files
- Fix unused destructure vars to use _tenantId pattern
2026-04-01 19:07:52 -04:00
Danny Avila
5e789f589f
🔏 fix: Strip Unnecessary Fields Across Write Paths in Conversation & Message Methods (#12498)
* fix: Exclude field from conversation and message updates

* fix: Remove tenantId from conversation and message update objects to prevent unintended data exposure.
* refactor: Adjust update logic in createConversationMethods and createMessageMethods to ensure tenantId is not included in the updates, maintaining data integrity.

* fix: Strip tenantId from all write paths in conversation and message methods

Extends the existing tenantId stripping to bulkSaveConvos, bulkSaveMessages,
recordMessage, and updateMessage — all of which previously passed caller-supplied
tenantId straight through to the update document. Renames discard alias from _t
to _tenantId for clarity. Adds regression tests for all six write paths.

* fix: Eliminate double-copy overhead and strengthen test assertions

Replace destructure-then-spread with spread-once-then-delete for saveConvo,
saveMessage, and recordMessage — removes one O(n) copy per call on hot paths.
Add missing not-null and positive data assertions to tenantId stripping tests.

* test: Add positive data assertions to bulkSaveMessages and recordMessage tests
2026-04-01 11:16:39 -04:00
Danny Avila
d9f216c11a
📦 chore: bump dependabot packages (#12487)
* chore: Update Handlebars and package versions in package-lock.json and package.json

- Upgrade Handlebars from version 4.7.7 to 4.7.9 in both package-lock.json and package.json for improved performance and security.
- Update librechat-data-provider version from 0.8.401 to 0.8.406 in package-lock.json.
- Update @librechat/data-schemas version from 0.0.40 to 0.0.48 in package-lock.json.

* chore: Upgrade @happy-dom/jest-environment and happy-dom versions in package-lock.json and package.json

- Update @happy-dom/jest-environment from version 20.8.3 to 20.8.9 for improved compatibility.
- Upgrade happy-dom from version 20.8.3 to 20.8.9 to ensure consistency across dependencies.

* chore: Upgrade @rollup/plugin-terser to version 1.0.0 in package-lock.json and package.json

- Update @rollup/plugin-terser from version 0.4.4 to 1.0.0 in both package-lock.json and package.json for improved performance and compatibility.
- Reflect the new version in the dependencies of data-provider and data-schemas packages.

* chore: Upgrade rollup-plugin-typescript2 to version 0.37.0 in package-lock.json and package.json

- Update rollup-plugin-typescript2 from version 0.35.0 to 0.37.0 in package-lock.json and all relevant package.json files for improved compatibility and performance.
- Adjust dependencies for semver and tslib to their latest versions in line with the rollup-plugin-typescript2 upgrade.

* chore: Upgrade nodemailer to version 8.0.4 in package-lock.json and package.json

- Update nodemailer from version 7.0.11 to 8.0.4 in both package-lock.json and package.json to enhance functionality and security.

* chore: Upgrade picomatch, yaml, brace-expansion versions in package-lock.json

- Update picomatch from version 4.0.3 to 4.0.4 across multiple dependencies for improved functionality.
- Upgrade brace-expansion from version 2.0.2 to 2.0.3 and from 5.0.3 to 5.0.5 to enhance compatibility and performance.
- Update yaml from version 1.10.2 to 1.10.3 for better stability.
2026-03-31 13:36:20 -04:00
Danny Avila
c0ce7fee91
🚫 refactor: Remove Interface Config from Override Processing (#12473)
Add INTERFACE_PERMISSION_FIELDS set defining the interface fields that
seed role permissions at startup (prompts, agents, marketplace, etc.).
These fields are now stripped from DB config overrides in the merge
layer because updateInterfacePermissions() only runs at boot — DB
overrides for these fields create a client/server permission mismatch.

Pure UI fields (endpointsMenu, modelSelect, parameters, presets,
sidePanel, customWelcome, etc.) continue to work in overrides as
before.

YAML startup path is completely unaffected.
2026-03-31 11:07:31 -04:00
Dustin Healy
3d1b883e9d
👨‍👨‍👦‍👦 feat: Admin Users API Endpoints (#12446)
* feat: add admin user management endpoints

Add /api/admin/users with list, search, and delete handlers gated by
ACCESS_ADMIN + READ_USERS/MANAGE_USERS system grants. Handler factory
in packages/api uses findUsers, countUsers, and deleteUserById from
data-schemas.

* fix: address convention violations in admin users handlers

* fix: add pagination, self-deletion guard, and DB-level search limit

- listUsers now uses parsePagination + countUsers for proper pagination
  matching the roles/groups pattern
- findUsers extended with optional limit/offset options
- deleteUser returns 403 when caller tries to delete own account
- searchUsers passes limit to DB query instead of fetching all and
  slicing in JS
- Fix import ordering per CLAUDE.md, complete logger mock
- Replace fabricated date fallback with undefined

* fix: deterministic sort, null-safe pagination, consistent search filter

- Add sort option to findUsers; listUsers sorts by createdAt desc for
  deterministic pagination
- Use != null guards for offset/limit to handle zero values correctly
- Remove username from search filter since it is not in the projection
  or AdminUserSearchResult response type

* fix: last-admin deletion guard and search query max-length

- Prevent deleting the last admin user (look up target role, count
  admins, reject with 400 if count <= 1)
- Cap search query at 200 characters to prevent regex DoS
- Add tests for both guards

* fix: include missing capability name in 403 Forbidden response

* fix: cascade user deletion cleanup, search username, parallel capability checks

- Cascade Config, AclEntry, and SystemGrant cleanup on user deletion
  (matching the pattern in roles/groups handlers)
- Add username to admin search $or filter for parity with searchUsers
- Parallelize READ_* capability checks in listAllGrants with Promise.all

* fix: TOCTOU safety net, capability info leak, DRY/style cleanup, data-layer tests

- Add post-delete admin recount with CRITICAL log if race leaves 0 admins
- Revert capability name from 403 response to server-side log only
- Document thin deleteUserById limitation (full cascade is a future task)
- DRY: extract query.trim() to local variable in searchUsersHandler
- Add username to search projection, response type, and AdminUserSearchResult
- Functional filter/map in grants.ts parallel capability check
- Consistent null guards and limit>0 guard in findUsers options
- Fallback for empty result.message on delete response
- Fix mockUser() to generate unique _id per call
- Break long destructuring across multiple lines
- Assert countUsers filter and non-admin skip in delete tests
- Add data-layer tests for findUsers limit, offset, sort, and pagination

* chore: comment out admin delete user endpoint (out of scope)

* fix: cast USER principalId to ObjectId for ACL entry cleanup

ACL entries store USER principalId as ObjectId (via grantPermission casting),
but deleteAclEntries is a raw deleteMany that passes the filter through.
Passing a string won't match stored ObjectIds, leaving orphaned entries.

* chore: comment out unused requireManageUsers alongside disabled delete route

* fix: add missing logger.warn mock in capabilities test

* fix: harden admin users handlers — type safety, response consistency, test coverage

- Unify response shape: AdminUserSearchResult.userId → id, add AdminUserListItem type
- Fix unsafe req.query type assertion in searchUsersHandler (typeof guards)
- Anchor search regex with ^ for prefix matching (enables index usage)
- Add total/capped to search response for truncation signaling
- Add parseInt radix, remove redundant new Date() wrap
- Add tests: countUsers throw, countUsers call args, array query param, capped flag

* fix: scope deleteGrantsForPrincipal to tenant, deterministic search sort, align test mocks

- Add tenantId option to AdminUsersDeps.deleteGrantsForPrincipal and
  pass req.user.tenantId at the call site, matching the pattern already
  used by the roles and groups handlers
- Add sort: { name: 1 } to searchUsersHandler for deterministic results
- Align test mock deleteUserById messages with production output
  ('User was deleted successfully.')
- Make capped-results test explicitly set limit: '20' instead of
  relying on the implicit default

* test: add tenantId propagation test for deleteGrantsForPrincipal

Add tenantId to createReqRes user type and test that a non-undefined
tenantId is threaded through to deleteGrantsForPrincipal.

* test: remove redundant deleteUserById override in tenantId test

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-30 23:06:50 -04:00
Danny Avila
fda72ac621
🏗️ refactor: Remove Redundant Caching, Migrate Config Services to TypeScript (#12466)
* ♻️ refactor: Remove redundant scopedCacheKey caching, support user-provided key model fetching

Remove redundant cache layers that used `scopedCacheKey()` (tenant-only scoping)
on top of `getAppConfig()` which already caches per-principal (role+user+tenant).
This caused config overrides for different principals within the same tenant to
be invisible due to stale cached data.

Changes:
- Add `requireJwtAuth` to `/api/endpoints` route for proper user context
- Remove ENDPOINT_CONFIG, STARTUP_CONFIG, PLUGINS, TOOLS, and MODELS_CONFIG
  cache layers — all derive from `getAppConfig()` with cheap computation
- Enhance MODEL_QUERIES cache: hash(baseURL+apiKey) keys, 2-minute TTL,
  caching centralized in `fetchModels()` base function
- Support fetching models with user-provided API keys in `loadConfigModels`
  via `getUserKeyValues` lookup (no caching for user keys)
- Update all affected tests

Closes #1028

* ♻️ refactor: Migrate config services to TypeScript in packages/api

Move core config logic from CJS /api wrappers to typed TypeScript in
packages/api using dependency injection factories:

- `createEndpointsConfigService` — endpoint config merging + checkCapability
- `createLoadConfigModels` — custom endpoint model loading with user key support
- `createMCPToolCacheService` — MCP tool cache operations (update, merge, cache)

/api files become thin wrappers that wire dependencies (getAppConfig,
loadDefaultEndpointsConfig, getUserKeyValues, getCachedTools, etc.)
into the typed factories.

Also moves existing `endpoints/config.ts` → `endpoints/config/providers.ts`
to accommodate the new `config/` directory structure.

* 🔄 fix: Invalidate models query when user API key is set or revoked

Without this, users had to refresh the page after entering their API key
to see the updated model list fetched with their credentials.

- Invalidate QueryKeys.models in useUpdateUserKeysMutation onSuccess
- Invalidate QueryKeys.models in useRevokeUserKeyMutation onSuccess
- Invalidate QueryKeys.models in useRevokeAllUserKeysMutation onSuccess

* 🗺️ fix: Remap YAML-level override keys to AppConfig equivalents in mergeConfigOverrides

Config overrides stored in the DB use YAML-level keys (TCustomConfig),
but they're merged into the already-processed AppConfig where some fields
have been renamed by AppService. This caused mcpServers overrides to land
on a nonexistent key instead of mcpConfig, so config-override MCP servers
never appeared in the UI.

- Add OVERRIDE_KEY_MAP to remap mcpServers→mcpConfig, interface→interfaceConfig
- Apply remapping before deep merge in mergeConfigOverrides
- Add test for YAML-level key remapping behavior
- Update existing tests to use AppConfig field names in assertions

* 🧪 test: Update service.spec to use AppConfig field names after override key remapping

* 🛡️ fix: Address code review findings — reliability, types, tests, and performance

- Pass tenant context (getTenantId) in importers.js getEndpointsConfig call
- Add 5 tests for user-provided API key model fetching (key found, no key,
  DB error, missing userId, apiKey-only with fixed baseURL)
- Distinguish NO_USER_KEY (debug) from infrastructure errors (warn) in catch
- Switch fetchPromisesMap from Promise.all to Promise.allSettled so one
  failing provider doesn't kill the entire model config
- Parallelize getUserKeyValues DB lookups via batched Promise.allSettled
  instead of sequential awaits in the loop
- Hoist standardCache instance in fetchModels to avoid double instantiation
- Replace Record<string, unknown> types with Partial<TConfig>-based types;
  remove as unknown as T double-cast in endpoints config
- Narrow Bedrock availableRegions to typed destructure
- Narrow version field from string|number|undefined to string|undefined
- Fix import ordering in mcp/tools.ts and config/models.ts per AGENTS.md
- Add JSDoc to getModelsConfig alias clarifying caching semantics

* fix: Guard against null getCachedTools in mergeAppTools

* 🔍 fix: Address follow-up review — deduplicate extractEnvVariable, fix error discrimination, add log-level tests

- Deduplicate extractEnvVariable calls: resolve apiKey/baseURL once, reuse
  for both the entry and isUserProvided checks (Finding A)
- Move ResolvedEndpoint interface from function closure to module scope (Finding B)
- Replace fragile msg.includes('NO_USER_KEY') with ErrorTypes.NO_USER_KEY
  enum check against actual error message format (Finding C). Also handle
  ErrorTypes.INVALID_USER_KEY as an expected "no key" case.
- Add test asserting logger.warn is called for infra errors (not debug)
- Add test asserting logger.debug is called for NO_USER_KEY errors (not warn)

* fix: Preserve numeric assistants version via String() coercion

* 🐛 fix: Address secondary review — Ollama cache bypass, cache tests, type safety

- Fix Ollama success path bypassing cache write in fetchModels (CRITICAL):
  store result before returning so Ollama models benefit from 2-minute TTL
- Add 4 fetchModels cache behavior tests: cache write with TTL, cache hit
  short-circuits HTTP, skipCache bypasses read+write, empty results not cached
- Type-safe OVERRIDE_KEY_MAP: Partial<Record<keyof TCustomConfig, keyof AppConfig>>
  so compiler catches future field rename mismatches
- Fix import ordering in config/models.ts (package types longest→shortest)
- Rename ToolCacheDeps → MCPToolCacheDeps for naming consistency
- Expand getModelsConfig JSDoc to explain caching granularity

* fix: Narrow OVERRIDE_KEY_MAP index to satisfy strict tsconfig

* 🧩 fix: Add allowedProviders to TConfig, remove Record<string, unknown> from PartialEndpointEntry

The agents endpoint config includes allowedProviders (used by the frontend
AgentPanel to filter available providers), but it was missing from TConfig.
This forced PartialEndpointEntry to use & Record<string, unknown> as an
escape hatch, violating AGENTS.md type policy.

- Add allowedProviders?: (string | EModelEndpoint)[] to TConfig
- Remove Record<string, unknown> from PartialEndpointEntry — now just Partial<TConfig>

* 🛡️ fix: Isolate Ollama cache write from fetch try-catch, add Ollama cache tests

- Separate Ollama fetch and cache write into distinct scopes so a cache
  failure (e.g., Redis down) doesn't misattribute the error as an Ollama
  API failure and fall through to the OpenAI-compatible path (Issue A)
- Add 2 Ollama-specific cache tests: models written with TTL on fetch,
  cached models returned without hitting server (Issue B)
- Replace hardcoded 120000 with Time.TWO_MINUTES constant in cache TTL
  test assertion (Issue C)
- Fix OVERRIDE_KEY_MAP JSDoc to accurately describe runtime vs compile-time
  type enforcement (Issue D)
- Add global beforeEach for cache mock reset to prevent cross-test leakage

* 🧪 fix: Address third review — DI consistency, cache key width, MCP tests

- Inject loadCustomEndpointsConfig via EndpointsConfigDeps with default
  fallback, matching loadDefaultEndpointsConfig DI pattern (Finding 3)
- Widen modelsCacheKey from 64-bit (.slice(0,16)) to 128-bit (.slice(0,32))
  for collision-sensitive cross-credential cache key (Finding 4)
- Add fetchModels.mockReset() in loadConfigModels.spec beforeEach to
  prevent mock implementation leaks across tests (Finding 5)
- Add 11 unit tests for createMCPToolCacheService covering all three
  functions: null/empty input, successful ops, error propagation,
  cold-cache merge (Finding 2)
- Simplify getModelsConfig JSDoc to @see reference (Finding 10)

* ♻️ refactor: Address remaining follow-ups from reviews

OVERRIDE_KEY_MAP completeness:
- Add missing turnstile→turnstileConfig mapping
- Add exhaustiveness test verifying all three renamed keys are remapped
  and original YAML keys don't leak through

Import role context:
- Pass userRole through importConversations job → importLibreChatConvo
  so role-based endpoint overrides are honored during conversation import
- Update convos.js route to include req.user.role in the job payload

createEndpointsConfigService unit tests:
- Add 8 tests covering: default+custom merge, Azure/AzureAssistants/
  Anthropic Vertex/Bedrock config enrichment, assistants version
  coercion, agents allowedProviders, req.config bypass

Plugins/tools efficiency:
- Use Set for includedTools/filteredTools lookups (O(1) vs O(n) per plugin)
- Combine auth check + filter into single pass (eliminates intermediate array)
- Pre-compute toolDefKeys Set for O(1) tool definition lookups

* fix: Scope model query cache by user when userIdQuery is enabled

* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types

- When userIdQuery is true, skip caching entirely (like user_provided keys)
  to avoid cross-user model list leakage without duplicating cache data
- Fix AgentCapabilities type error in endpoints.spec.ts — use enum values
  and appConfig() helper for partial mock typing

* 🐛 fix: Restore filteredTools+includedTools composition, add checkCapability tests

- Fix filteredTools regression: whitelist and blacklist are now applied
  independently (two flat guards), matching original behavior where
  includedTools=['a','b'] + filteredTools=['b'] produces ['a'] (Finding A)
- Fix Set spread in toolkit loop: pre-compute toolDefKeysList array once
  alongside the Set, reuse for .some() without per-plugin allocation (Finding B)
- Add 2 filteredTools tests: blacklist-only path and combined
  whitelist+blacklist composition (Finding C)
- Add 3 checkCapability tests: capability present, capability absent,
  fallback to defaultAgentCapabilities for non-agents endpoints (Finding D)

* 🔑 fix: Include config-override MCP servers in filterAuthorizedTools

Config-override MCP servers (defined via admin config overrides for
roles/groups) were rejected by filterAuthorizedTools because it called
getAllServerConfigs(userId) without the configServers parameter. Only
YAML and DB-backed user servers were included in the access check.

- Add configServers parameter to filterAuthorizedTools
- Resolve config servers via resolveConfigServers(req) at all 4 callsites
  (create, update, duplicate, revert) using parallel Promise.all
- Pass configServers through to getAllServerConfigs(userId, configServers)
  so the registry merges config-source servers into the access check
- Update filterAuthorizedTools.spec.js mock for resolveConfigServers

* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types

For user-provided key endpoints (userProvide: true), skip the full model
list re-fetch during message validation — the user already selected from
a list we served them, and re-fetching with skipCache:true on every
message send is both slow and fragile (5s provider timeout = rejected model).

Instead, validate the model string format only:
- Must be a string, max 256 chars
- Must match [a-zA-Z0-9][a-zA-Z0-9_.:\-/@+ ]* (covers all known provider
  model ID formats while rejecting injection attempts)

System-configured endpoints still get full model list validation as before.

* 🧪 test: Add regression tests for filterAuthorizedTools configServers and validateModel

filterAuthorizedTools:
- Add test verifying configServers is passed to getAllServerConfigs and
  config-override server tools are allowed through
- Guard resolveConfigServers in createAgentHandler to only run when
  MCP tools are present (skip for tool-free agent creates)

validateModel (12 new tests):
- Format validation: missing model, non-string, length overflow, leading
  special char, script injection, standard model ID acceptance
- userProvide early-return: next() called immediately, getModelsConfig
  not invoked (regression guard for the exact bug this fixes)
- System endpoint list validation: reject unknown model, accept known
  model, handle null/missing models config

Also fix unnecessary backslash escape in MODEL_PATTERN regex.

* 🧹 fix: Remove space from MODEL_PATTERN, trim input, clean up nits

- Remove space character from MODEL_PATTERN regex — no real model ID
  uses spaces; prevents spurious violation logs from whitespace artifacts
- Add model.trim() before validation to handle accidental whitespace
- Remove redundant filterUniquePlugins call on already-deduplicated output
- Add comment documenting intentional whitelist+blacklist composition
- Add getUserKeyValues.mockReset() in loadConfigModels.spec beforeEach
- Remove narrating JSDoc from getModelsConfig one-liner
- Add 2 tests: trim whitespace handling, reject spaces in model ID

* fix: Match startup tool loader semantics — includedTools takes precedence over filteredTools

The startup tool loader (loadAndFormatTools) explicitly ignores
filteredTools when includedTools is set, with a warning log. The
PluginController was applying both independently, creating inconsistent
behavior where the same config produced different results at startup
vs plugin listing time.

Restored mutually exclusive semantics: when includedTools is non-empty,
filteredTools is not evaluated.

* 🧹 chore: Simplify validateModel flow, note auth requirement on endpoints route

- Separate missing-model from invalid-model checks cleanly: type+presence
  guard first, then trim+format guard (reviewer NIT)
- Add route comment noting auth is required for role/tenant scoping

* fix: Write trimmed model back to req.body.model for downstream consumers
2026-03-30 16:49:48 -04:00
Dustin Healy
a4a17ac771
⛩️ feat: Admin Grants API Endpoints (#12438)
* feat: add System Grants handler factory with tests

Handler factory with 4 endpoints: getEffectiveCapabilities (expanded
capability set for authenticated user), getPrincipalGrants (list grants
for a specific principal), assignGrant, and revokeGrant. Write ops
dynamically check MANAGE_ROLES/GROUPS/USERS based on target principal
type. 31 unit tests covering happy paths, validation, 403, and errors.

* feat: wire System Grants REST routes

Mount /api/admin/grants with requireJwtAuth + ACCESS_ADMIN gate.
Add barrel export for createAdminGrantsHandlers and AdminGrantsDeps.

* fix: cascade grant cleanup on role deletion

Add deleteGrantsForPrincipal to AdminRolesDeps and call it in
deleteRoleHandler via Promise.allSettled after successful deletion,
matching the groups cleanup pattern. 3 tests added for cleanup call,
skip on 404, and resilience to cleanup failure.

* fix: simplify cascade grant cleanup on role deletion

Replace Promise.allSettled wrapper with a direct try/catch for the
single deleteGrantsForPrincipal call.

* fix: harden grant handlers with auth, validation, types, and RESTful revoke

- Add per-handler auth checks (401) and granular capability gates
  (READ_* for getPrincipalGrants, possession check for assignGrant)
- Extract validatePrincipal helper; rewrite validateGrantBody to use
  direct type checks instead of unsafe `as string` casts
- Align DI types with data layer (ResolvedPrincipal.principalType
  widened to string, getUserPrincipals role made optional)
- Switch revoke route from DELETE body to RESTful URL params
- Return 201 for assignGrant to match roles/groups create convention
- Handle null grantCapability return with 500
- Add comprehensive test coverage for new auth/validation paths

* fix: deduplicate ResolvedPrincipal, typed body, defensive auth checks

- Remove duplicate ResolvedPrincipal from capabilities.ts; import the
  canonical export from grants.ts
- Replace Record<string, unknown> with explicit GrantRequestBody interface
- Add defensive 403 when READ_CAPABILITY_BY_TYPE lookup misses
- Document revoke asymmetry (no possession check) with JSDoc
- Use _id only in resolveUser (avoid Mongoose virtual reliance)
- Improve null-grant error message
- Complete logger mock in tests

* refactor: move ResolvedPrincipal to shared types to fix circular dep

Extract ResolvedPrincipal from admin/grants.ts to types/principal.ts
so middleware/capabilities.ts imports from shared types rather than
depending upward on the admin handler layer.

* chore: remove dead re-export, align logger mocks across admin tests

- Remove unused ResolvedPrincipal re-export from grants.ts (canonical
  source is types/principal.ts)
- Align logger mocks in roles.spec.ts and groups.spec.ts to include
  all log levels (error, warn, info, debug) matching grants.spec.ts

* fix: cascade Config and AclEntry cleanup on role deletion

Add deleteConfig and deleteAclEntries to role deletion cascade,
matching the group deletion pattern. Previously only grants were
cleaned up, leaving orphaned config overrides and ACL entries.

* perf: single-query batch for getEffectiveCapabilities

Add getCapabilitiesForPrincipals (plural) to the data layer — a single
$or query across all principals instead of N+1 parallel queries. Wire
it into the grants handler so getEffectiveCapabilities hits the DB once
regardless of how many principals the user has.

* fix: defer SystemCapabilities access to factory call time

Move all SystemCapabilities usage (VALID_CAPABILITIES,
MANAGE_CAPABILITY_BY_TYPE, READ_CAPABILITY_BY_TYPE) inside the
createAdminGrantsHandlers factory. External test suites that mock
@librechat/data-schemas without providing SystemCapabilities crashed
at import time when grants.ts was loaded transitively.

* test: add data-layer and handler test coverage for review findings

- Add 6 mongodb-memory-server tests for getCapabilitiesForPrincipals:
  multi-principal batch, empty array, filtering, tenant scoping
- Add handler test: all principals filtered (only PUBLIC)
- Add handler test: granting an implied capability succeeds
- Add handler test: all cascade cleanup operations fail simultaneously
- Document platform-scope-only tenantId behavior in JSDoc

* fix: resolveUser fallback to user.id, early-return empty principals

- Match capabilities middleware pattern: _id?.toString() ?? user.id
  to handle JWT-deserialized users without Mongoose _id
- Move empty-array guard before principals.map() to skip unnecessary
  normalizePrincipalId calls
- Add comment explaining VALID_PRINCIPAL_TYPES module-scope asymmetry

* refactor: derive VALID_PRINCIPAL_TYPES from capability maps

Make MANAGE_CAPABILITY_BY_TYPE and READ_CAPABILITY_BY_TYPE
non-Partial Records over a shared GrantPrincipalType union, then
derive VALID_PRINCIPAL_TYPES from the map keys. This makes divergence
between the three data structures structurally impossible.

* feat: add GET /api/admin/grants list-all-grants endpoint

Add listAllGrants data-layer method and handler so the admin panel
can fetch all grants in a single request instead of fanning out
N+M calls per role and group. Response is filtered to only include
grants for principal types the caller has read access to.

* fix: update principalType to use GrantPrincipalType for consistency in grants handling

- Refactor principalType in createAdminGrantsHandlers to use GrantPrincipalType instead of PrincipalType for better type accuracy.
- Ensure type consistency across the grants handling logic in the API.

* fix: address admin grants review findings — tenantId propagation, capability validation, pagination, and test coverage

Propagate tenantId through all grant operations for multi-tenancy support.
Extract isValidCapability to accept full SystemCapability union (base, section,
assign) and reuse it in both Mongoose schema validation and handler input checks.
Replace listAllGrants with paginated listGrants + countGrants. Filter PUBLIC
principals from getCapabilitiesForPrincipals queries. Export getCachedPrincipals
from ALS store for fast-path principal resolution. Move DELETE capability param
to query string to avoid colon-in-URL issues. Remove dead code and add
comprehensive handler and data-layer test coverage.

* refactor: harden admin grants — FilterQuery types, auth-first ordering, DELETE path param, isValidCapability tests

Replace Record<string, unknown> with FilterQuery<ISystemGrant> across all
data-layer query filters. Refactor buildTenantFilter to a pure tenantCondition
function that returns a composable FilterQuery fragment, eliminating the $or
collision between tenant and principal queries. Move auth check before input
validation in getPrincipalGrantsHandler, assignGrantHandler, and
revokeGrantHandler to avoid leaking valid type names to unauthenticated callers.
Switch DELETE route from query param back to path param (/:capability) with
encodeURIComponent per project conventions. Add compound index for listGrants
sort. Type VALID_PRINCIPAL_TYPES as Set<GrantPrincipalType>. Remove unused
GetCachedPrincipalsFn type export. Add dedicated isValidCapability unit tests
and revokeGrant idempotency test.

* refactor: batch capability checks in listGrantsHandler via getHeldCapabilities

Replace 3 parallel hasCapabilityForPrincipals DB calls with a single
getHeldCapabilities query that returns the subset of capabilities any
principal holds. Also: defensive limit(0) clamp, parallelized assignGrant
auth checks, principalId type-vs-required error split, tenantCondition
hoisted to factory top, JSDoc on cascade deps, DELETE route encoding note.

* fix: normalize principalId and filter undefined in getHeldCapabilities

Add normalizePrincipalId + null guard to getHeldCapabilities, matching
the contract of getCapabilitiesForPrincipals. Simplify allCaps build
with flatMap, add no-tenantId cross-check and undefined-principalId
test cases.

* refactor: use concrete types in GrantRequestBody, rename encoding test

Replace unknown fields with explicit string types in GrantRequestBody,
matching the established pattern in roles/groups/config handlers. Rename
misleading 'encoded' test to 'with colons' since Express auto-decodes
req.params.

* fix: support hierarchical parent capabilities in possession checks

hasCapabilityForPrincipals and getHeldCapabilities now resolve parent
base capabilities for section/assignment grants. An admin holding
manage:configs can now grant manage:configs:<section> and transitively
read:configs:<section>. Fixes anti-escalation 403 blocking config
capability delegation.

* perf: use getHeldCapabilities in assignGrant to halve DB round-trips

assignGrantHandler was making two parallel hasCapabilityForPrincipals
calls to check manage + capability possession. getHeldCapabilities was
introduced in this PR specifically for this pattern. Replace with a
single batched call. Update corresponding spec assertions.

* fix: validate role existence before granting capabilities

Grants for non-existent role names were silently persisted, creating
orphaned grants that could surprise-activate if a role with that name
was later created. Add optional checkRoleExists dep to assignGrant and
wire it to getRoleByName in the route file.

* refactor: tighten principalType typing and use grantCapability in tests

Narrow getCapabilitiesForPrincipals parameter from string to
PrincipalType, removing the redundant cast. Replace direct
SystemGrant.create() calls in getCapabilitiesForPrincipals tests with
methods.grantCapability() to honor the schema's normalization invariant.
Add getHeldCapabilities extended capability tests.

* test: rename misleading cascade cleanup test name

The test only injects failure into deleteGrantsForPrincipal, not all
cascade operations. Rename from 'cascade cleanup fails' to 'grant
cleanup fails' to match the actual scope.

* fix: reorder role check after permission guard, add tenantId to index

Move checkRoleExists after the getHeldCapabilities permission check so
that a sub-MANAGE_ROLES admin cannot probe role name existence via
400 vs 403 response codes.

Add tenantId to the { principalType, capability } index so listGrants
queries in multi-tenant deployments can use a covering index instead
of post-scanning for tenant condition.

Add missing test for checkRoleExists throwing.

* fix: scope deleteGrantsForPrincipal to tenant on role deletion

deleteGrantsForPrincipal previously filtered only on principalType +
principalId, deleting grants across all tenants. Since the role schema
supports multi-tenancy (compound unique index on name + tenantId), two
tenants can share a role name like 'editor'. Deleting that role in one
tenant would wipe grants for identically-named roles in other tenants.

Add optional tenantId parameter to deleteGrantsForPrincipal. When
provided, scopes the delete to that tenant plus platform-level grants.
Propagate req.user.tenantId through the role deletion cascade.

* fix: scope grant cleanup to tenant on group deletion

Same cross-tenant gap as the role deletion path: deleteGroupHandler
called deleteGrantsForPrincipal without tenantId, so deleting a group
would wipe its grants across all tenants. Extract req.user.tenantId
and pass it through.

* test: add HTTP integration test for admin grants routes

Supertest-based test with real MongoMemoryServer exercising the full
Express wiring: route registration, injected auth middleware, handler
DI deps, and real DB round-trips.

Covers GET /, GET /effective, POST / + DELETE / lifecycle, role
existence validation, and 401 for unauthenticated callers.

Also documents the expandImplications scope: the /effective endpoint
returns base-level capabilities only; section-level resolution is
handled at authorization check time by getParentCapabilities.

* fix: use exact tenant match in deleteGrantsForPrincipal, normalize principalId, harden API

CRITICAL: deleteGrantsForPrincipal was using tenantCondition (a
read-query helper) for deleteMany, which includes the
{ tenantId: { $exists: false } } arm. This silently destroyed
platform-level grants when a tenant-scoped role/group deletion
occurred. Replace with exact { tenantId } match for deletes so
platform-level grants survive tenant-scoped cascade cleanup.

Refactor deleteGrantsForPrincipal signature from fragile positional
overload (sessionOrTenantId union + maybeSession) to a clean options
object: { tenantId?, session? }. Update all callers and test assertions.

Add normalizePrincipalId to hasCapabilityForPrincipals to match the
pattern already used by getHeldCapabilities — prevents string/ObjectId
type mismatch on USER/GROUP principal queries.

Also: export GrantPrincipalType from barrel, add upper-bound cap to
listGrants, document GROUP/USER existence check trade-off, add
integration tests for tenant-isolation property of deleteGrantsForPrincipal.

* fix: forward tenantId to getUserPrincipals in resolvePrincipals

resolvePrincipals had tenantId available from the caller but only
forwarded it to getCachedPrincipals (cache lookup). The DB fallback
via getUserPrincipals omitted it. While the Group schema's
applyTenantIsolation Mongoose plugin handles scoping via
AsyncLocalStorage in HTTP request context, explicitly passing tenantId
makes the contract visible and prevents silent cross-tenant group
resolution if called outside request context.

* fix: remove unused import and add assertion to 401 integration test

Remove unused SystemCapabilities import flagged by ESLint. Add explicit
body assertion to the 401 test so it has a jest expect() call.

* chore: hoist grant limit constants to scope, remove dead isolateModules

Move GRANTS_DEFAULT_LIMIT / GRANTS_MAX_LIMIT from inside listGrants
function body to createSystemGrantMethods scope so they are evaluated
once at module load. Remove dead jest.isolateModules + jest.doMock
block in integration test — the ~/models mock was never exercised
since handlers are built with explicit DI deps.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-30 16:49:23 -04:00
Danny Avila
fda1bfc3cc
🔬 ci: Add TypeScript Type Checks to Backend Workflow and Fix All Type Errors (#12451)
* fix(data-schemas): resolve TypeScript strict type check errors in source files

- Constrain ConfigSection to string keys via `string & keyof TCustomConfig`
- Replace broken `z` import from data-provider with TCustomConfig derivation
- Add `_id: Types.ObjectId` to IUser matching other Document interfaces
- Add `federatedTokens` and `openidTokens` optional fields to IUser
- Type mongoose model accessors as `Model<IRole>` and `Model<IUser>`
- Widen `getPremiumRate` param to accept `number | null`
- Widen `bulkWriteAclEntries` ops to untyped `AnyBulkWriteOperation[]`
- Fix `getUserPrincipals` return type to use `PrincipalType` enum
- Add non-null assertions for `connection.db` in migration files
- Import DailyRotateFile constructor directly instead of relying on
  broken module augmentation across mismatched node_modules trees
- Add winston-daily-rotate-file as devDependency for type resolution

* fix(data-schemas): resolve TypeScript type errors in test files

- Replace arbitrary test keys with valid TCustomConfig properties in config.spec
- Use non-null assertions for permission objects in role.methods.spec
- Replace `.SHARED_GLOBAL` access with `.not.toHaveProperty()` for legacy field
- Add non-null assertions for balance, writeRate, readRate in spendTokens.spec
- Update mock user _id to use ObjectId in user.test
- Remove unused Schema import in tenantIndexes.spec

* fix(api): resolve TypeScript strict type check errors across source and test files

- Widen getUserPrincipals dep type in capabilities middleware
- Fix federatedTokens type in createSafeUser return
- Use proper mock req type for read-only properties in preAuthTenant.spec
- Replace `as IUser` casts with ObjectId-typed mocks in openid/oidc specs
- Use TokenExchangeMethodEnum values instead of string literals in MCP specs
- Fix SessionStore type compatibility in sessionCache specs
- Replace `catch (error: any)` with `(error as Error)` in redis specs
- Remove invalid properties from test data in initialize and MCP specs
- Add String.prototype.isWellFormed declaration for sanitizeTitle spec

* fix(client): resolve TypeScript type errors in shared client components

- Add default values for destructured bindings in OGDialogTemplate
- Replace broken ExtendedFile import with inline type in FileIcon

* ci: add TypeScript type-check job to backend review workflow

Add a `typecheck` job that runs `tsc --noEmit` on all four TypeScript
workspaces (data-provider, data-schemas, @librechat/api, @librechat/client)
after the build step. Catches type errors that rollup builds may miss.

* fix(data-schemas): add local type declaration for DailyRotateFile transport

The `winston-daily-rotate-file` package ships a module augmentation for
`winston/lib/winston/transports`, but it fails when winston and
winston-daily-rotate-file resolve from different node_modules trees
(which happens in this monorepo due to npm hoisting).

Add a local `.d.ts` declaration that augments the same module path from
within data-schemas' compilation unit, so `tsc --noEmit` passes while
keeping the original runtime pattern (`new winston.transports.DailyRotateFile`).

* fix: address code review findings from PR #12451

- Restore typed `AnyBulkWriteOperation<AclEntry>[]` on bulkWriteAclEntries,
  cast to untyped only at the tenantSafeBulkWrite call site (Finding 1)
- Type `findUser` model accessor consistently with `findUsers` (Finding 2)
- Replace inline `import('mongoose').ClientSession` with top-level import type
- Use `toHaveLength` for spy assertions in playwright-expect spec file
- Replace numbered Record casts with `.not.toHaveProperty()` in
  role.methods.spec for SHARED_GLOBAL assertions
- Use per-test ObjectIds instead of shared testUserId in openid.spec
- Replace inline `import()` type annotations with top-level SessionData
  import in sessionCache spec
- Remove extraneous blank line in user.ts searchUsers

* refactor: address remaining review findings (4–7)

- Extract OIDCTokens interface in user.ts; deduplicate across IUser fields
  and oidc.ts FederatedTokens (Finding 4)
- Move String.isWellFormed declaration from spec file to project-level
  src/types/es2024-string.d.ts (Finding 5)
- Replace verbose `= undefined` defaults in OGDialogTemplate with null
  coalescing pattern (Finding 6)
- Replace `Record<string, unknown>` TestConfig with named interface
  containing explicit test fields (Finding 7)
2026-03-28 21:06:39 -04:00
Danny Avila
877c2efc85
🏗️ feat: bulkWrite isolation, pre-auth context, strict-mode fixes (#12445)
* fix: wrap seedDatabase() in runAsSystem() for strict tenant mode

seedDatabase() was called without tenant context at startup, causing
every Mongoose operation inside it to throw when
TENANT_ISOLATION_STRICT=true. Wrapping in runAsSystem() gives it the
SYSTEM_TENANT_ID sentinel so the isolation plugin skips filtering,
matching the pattern already used for performStartupChecks and
updateInterfacePermissions.

* fix: chain tenantContextMiddleware in optionalJwtAuth

optionalJwtAuth populated req.user but never established ALS tenant
context, unlike requireJwtAuth which chains tenantContextMiddleware
after successful auth. Authenticated users hitting routes with
optionalJwtAuth (e.g. /api/banner) had no tenant isolation.

* feat: tenant-safe bulkWrite wrapper and call-site migration

Mongoose's bulkWrite() does not trigger schema-level middleware hooks,
so the applyTenantIsolation plugin cannot intercept it. This adds a
tenantSafeBulkWrite() utility that injects the current ALS tenant
context into every operation's filter/document before delegating to
native bulkWrite.

Migrates all 8 runtime bulkWrite call sites:
- agentCategory (seedCategories, ensureDefaultCategories)
- conversation (bulkSaveConvos)
- message (bulkSaveMessages)
- file (batchUpdateFiles)
- conversationTag (updateTagsForConversation, bulkIncrementTagCounts)
- aclEntry (bulkWriteAclEntries)

systemGrant.seedSystemGrants is intentionally not migrated — it uses
explicit tenantId: { $exists: false } filters and is exempt from the
isolation plugin.

* feat: pre-auth tenant middleware and tenant-scoped config cache

Adds preAuthTenantMiddleware that reads X-Tenant-Id from the request
header and wraps downstream in tenantStorage ALS context. Wired onto
/oauth, /api/auth, /api/config, and /api/share — unauthenticated
routes that need tenant scoping before JWT auth runs.

The /api/config cache key is now tenant-scoped
(STARTUP_CONFIG:${tenantId}) so multi-tenant deployments serve the
correct login page config per tenant.

The middleware is intentionally minimal — no subdomain parsing, no
OIDC claim extraction. The private fork's reverse proxy or auth
gateway sets the header.

* feat: accept optional tenantId in updateInterfacePermissions

When tenantId is provided, the function re-enters inside
tenantStorage.run({ tenantId }) so all downstream Mongoose queries
target that tenant's roles instead of the system context. This lets
the private fork's tenant provisioning flow call
updateInterfacePermissions per-tenant after creating tenant-scoped
ADMIN/USER roles.

* fix: tenant-filter $lookup in getPromptGroup aggregation

The $lookup stage in getPromptGroup() queried the prompts collection
without tenant filtering. While the outer PromptGroup aggregate is
protected by the tenantIsolation plugin's pre('aggregate') hook,
$lookup runs as an internal MongoDB operation that bypasses Mongoose
hooks entirely.

Converts from simple field-based $lookup to pipeline-based $lookup
with an explicit tenantId match when tenant context is active.

* fix: replace field-level unique indexes with tenant-scoped compounds

Field-level unique:true creates a globally-unique single-field index in
MongoDB, which would cause insert failures across tenants sharing the
same ID values.

- agent.id: removed field-level unique, added { id, tenantId } compound
- convo.conversationId: removed field-level unique (compound at line 50
  already exists: { conversationId, user, tenantId })
- message.messageId: removed field-level unique (compound at line 165
  already exists: { messageId, user, tenantId })
- preset.presetId: removed field-level unique, added { presetId, tenantId }
  compound

* fix: scope MODELS_CONFIG, ENDPOINT_CONFIG, PLUGINS, TOOLS caches by tenant

These caches store per-tenant configuration (available models, endpoint
settings, plugin availability, tool definitions) but were using global
cache keys. In multi-tenant mode, one tenant's cached config would be
served to all tenants.

Appends :${tenantId} to cache keys when tenant context is active.
Falls back to the unscoped key when no tenant context exists (backward
compatible for single-tenant OSS deployments).

Covers all read, write, and delete sites:
- ModelController.js: get/set MODELS_CONFIG
- PluginController.js: get/set PLUGINS, get/set TOOLS
- getEndpointsConfig.js: get/set/delete ENDPOINT_CONFIG
- app.js: delete ENDPOINT_CONFIG (clearEndpointConfigCache)
- mcp.js: delete TOOLS (updateMCPTools, mergeAppTools)
- importers.js: get ENDPOINT_CONFIG

* fix: add getTenantId to PluginController spec mock

The data-schemas mock was missing getTenantId, causing all
PluginController tests to throw when the controller calls
getTenantId() for tenant-scoped cache keys.

* fix: address review findings — migration, strict-mode, DRY, types

Addresses all CRITICAL, MAJOR, and MINOR review findings:

F1 (CRITICAL): Add agents, conversations, messages, presets to
SUPERSEDED_INDEXES in tenantIndexes.ts so dropSupersededTenantIndexes()
drops the old single-field unique indexes that block multi-tenant inserts.

F2 (CRITICAL): Unknown bulkWrite op types now throw in strict mode
instead of silently passing through without tenant injection.

F3 (MAJOR): Replace wildcard export with named export for
tenantSafeBulkWrite, hiding _resetBulkWriteStrictCache from the
public package API.

F5 (MAJOR): Restore AnyBulkWriteOperation<IAclEntry>[] typing on
bulkWriteAclEntries — the unparameterized wrapper accepts parameterized
ops as a subtype.

F7 (MAJOR): Fix config.js tenant precedence — JWT-derived
req.user.tenantId now takes priority over the X-Tenant-Id header for
authenticated requests.

F8 (MINOR): Extract scopedCacheKey() helper into tenantContext.ts and
replace all 11 inline occurrences across 7 files.

F9 (MINOR): Use simple localField/foreignField $lookup for the
non-tenant getPromptGroup path (more efficient index seeks).

F12 (NIT): Remove redundant BulkOp type alias.
F13 (NIT): Remove debug log that leaked raw tenantId.

* fix: add new superseded indexes to tenantIndexes test fixture

The test creates old indexes to verify the migration drops them.
Missing fixture entries for agents.id_1, conversations.conversationId_1,
messages.messageId_1, and presets.presetId_1 caused the count assertion
to fail (expected 22, got 18).

* fix: restore logger.warn for unknown bulk op types in non-strict mode

* fix: block SYSTEM_TENANT_ID sentinel from external header input

CRITICAL: preAuthTenantMiddleware accepted any string as X-Tenant-Id,
including '__SYSTEM__'. The tenantIsolation plugin treats SYSTEM_TENANT_ID
as an explicit bypass — skipping ALL query filters. A client sending
X-Tenant-Id: __SYSTEM__ to pre-auth routes (/api/share, /api/config,
/api/auth, /oauth) would execute Mongoose operations without tenant
isolation.

Fixes:
- preAuthTenantMiddleware rejects SYSTEM_TENANT_ID in header
- scopedCacheKey returns the base key (not key:__SYSTEM__) in system
  context, preventing stale cache entries during runAsSystem()
- updateInterfacePermissions guards tenantId against SYSTEM_TENANT_ID
- $lookup pipeline separates $expr join from constant tenantId match
  for better index utilization
- Regression test for sentinel rejection in preAuthTenant.spec.ts
- Remove redundant getTenantId() call in config.js

* test: add missing deleteMany/replaceOne coverage, fix vacuous ALS assertions

bulkWrite spec:
- deleteMany: verifies tenant-scoped deletion leaves other tenants untouched
- replaceOne: verifies tenantId injected into both filter and replacement
- replaceOne overwrite: verifies a conflicting tenantId in the replacement
  document is overwritten by the ALS tenant (defense-in-depth)
- empty ops array: verifies graceful handling

preAuthTenant spec:
- All negative-case tests now use the capturedNext pattern to verify
  getTenantId() inside the middleware's execution context, not the
  test runner's outer frame (which was always undefined regardless)

* feat: tenant-isolate MESSAGES cache, FLOWS cache, and GenerationJobManager

MESSAGES cache (streamAudio.js):
- Cache key now uses scopedCacheKey(messageId) to prefix with tenantId,
  preventing cross-tenant message content reads during TTS streaming.

FLOWS cache (FlowStateManager):
- getFlowKey() now generates ${type}:${tenantId}:${flowId} when tenant
  context is active, isolating OAuth flow state per tenant.

GenerationJobManager:
- tenantId added to SerializableJobData and GenerationJobMetadata
- createJob() captures the current ALS tenant context (excluding
  SYSTEM_TENANT_ID) and stores it in job metadata
- SSE subscription endpoint validates job.metadata.tenantId matches
  req.user.tenantId, blocking cross-tenant stream access
- Both InMemoryJobStore and RedisJobStore updated to accept tenantId

* fix: add getTenantId and SYSTEM_TENANT_ID to MCP OAuth test mocks

FlowStateManager.getFlowKey() now calls getTenantId() for tenant-scoped
flow keys. The 4 MCP OAuth test files mock @librechat/data-schemas
without these exports, causing TypeError at runtime.

* fix: correct import ordering per AGENTS.md conventions

Package imports sorted shortest to longest line length, local imports
sorted longest to shortest — fixes ordering violations introduced by
our new imports across 8 files.

* fix: deserialize tenantId in RedisJobStore — cross-tenant SSE guard was no-op in Redis mode

serializeJob() writes tenantId to the Redis hash via Object.entries,
but deserializeJob() manually enumerates fields and omitted tenantId.
Every getJob() from Redis returned tenantId: undefined, causing the
SSE route's cross-tenant guard to short-circuit (undefined && ... → false).

* test: SSE tenant guard, FlowStateManager key consistency, ALS scope docs

SSE stream tenant tests (streamTenant.spec.js):
- Cross-tenant user accessing another tenant's stream → 403
- Same-tenant user accessing own stream → allowed
- OSS mode (no tenantId on job) → tenant check skipped

FlowStateManager tenant tests (manager.tenant.spec.ts):
- completeFlow finds flow created under same tenant context
- completeFlow does NOT find flow under different tenant context
- Unscoped flows are separate from tenant-scoped flows

Documentation:
- JSDoc on getFlowKey documenting ALS context consistency requirement
- Comment on streamAudio.js scopedCacheKey capture site

* fix: SSE stream tests hang on success path, remove internal fork references

The success-path tests entered the SSE streaming code which never
closes, causing timeout. Mock subscribe() to end the response
immediately. Restructured assertions to verify non-403/non-404.

Removed "private fork" and "OSS" references from code and test
descriptions — replaced with "deployment layer", "multi-tenant
deployments", and "single-tenant mode".

* fix: address review findings — test rigor, tenant ID validation, docs

F1: SSE stream tests now mock subscribe() with correct signature
(streamId, writeEvent, onDone, onError) and assert 200 status,
verifying the tenant guard actually allows through same-tenant users.

F2: completeFlow logs the attempted key and ALS tenantId when flow
is not found, so reverse proxy misconfiguration (missing X-Tenant-Id
on OAuth callback) produces an actionable warning.

F3/F10: preAuthTenantMiddleware validates tenant ID format — rejects
colons, special characters, and values exceeding 128 chars. Trims
whitespace. Prevents cache key collisions via crafted headers.

F4: Documented cache invalidation scope limitation in
clearEndpointConfigCache — only the calling tenant's key is cleared;
other tenants expire via TTL.

F7: getFlowKey JSDoc now lists all 8 methods requiring consistent
ALS context.

F8: Added dedicated scopedCacheKey unit tests — base key without
context, base key in system context, scoped key with tenant, no
ALS leakage across scope boundaries.

* fix: revert flow key tenant scoping, fix SSE test timing

FlowStateManager: Reverts tenant-scoped flow keys. OAuth callbacks
arrive without tenant ALS context (provider redirects don't carry
X-Tenant-Id), so completeFlow/failFlow would never find flows
created under tenant context. Flow IDs are random UUIDs with no
collision risk, and flow data is ephemeral (TTL-bounded).

SSE tests: Use process.nextTick for onDone callback so Express
response headers are flushed before res.write/res.end are called.

* fix: restore getTenantId import for completeFlow diagnostic log

* fix: correct completeFlow warning message, add missing flow test

The warning referenced X-Tenant-Id header consistency which was only
relevant when flow keys were tenant-scoped (since reverted). Updated
to list actual causes: TTL expiry, missing flow, or routing to a
different instance without shared Keyv storage.

Removed the getTenantId() call and import — no longer needed since
flow keys are unscoped.

Added test for the !flowState branch in completeFlow — verifies
return false and logger.warn on nonexistent flow ID.

* fix: add explicit return type to recursive updateInterfacePermissions

The recursive call (tenantId branch calls itself without tenantId)
causes TypeScript to infer circular return type 'any'. Adding
explicit Promise<void> satisfies the rollup typescript plugin.

* fix: update MCPOAuthRaceCondition test to match new completeFlow warning

* fix: clearEndpointConfigCache deletes both scoped and unscoped keys

Unauthenticated /api/endpoints requests populate the unscoped
ENDPOINT_CONFIG key. Admin config mutations clear only the
tenant-scoped key, leaving the unscoped entry stale indefinitely.
Now deletes both when in tenant context.

* fix: tenant guard on abort/status endpoints, warn logs, test coverage

F1: Add tenant guard to /chat/status/:conversationId and /chat/abort
matching the existing guard on /chat/stream/:streamId. The status
endpoint exposes aggregatedContent (AI response text) which requires
tenant-level access control.

F2: preAuthTenantMiddleware now logs warn for rejected __SYSTEM__
sentinel and malformed tenant IDs, providing observability for
bypass probing attempts.

F3: Abort fallback path (getActiveJobIdsForUser) now has tenant
check after resolving the job.

F4: Test for strict mode + SYSTEM_TENANT_ID — verifies runAsSystem
bypasses tenantSafeBulkWrite without throwing in strict mode.

F5: Test for job with tenantId + user without tenantId → 403.

F10: Regex uses idiomatic hyphen-at-start form.

F11: Test descriptions changed from "rejects" to "ignores" since
middleware calls next() (not 4xx).

Also fixes MCPOAuthRaceCondition test assertion to match updated
completeFlow warning message.

* fix: test coverage for logger.warn, status/abort guards, consistency

A: preAuthTenant spec now mocks logger and asserts warn calls for
__SYSTEM__ sentinel, malformed characters, and oversized headers.

B: streamTenant spec expanded with status and abort endpoint tests —
cross-tenant status returns 403, same-tenant returns 200 with body,
cross-tenant abort returns 403.

C: Abort endpoint uses req.user.tenantId (not req.user?.tenantId)
matching stream/status pattern — requireJwtAuth guarantees req.user.

D: Malformed header warning now includes ip in log metadata,
matching the sentinel warning for consistent SOC correlation.

* fix: assert ip field in malformed header warn tests

* fix: parallelize cache deletes, document tenant guard, fix import order

- clearEndpointConfigCache uses Promise.all for independent cache
  deletes instead of sequential awaits
- SSE stream tenant guard has inline comment explaining backward-compat
  behavior for untenanted legacy jobs
- conversation.ts local imports reordered longest-to-shortest per
  AGENTS.md

* fix: tenant-qualify userJobs keys, document tenant guard backward-compat

Job store userJobs keys now include tenantId when available:
- Redis: stream:user:{tenantId:userId}:jobs (falls back to
  stream:user:{userId}:jobs when no tenant)
- InMemory: composite key tenantId:userId in userJobMap

getActiveJobIdsByUser/getActiveJobIdsForUser accept optional tenantId
parameter, threaded through from req.user.tenantId at all call sites
(/chat/active and /chat/abort fallback).

Added inline comments on all three SSE tenant guards explaining the
backward-compat design: untenanted legacy jobs remain accessible
when the userId check passes.

* fix: parallelize cache deletes, document tenant guard, fix import order

Fix InMemoryJobStore.getActiveJobIdsByUser empty-set cleanup to use
the tenant-qualified userKey instead of bare userId — prevents
orphaned empty Sets accumulating in userJobMap for multi-tenant users.

Document cross-tenant staleness in clearEndpointConfigCache JSDoc —
other tenants' scoped keys expire via TTL, not active invalidation.

* fix: cleanup userJobMap leak, startup warning, DRY tenant guard, docs

F1: InMemoryJobStore.cleanup() now removes entries from userJobMap
before calling deleteJob, preventing orphaned empty Sets from
accumulating with tenant-qualified composite keys.

F2: Startup warning when TENANT_ISOLATION_STRICT is active — reminds
operators to configure reverse proxy to control X-Tenant-Id header.

F3: mergeAppTools JSDoc documents that tenant-scoped TOOLS keys are
not actively invalidated (matching clearEndpointConfigCache pattern).

F5: Abort handler getActiveJobIdsForUser call uses req.user.tenantId
(not req.user?.tenantId) — consistent with stream/status handlers.

F6: updateInterfacePermissions JSDoc clarifies SYSTEM_TENANT_ID
behavior — falls through to caller's ALS context.

F7: Extracted hasTenantMismatch() helper, replacing three identical
inline tenant guard blocks across stream/status/abort endpoints.

F9: scopedCacheKey JSDoc documents both passthrough cases (no context
and SYSTEM_TENANT_ID context).

* fix: clean userJobMap in evictOldest — same leak as cleanup()
2026-03-28 16:43:50 -04:00
Dustin Healy
5972a21479
🪪 feat: Admin Roles API Endpoints (#12400)
* feat: add createRole and deleteRole methods to role

* feat: add admin roles handler factory and Express routes

* fix: address convention violations in admin roles handlers

* fix: rename createRole/deleteRole to avoid AccessRole name collision

The existing accessRole.ts already exports createRole/deleteRole for the
AccessRole model. In createMethods index.ts, these are spread after
roleMethods, overwriting them. Renamed our Role methods to
createRoleByName/deleteRoleByName to match the existing pattern
(getRoleByName, updateRoleByName) and avoid the collision.

* feat: add description field to Role model

- Add description to IRole, CreateRoleRequest, UpdateRoleRequest types
- Add description field to Mongoose roleSchema (default: '')
- Wire description through createRoleHandler and updateRoleHandler
- Include description in listRoles select clause so it appears in list

* fix: address Copilot review findings in admin roles handlers

* test: add unit tests for admin roles and groups handlers

* test: add data-layer tests for createRoleByName, deleteRoleByName, listUsersByRole

* fix: allow system role updates when name is unchanged

The updateRoleHandler guard rejected any request where body.name matched
a system role, even when the name was not being changed. This blocked
editing a system role's description. Compare against the URL param to
only reject actual renames to reserved names.

* fix: address external review findings for admin roles

- Block renaming system roles (ADMIN/USER) and add user migration on rename
- Add input validation: name max-length, trim on update, duplicate name check
- Replace fragile String.includes error matching with prefix-based classification
- Catch MongoDB 11000 duplicate key in createRoleByName
- Add pagination (limit/offset/total) to getRoleMembersHandler
- Reverse delete order in deleteRoleByName — reassign users before deletion
- Add role existence check in removeRoleMember; drop unused createdAt select
- Add Array.isArray guard for permissions input; use consistent ?? coalescing
- Fix import ordering per AGENTS.md conventions
- Type-cast mongoose.models.User as Model<IUser> for proper TS inference
- Add comprehensive tests: rename guards, pagination, validation, 500 paths

* fix: address re-review findings for admin roles

- Gate deleteRoleByName on existence check — skip user reassignment and
  cache invalidation when role doesn't exist (fixes test mismatch)
- Reverse rename order: migrate users before renaming role so a migration
  failure leaves the system in a consistent state
- Add .sort({ _id: 1 }) to listUsersByRole for deterministic pagination
- Import shared AdminMember type from data-schemas instead of local copy;
  make joinedAt optional since neither groups nor roles populate it
- Change IRole.description from optional to required to match schema default
- Add data-layer tests for updateUsersByRole and countUsersByRole
- Add handler test verifying users-first rename ordering and migration
  failure safety

* fix: add rollback on rename failure and update PR description

- Roll back user migration if updateRoleByName returns null during a
  rename (race: role deleted between existence check and update)
- Add test verifying rollback calls updateUsersByRole in reverse
- Update PR #12400 description to reflect current test counts (56
  handler tests, 40 data-layer tests) and safety features

* fix: rollback on rename throw, description validation, delete/DRY cleanup

- Hoist isRename/trimmedName above try block so catch can roll back user
  migration when updateRoleByName throws (not just returns null)
- Add description type + max-length (2000) validation in create and update,
  consistent with groups handler
- Remove redundant getRoleByName existence check in deleteRoleHandler —
  use deleteRoleByName return value directly
- Skip no-op name write when body.name equals current name (use isRename)
- Extract getUserModel() accessor to DRY repeated Model<IUser> casts
- Use name.trim() consistently in createRoleByName error messages
- Add tests: rename-throw rollback, description validation (create+update),
  update delete test mocks to match simplified handler

* fix: guard spurious rollback, harden createRole error path, validate before DB calls

- Add migrationRan flag to prevent rollback of user migration that never ran
- Return generic message on 500 in createRoleHandler, specific only for 409
- Move description validation before DB queries in updateRoleHandler
- Return existing role early when update body has no changes
- Wrap cache.set in createRoleByName with try/catch to prevent masking DB success
- Add JSDoc on 11000 catch explaining compound unique index
- Add tests: spurious rollback guard, empty update body, description validation
  ordering, listUsersByRole pagination

* fix: validate permissions in create, RoleConflictError, rollback safety, cache consistency

- Add permissions type/array validation in createRoleHandler
- Introduce RoleConflictError class replacing fragile string-prefix matching
- Wrap rollback in !role null path with try/catch for correct 404 response
- Wrap deleteRoleByName cache.set in try/catch matching createRoleByName
- Narrow updateRoleHandler body type to { name?, description? }
- Add tests: non-string description in create, rollback failure logging,
  permissions array rejection, description max-length assertion fix

* feat: prevent removing the last admin user

Add guard in removeRoleMember that checks countUsersByRole before
demoting an ADMIN user, returning 400 if they are the last one.

* fix: move interleaved export below imports, add await to countUsersByRole

* fix: paginate listRoles, null-guard permissions handler, fix export ordering

- Add limit/offset/total pagination to listRoles matching the groups pattern
- Add countRoles data-layer method
- Omit permissions from listRoles select (getRole returns full document)
- Null-guard re-fetched role in updateRolePermissionsHandler
- Move interleaved export below all imports in methods/index.ts

* fix: address review findings — race safety, validation DRY, type accuracy, test coverage

- Add post-write admin count verification in removeRoleMember to prevent
  zero-admin race condition (TOCTOU → rollback if count hits 0)
- Make IRole.description optional; backfill in initializeRoles for
  pre-existing roles that lack the field (.lean() bypasses defaults)
- Extract parsePagination, validateNameParam, validateRoleName, and
  validateDescription helpers to eliminate duplicated validation
- Add validateNameParam guard to all 7 handlers reading req.params.name
- Catch 11000 in updateRoleByName and surface as 409 via RoleConflictError
- Add idempotent skip in addRoleMember when user already has target role
- Verify updateRolePermissions test asserts response body
- Add data-layer tests: listRoles sort/pagination/projection, countRoles,
  and createRoleByName 11000 duplicate key race

* fix: defensive rollback in removeRoleMember, type/style cleanup, test coverage

- Wrap removeRoleMember post-write admin rollback in try/catch so a
  transient DB failure cannot leave the system with zero administrators
- Replace double `as unknown[] as IRole[]` cast with `.lean<IRole[]>()`
- Type parsePagination param explicitly; extract DEFAULT/MAX page constants
- Preserve original error cause in updateRoleByName re-throw
- Add test for rollback failure path in removeRoleMember (returns 400)
- Add test for pre-existing roles missing description field (.lean())

* chore: bump @librechat/data-schemas to 0.0.47

* fix: stale cache on rename, extract renameRole helper, shared pagination, cleanup

- Fix updateRoleByName cache bug: invalidate old key and populate new key
  when updates.name differs from roleName (prevents stale cache after rename)
- Extract renameRole helper to eliminate mutable outer-scope state flags
  (isRename, trimmedName, migrationRan) in updateRoleHandler
- Unify system-role protection to 403 for both rename-from and rename-to
- Extract parsePagination to shared admin/pagination.ts; use in both
  roles.ts and groups.ts
- Extract name.trim() to local const in createRoleByName (was called 5×)
- Remove redundant findOne pre-check in deleteRoleByName
- Replace getUserModel closure with local const declarations
- Remove redundant description ?? '' in createRoleHandler (schema default)
- Add doc comment on updateRolePermissionsHandler noting cache dependency
- Add data-layer tests for cache rename behavior (old key null, new key set)

* fix: harden role guards, add User.role index, validate names, improve tests

- Add index on User.role field for efficient member queries at scale
- Replace fragile SystemRoles key lookup with value-based Set check (6 sites)
- Elevate rename rollback failure logging to CRITICAL (matches removeRoleMember)
- Guard removeRoleMember against non-ADMIN system roles (403 for USER)
- Fix parsePagination limit=0 gotcha: use parseInt + NaN check instead of ||
- Add control character and reserved path segment validation to role names
- Simplify validateRoleName: remove redundant casts and dead conditions
- Add JSDoc to deleteRoleByName documenting non-atomic window
- Split mixed value+type import in methods/index.ts per AGENTS.md
- Add 9 new tests: permissions assertion, combined rename+desc, createRole
  with permissions, pagination edge cases, control char/reserved name
  rejection, system role removeRoleMember guard

* fix: exact-case reserved name check, consistent validation, cleaner createRole

- Remove .toLowerCase() from reserved name check so only exact matches
  (members, permissions) are rejected, not legitimate names like "Members"
- Extract trimmed const in validateRoleName for consistent validation
- Add control char check to validateNameParam for parity with body validation
- Build createRole roleData conditionally to avoid passing description: undefined
- Expand deleteRoleByName JSDoc documenting self-healing design and no-op trade-off

* fix: scope rename rollback to only migrated users, prevent cross-role corruption

Capture user IDs before forward migration so the rollback path only
reverts users this request actually moved. Previously the rollback called
updateUsersByRole(newName, currentName) which would sweep all users with
the new role — including any independently assigned by a concurrent admin
request — causing silent cross-role data corruption.

Adds findUserIdsByRole and updateUsersRoleByIds to the data layer.
Extracts rollbackMigratedUsers helper to deduplicate rollback sites.

* fix: guard last admin in addRoleMember to prevent zero-admin lockout

Since each user has exactly one role, addRoleMember implicitly removes
the user from their current role. Without a guard, reassigning the sole
admin to a non-admin role leaves zero admins and locks out admin
management. Adds the same countUsersByRole check used in removeRoleMember.

* fix: wire findUserIdsByRole and updateUsersRoleByIds into roles route

The scoped rollback deps added in c89b5db were missing from the route
DI wiring, causing renameRole to call undefined and return a 500.

* fix: post-write admin guard in addRoleMember, compound role index, review cleanup

- Add post-write admin count check + rollback to addRoleMember to match
  removeRoleMember's two-phase TOCTOU protection (prevents zero-admin via
  concurrent requests)
- Replace single-field User.role index with compound { role: 1, tenantId: 1 }
  to align with existing multi-tenant index pattern (email, OAuth IDs)
- Narrow listRoles dep return type to RoleListItem (projected fields only)
- Refactor validateDescription to early-return style per AGENTS.md
- Remove redundant double .lean() in updateRoleByName
- Document rename snapshot race window in renameRole JSDoc
- Document cache null-set behavior in deleteRoleByName
- Add routing-coupling comment on RESERVED_ROLE_NAMES
- Add test for addRoleMember post-write rollback

* fix: review cleanup — system-role guard, type safety, JSDoc accuracy, tests

- Add system-role guard to addRoleMember: block direct assignment to
  non-ADMIN system roles (403), symmetric with removeRoleMember
- Fix RESERVED_ROLE_NAMES comment: explain semantic URL ambiguity, not
  a routing conflict (Express resolves single vs multi-segment correctly)
- Replace _id: unknown with Types.ObjectId | string per AGENTS.md
- Narrow listRoles data-layer return type to Pick<IRole, 'name' | 'description'>
  to match the actual .select() projection
- Move updateRoleHandler param check inside try/catch for consistency
- Include user IDs in all CRITICAL rollback failure logs for operator recovery
- Clarify deleteRoleByName JSDoc: replace "self-healing" with "idempotent",
  document that recovery requires caller retry
- Add tests: system-role guard, promote non-admin to ADMIN,
  findUserIdsByRole throw prevents migration

* fix: include _id in listRoles return type to match RoleListItem

Pick<IRole, 'name' | 'description'> omits _id, making it incompatible
with the handler dep's RoleListItem which requires _id.

* fix: case-insensitive system role guard, reject null permissions, check updateUser result

- System role name checks now use case-insensitive comparison via
  toUpperCase() — prevents creating 'admin' or 'user' which would
  collide with the legacy roles route that uppercases params
- Reject permissions: null in createRole (typeof null === 'object'
  was bypassing the validation)
- Check updateUser return in addRoleMember — return 404 if the user
  was deleted between the findUser and updateUser calls

* fix: check updateUser return in removeRoleMember for concurrent delete safety

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-27 15:44:47 -04:00
Dustin Healy
2e3d66cfe2
👥 feat: Admin Groups API Endpoints (#12387)
* feat: add listGroups and deleteGroup methods to userGroup

* feat: add admin groups handler factory and Express routes

* fix: address convention violations in admin groups handlers

* fix: address Copilot review findings in admin groups handlers

- Escape regex in listGroups to prevent injection/ReDoS
- Validate ObjectId format in all handlers accepting id/userId params
- Replace N+1 findUser loop with batched findUsers query
- Remove unused findGroupsByMemberId from dep interface
- Map Mongoose ValidationError to 400 in create/update handlers
- Validate name in updateGroupHandler (reject empty/whitespace)
- Handle null updateGroupById result (race condition)
- Tighten error message matching in add/remove member handlers

* test: add unit tests for admin groups handlers

* fix: address code review findings for admin groups

Atomic delete/update handlers (single DB trip), pass through
idOnTheSource, add removeMemberById for non-ObjectId members,
deduplicate member results, fix error message exposure, add hard
cap/sort to listGroups, replace GroupListFilter with Pick of
GroupFilterOptions, validate memberIds as array, trim name in
update, fix import order, and improve test hygiene with fresh
IDs per test.

* fix: cascade cleanup, pagination, and test coverage for admin groups

Add deleteGrantsForPrincipal to systemGrant data layer and wire cascade
cleanup (Config, AclEntry, SystemGrant) into deleteGroupHandler. Add
limit/offset pagination to getGroupMembers. Guard empty PATCH bodies with
400. Remove dead type guard and unnecessary type cast. Add 11 new tests
covering cascade delete, idempotent member removal, empty update, search
filter, 500 error paths, and pagination.

* fix: harden admin groups with cascade resilience, type safety, and fallback removal

Wrap cascade cleanup in inner try/catch so partial failure logs but still
returns 200 (group is already deleted). Replace Record<string, unknown> on
deleteAclEntries with proper typed filter. Log warning for unmapped user
ObjectIds in createGroup memberIds. Add removeMemberById fallback when
removeUserFromGroup throws User not found for ObjectId-format userId.
Extract VALID_GROUP_SOURCES constant. Add 3 new tests (60 total).

* refactor: add countGroups, pagination, and projection type to data layer

Extract buildGroupQuery helper, add countGroups method, support
limit/offset/skip in listGroups, standardize session handling to
.session(session ?? null), and tighten projection parameter from
Record<string, unknown> to Record<string, 0 | 1>.

* fix: cascade resilience, pagination, validation, and error clarity for admin groups

- Use Promise.allSettled for cascade cleanup so all steps run even if
  one fails; log individual rejections
- Echo deleted group id in delete response
- Add countGroups dep and wire limit/offset pagination for listGroups
- Deduplicate memberIds before computing total in getGroupMembers
- Use { memberIds: 1 } projection in getGroupMembers
- Cap memberIds at 500 entries in createGroup
- Reject search queries exceeding 200 characters
- Clarify addGroupMember error for non-ObjectId userId
- Document deleted-user fallback limitation in removeGroupMember

* test: extend handler and DB-layer test coverage for admin groups

Handler tests: projection assertion, dedup total, memberIds cap,
search max length, non-ObjectId memberIds passthrough, cascade partial
failure resilience, dedup scenarios, echo id in delete response.

DB-layer tests: listGroups sort/filter/pagination, countGroups,
deleteGroup, removeMemberById, deleteGrantsForPrincipal.

* fix: cast group principalId to ObjectId for ACL entry cleanup

deleteAclEntries is a thin deleteMany wrapper with no type casting,
but grantPermission stores group principalId as ObjectId. Passing the
raw string from req.params would leave orphaned ACL entries on group
deletion.

* refactor: remove redundant pagination clamping from DB listGroups

Handler already clamps limit/offset at the API boundary. The DB
method is a general-purpose building block and should not re-validate.

* fix: add source and name validation, import order, and test coverage for admin groups

- Validate source against VALID_GROUP_SOURCES in createGroupHandler
- Cap name at 500 characters in both create and update handlers
- Document total as upper bound in getGroupMembers response
- Document ObjectId requirement for deleteAclEntries in cascade
- Fix import ordering in test file (local value after type imports)
- Add tests for updateGroup with description, email, avatar fields
- Add tests for invalid source and name max-length in both handlers

* fix: add field length caps, flatten nested try/catch, and fix logger level in admin groups

Add max-length validation for description, email, avatar, and
idOnTheSource in create/update handlers. Extract removeObjectIdMember
helper to flatten nested try/catch per never-nesting convention. Downgrade
unmapped-memberIds log from error to warn. Fix type import ordering and
add missing await in removeMemberById for consistency.
2026-03-26 17:36:18 -04:00
Danny Avila
9f6d8c6e93
🧵 feat: ALS Context Middleware, Tenant Threading, and Config Cache Invalidation (#12407)
* feat: add tenant context middleware for ALS-based isolation

Introduces tenantContextMiddleware that propagates req.user.tenantId
into AsyncLocalStorage, activating the Mongoose applyTenantIsolation
plugin for all downstream DB queries within a request.

- Strict mode (TENANT_ISOLATION_STRICT=true) returns 403 if no tenantId
- Non-strict mode passes through for backward compatibility
- No-op for unauthenticated requests
- Includes 6 unit tests covering all paths

* feat: register tenant middleware and wrap startup/auth in runAsSystem()

- Register tenantContextMiddleware in Express app after capability middleware
- Wrap server startup initialization in runAsSystem() for strict mode compat
- Wrap auth strategy getAppConfig() calls in runAsSystem() since they run
  before user context is established (LDAP, SAML, OpenID, social login, AuthService)

* feat: thread tenantId through all getAppConfig callers

Pass tenantId from req.user to getAppConfig() across all callers that
have request context, ensuring correct per-tenant cache key resolution.

Also fixes getBaseConfig admin endpoint to scope to requesting admin's
tenant instead of returning the unscoped base config.

Files updated:
- Controllers: UserController, PluginController
- Middleware: checkDomainAllowed, balance
- Routes: config
- Services: loadConfigModels, loadDefaultModels, getEndpointsConfig, MCP
- Audio services: TTSService, STTService, getVoices, getCustomConfigSpeech
- Admin: getBaseConfig endpoint

* feat: add config cache invalidation on admin mutations

- Add clearOverrideCache(tenantId?) to flush per-principal override caches
  by enumerating Keyv store keys matching _OVERRIDE_: prefix
- Add invalidateConfigCaches() helper that clears base config, override
  caches, tool caches, and endpoint config cache in one call
- Wire invalidation into all 5 admin config mutation handlers
  (upsert, patch, delete field, delete overrides, toggle active)
- Add strict mode warning when __default__ tenant fallback is used
- Add 3 new tests for clearOverrideCache (all/scoped/base-preserving)

* chore: update getUserPrincipals comment to reflect ALS-based tenant filtering

The TODO(#12091) about missing tenantId filtering is resolved by the
tenant context middleware + applyTenantIsolation Mongoose plugin.
Group queries are now automatically scoped by tenantId via ALS.

* fix: replace runAsSystem with baseOnly for pre-tenant code paths

App configs are tenant-owned — runAsSystem() would bypass tenant
isolation and return cross-tenant DB overrides. Instead, add
baseOnly option to getAppConfig() that returns YAML-derived config
only, with zero DB queries.

All startup code, auth strategies, and MCP initialization now use
getAppConfig({ baseOnly: true }) to get the YAML config without
touching the Config collection.

* fix: address PR review findings — middleware ordering, types, cache safety

- Chain tenantContextMiddleware inside requireJwtAuth after passport auth
  instead of global app.use() where req.user is always undefined (Finding 1)
- Remove global tenantContextMiddleware registration from index.js
- Update BalanceMiddlewareOptions to include tenantId, remove redundant cast (Finding 4)
- Add warning log when clearOverrideCache cannot enumerate keys on Redis (Finding 3)
- Use startsWith instead of includes for cache key filtering (Finding 12)
- Use generator loop instead of Array.from for key enumeration (Finding 3)
- Selective barrel export — exclude _resetTenantMiddlewareStrictCache (Finding 5)
- Move isMainThread check to module level, remove per-request check (Finding 9)
- Move mid-file require to top of app.js (Finding 8)
- Parallelize invalidateConfigCaches with Promise.all (Finding 10)
- Remove clearOverrideCache from public app.js exports (internal only)
- Strengthen getUserPrincipals comment re: ALS dependency (Finding 2)

* fix: restore runAsSystem for startup DB ops, consolidate require, clarify baseOnly

- Restore runAsSystem() around performStartupChecks, updateInterfacePermissions,
  initializeMCPs, and initializeOAuthReconnectManager — these make Mongoose
  queries that need system context in strict tenant mode (NEW-3)
- Consolidate duplicate require('@librechat/api') in requireJwtAuth.js (NEW-1)
- Document that baseOnly ignores role/userId/tenantId in JSDoc (NEW-2)

* test: add requireJwtAuth tenant chaining + invalidateConfigCaches tests

- requireJwtAuth: 5 tests verifying ALS tenant context is set after
  passport auth, isolated between concurrent requests, and not set
  when user has no tenantId (Finding 6)
- invalidateConfigCaches: 4 tests verifying all four caches are cleared,
  tenantId is threaded through, partial failure is handled gracefully,
  and operations run in parallel via Promise.all (Finding 11)

* fix: address Copilot review — passport errors, namespaced cache keys, /base scoping

- Forward passport errors in requireJwtAuth before entering tenant
  middleware — prevents silent auth failures from reaching handlers (P1)
- Account for Keyv namespace prefix in clearOverrideCache — stored keys
  are namespaced as "APP_CONFIG:_OVERRIDE_:..." not "_OVERRIDE_:...",
  so override caches were never actually matched/cleared (P2)
- Remove role from getBaseConfig — /base should return tenant-scoped
  base config, not role-merged config that drifts per admin role (P2)
- Return tenantStorage.run() for cleaner async semantics
- Update mock cache in service.spec.ts to simulate Keyv namespacing

* fix: address second review — cache safety, code quality, test reliability

- Decouple cache invalidation from mutation response: fire-and-forget
  with logging so DB mutation success is not masked by cache failures
- Extract clearEndpointConfigCache helper from inline IIFE
- Move isMainThread check to lazy once-per-process guard (no import
  side effect)
- Memoize process.env read in overrideCacheKey to avoid per-request
  env lookups and log flooding in strict mode
- Remove flaky timer-based parallelism assertion, use structural check
- Merge orphaned double JSDoc block on getUserPrincipals
- Fix stale [getAppConfig] log prefix → [ensureBaseConfig]
- Fix import order in tenant.spec.ts (package types before local values)
- Replace "Finding 1" reference with self-contained description
- Use real tenantStorage primitives in requireJwtAuth spec mock

* fix: move JSDoc to correct function after clearEndpointConfigCache extraction

* refactor: remove Redis SCAN from clearOverrideCache, rely on TTL expiry

Redis SCAN causes 60s+ stalls under concurrent load (see #12410).
APP_CONFIG defaults to FORCED_IN_MEMORY_CACHE_NAMESPACES, so the
in-memory store.keys() path handles the standard case. When APP_CONFIG
is Redis-backed, overrides expire naturally via overrideCacheTtl (60s
default) — an acceptable window for admin config mutations.

* fix: remove return from tenantStorage.run to satisfy void middleware signature

* fix: address second review — cache safety, code quality, test reliability

- Switch invalidateConfigCaches from Promise.all to Promise.allSettled
  so partial failures are logged individually instead of producing one
  undifferentiated error (Finding 3)
- Gate overrideCacheKey strict-mode warning behind a once-per-process
  flag to prevent log flooding under load (Finding 4)
- Add test for passport error forwarding in requireJwtAuth — the
  if (err) { return next(err) } branch now has coverage (Finding 5)
- Add test for real partial failure in invalidateConfigCaches where
  clearAppConfigCache rejects (not just the swallowed endpoint error)

* chore: reorder imports in index.js and app.js for consistency

- Moved logger and runAsSystem imports to maintain a consistent import order across files.
- Improved code readability by ensuring related imports are grouped together.
2026-03-26 17:35:00 -04:00
Danny Avila
4b6d68b3b5
🎛️ feat: DB-Backed Per-Principal Config System (#12354)
* ✨ feat: Add Config schema, model, and methods for role-based DB config overrides

Add the database foundation for principal-based configuration overrides
(user, group, role) in data-schemas. Includes schema with tenantId and
tenant isolation, CRUD methods, and barrel exports.

* 🔧 fix: Add shebang and enforce LF line endings for git hooks

The pre-commit hook was missing #!/bin/sh, and core.autocrlf=true was
converting it to CRLF, both causing "Exec format error" on Windows.
Add .gitattributes to force LF for .husky/* and *.sh files.

* ✨ feat: Add admin config API routes with section-level capability checks

Add /api/admin/config endpoints for managing per-principal config
overrides (user, group, role). Handlers in @librechat/api use DI pattern
with section-level hasConfigCapability checks for granular access control.

Supports full overrides replacement, per-field PATCH via dot-paths, field
deletion, toggle active, and listing.

* 🐛 fix: Move deleteConfigField fieldPath from URL param to request body

The path-to-regexp wildcard syntax (:fieldPath(*)) is not supported by
the version used in Express. Send fieldPath in the DELETE request body
instead, which also avoids URL-encoding issues with dotted paths.

* ✨ feat: Wire config resolution into getAppConfig with override caching

Add mergeConfigOverrides utility in data-schemas for deep-merging DB
config overrides into base AppConfig by priority order.

Update getAppConfig to query DB for applicable configs when role/userId
is provided, with short-TTL caching and a hasAnyConfigs feature flag
for zero-cost when no DB configs exist.

Also: add unique compound index on Config schema, pass userId from
config middleware, and signal config changes from admin API handlers.

* 🔄 refactor: Extract getAppConfig logic into packages/api as TS service

Move override resolution, caching strategy, and signalConfigChange from
api/server/services/Config/app.js into packages/api/src/app/appConfigService.ts
using the DI factory pattern (createAppConfigService). The JS file becomes
a thin wiring layer injecting loadBaseConfig, cache, and DB dependencies.

* 🧹 chore: Rename configResolution.ts to resolution.ts

* ✨ feat: Move admin types & capabilities to librechat-data-provider

Move SystemCapabilities, CapabilityImplications, and utility functions
(hasImpliedCapability, expandImplications) from data-schemas to
data-provider so they are available to external consumers like the
admin panel without a data-schemas dependency.

Add API-friendly admin types: TAdminConfig, TAdminSystemGrant,
TAdminAuditLogEntry, TAdminGroup, TAdminMember, TAdminUserSearchResult,
TCapabilityCategory, and CAPABILITY_CATEGORIES.

data-schemas re-exports these from data-provider and extends with
config-schema-derived types (ConfigSection, SystemCapability union).

Bump version to 0.8.500.

* feat: Add JSON-serializable admin config API response types to data-schemas

Add AdminConfig, AdminConfigListResponse, AdminConfigResponse, and
AdminConfigDeleteResponse types so both LibreChat API handlers and the
admin panel can share the same response contract. Bump version to 0.0.41.

* refactor: Move admin capabilities & types from data-provider to data-schemas

SystemCapabilities, CapabilityImplications, utility functions,
CAPABILITY_CATEGORIES, and admin API response types should not be in
data-provider as it gets compiled into the frontend bundle, exposing
the capability surface. Moved everything to data-schemas (server-only).

All consumers already import from @librechat/data-schemas, so no
import changes needed elsewhere. Consolidated duplicate AdminConfig
type (was in both config.ts and admin.ts).

* chore: Bump @librechat/data-schemas to 0.0.42

* refactor: Reorganize admin capabilities into admin/ and types/admin.ts

Split systemCapabilities.ts following data-schemas conventions:
- Types (BaseSystemCapability, SystemCapability, AdminConfig, etc.)
  → src/types/admin.ts
- Runtime code (SystemCapabilities, CapabilityImplications, utilities)
  → src/admin/capabilities.ts

Revert data-provider version to 0.8.401 (no longer modified).

* chore: Fix import ordering, rename appConfigService to service

- Rename app/appConfigService.ts → app/service.ts (directory provides context)
- Fix import order in admin/config.ts, types/admin.ts, types/config.ts
- Add naming convention to AGENTS.md

* feat: Add DB base config support (role/__base__)

- Add BASE_CONFIG_PRINCIPAL_ID constant for reserved base config doc
- getApplicableConfigs always includes __base__ in queries
- getAppConfig queries DB even without role/userId when DB configs exist
- Bump @librechat/data-schemas to 0.0.43

* fix: Address PR review issues for admin config

- Add listAllConfigs method; listConfigs endpoint returns all active
  configs instead of only __base__
- Normalize principalId to string in all config methods to prevent
  ObjectId vs string mismatch on user/group lookups
- Block __proto__ and all dunder-prefixed segments in field path
  validation to prevent prototype pollution
- Fix configVersion off-by-one: default to 0, guard pre('save') with
  !isNew, use $inc on findOneAndUpdate
- Remove unused getApplicableConfigs from admin handler deps

* fix: Enable tree-shaking for data-schemas, bump packages

- Switch data-schemas Rollup output to preserveModules so each source
  file becomes its own chunk; consumers (admin panel) can now import
  just the modules they need without pulling in winston/mongoose/etc.
- Add sideEffects: false to data-schemas package.json
- Bump data-schemas to 0.0.44, data-provider to 0.8.402

* feat: add capabilities subpath export to data-schemas

Adds `@librechat/data-schemas/capabilities` subpath export so browser
consumers can import BASE_CONFIG_PRINCIPAL_ID and capability constants
without pulling in Node.js-only modules (winston, async_hooks, etc.).

Bump version to 0.0.45.

* fix: include dist/ in data-provider npm package

Add explicit files field so npm includes dist/types/ in the published
package. Without this, the root .gitignore exclusion of dist/ causes
npm to omit type declarations, breaking TypeScript consumers.

* chore: bump librechat-data-provider to 0.8.403

* feat: add GET /api/admin/config/base for raw AppConfig

Returns the full AppConfig (YAML + DB base merged) so the admin panel
can display actual config field values and structure. The startup config
endpoint (/api/config) returns TStartupConfig which is a different shape
meant for the frontend app.

* chore: imports order

* fix: address code review findings for admin config

Critical:
- Fix clearAppConfigCache: was deleting from wrong cache store (CONFIG_STORE
  instead of APP_CONFIG), now clears BASE and HAS_DB_CONFIGS keys
- Eliminate race condition: patchConfigField and deleteConfigField now use
  atomic MongoDB $set/$unset with dot-path notation instead of
  read-modify-write cycles, removing the lost-update bug entirely
- Add patchConfigFields and unsetConfigField atomic DB methods

Major:
- Reorder cache check before principal resolution in getAppConfig so
  getUserPrincipals DB query only fires on cache miss
- Replace '' as ConfigSection with typed BROAD_CONFIG_ACCESS constant
- Parallelize capability checks with Promise.all instead of sequential
  awaits in for loops
- Use loose equality (== null) for cache miss check to handle both null
  and undefined returns from cache implementations
- Set HAS_DB_CONFIGS_KEY to true on successful config fetch

Minor:
- Remove dead pre('save') hook from config schema (all writes use
  findOneAndUpdate which bypasses document hooks)
- Consolidate duplicate type imports in resolution.ts
- Remove dead deepGet/deepSet/deepUnset functions (replaced by atomic ops)
- Add .sort({ priority: 1 }) to getApplicableConfigs query
- Rename _impliedBy to impliedByMap

* fix: self-referencing BROAD_CONFIG_ACCESS constant

* fix: replace type-cast sentinel with proper null parameter

Update hasConfigCapability to accept ConfigSection | null where null
means broad access check (MANAGE_CONFIGS or READ_CONFIGS only).
Removes the '' as ConfigSection type lie from admin config handlers.

* fix: remaining review findings + add tests

- listAllConfigs accepts optional { isActive } filter so admin listing
  can show inactive configs (#9)
- Standardize session application to .session(session ?? null) across
  all config DB methods (#15)
- Export isValidFieldPath and getTopLevelSection for testability
- Add 38 tests across 3 spec files:
  - config.spec.ts (api): path validation, prototype pollution rejection
  - resolution.spec.ts: deep merge, priority ordering, array replacement
  - config.spec.ts (data-schemas): full CRUD, ObjectId normalization,
    atomic $set/$unset, configVersion increment, toggle, __base__ query

* fix: address second code review findings

- Fix cross-user cache contamination: overrideCacheKey now handles
  userId-without-role case with its own cache key (#1)
- Add broad capability check before DB lookup in getConfig to prevent
  config existence enumeration (#2/#3)
- Move deleteConfigField fieldPath from request body to query parameter
  for proxy/load balancer compatibility (#5)
- Derive BaseSystemCapability from SystemCapabilities const instead of
  manual string union (#6)
- Return 201 on upsert creation, 200 on update (#11)
- Remove inline narration comments per AGENTS.md (#12)
- Type overrides as Partial<TCustomConfig> in DB methods and handler
  deps (#13)
- Replace double as-unknown-as casts in resolution.ts with generic
  deepMerge<T> (#14)
- Make override cache TTL injectable via AppConfigServiceDeps (#16)
- Add exhaustive never check in principalModel switch (#17)

* fix: remaining review findings — tests, rename, semantics

- Rename signalConfigChange → markConfigsDirty with JSDoc documenting
  the stale-window tradeoff and overrideCacheTtl knob
- Fix DEFAULT_OVERRIDE_CACHE_TTL naming convention
- Add createAppConfigService tests (14 cases): cache behavior, feature
  flag, cross-user key isolation, fallback on error, markConfigsDirty
- Add admin handler integration tests (13 cases): auth ordering,
  201/200 on create/update, fieldPath from query param, markConfigsDirty
  calls, capability checks

* fix: global flag corruption + empty overrides auth bypass

- Remove HAS_DB_CONFIGS_KEY=false optimization: a scoped query returning
  no configs does not mean no configs exist globally. Setting the flag
  false from a per-principal query short-circuited all subsequent users.
- Add broad manage capability check before section checks in
  upsertConfigOverrides: empty overrides {} no longer bypasses auth.

* test: add regression and invariant tests for config system

Regression tests:
- Bug 1: User A's empty result does not short-circuit User B's overrides
- Bug 2: Empty overrides {} returns 403 without MANAGE_CONFIGS

Invariant tests (applied across ALL handlers):
- All 5 mutation handlers call markConfigsDirty on success
- All 5 mutation handlers return 401 without auth
- All 5 mutation handlers return 403 without capability
- All 3 read handlers return 403 without capability

* fix: third review pass — all findings addressed

Service (service.ts):
- Restore HAS_DB_CONFIGS=false for base-only queries (no role/userId)
  so deployments with zero DB configs skip DB queries (#1)
- Resolve cache once at factory init instead of per-invocation (#8)
- Use BASE_CONFIG_PRINCIPAL_ID constant in overrideCacheKey (#10)
- Add JSDoc to clearAppConfigCache documenting stale-window (#4)
- Fix log message to not say "from YAML" (#14)

Admin handlers (config.ts):
- Use configVersion===1 for 201 vs 200, eliminating TOCTOU race (#2)
- Add Array.isArray guard on overrides body (#5)
- Import CapabilityUser from capabilities.ts, remove duplicate (#6)
- Replace as-unknown-as cast with targeted type assertion (#7)
- Add MAX_PATCH_ENTRIES=100 cap on entries array (#15)
- Reorder deleteConfigField to validate principalType first (#12)
- Export CapabilityUser from middleware/capabilities.ts

DB methods (config.ts):
- Remove isActive:true from patchConfigFields to prevent silent
  reactivation of disabled configs (#3)

Schema (config.ts):
- Change principalId from Schema.Types.Mixed to String (#11)

Tests:
- Add patchConfigField unsafe fieldPath rejection test (#9)
- Add base-only HAS_DB_CONFIGS=false test (#1)
- Update 201/200 tests to use configVersion instead of findConfig (#2)

* fix: add read handler 401 invariant tests + document flag behavior

- Add invariant: all 3 read handlers return 401 without auth
- Document on markConfigsDirty that HAS_DB_CONFIGS stays true after
  all configs are deleted until clearAppConfigCache or restart

* fix: remove HAS_DB_CONFIGS false optimization entirely

getApplicableConfigs([]) only queries for __base__, not all configs.
A deployment with role/group configs but no __base__ doc gets the
flag poisoned to false by a base-only query, silently ignoring all
scoped overrides. The optimization is not safe without a comprehensive
Config.exists() check, which adds its own DB cost. Removed entirely.

The flag is now write-once-true (set when configs are found or by
markConfigsDirty) and only cleared by clearAppConfigCache/restart.

* chore: reorder import statements in app.js for clarity

* refactor: remove HAS_DB_CONFIGS_KEY machinery entirely

The three-state flag (false/null/true) was the source of multiple bugs
across review rounds. Every attempt to safely set it to false was
defeated by getApplicableConfigs querying only a subset of principals.

Removed: HAS_DB_CONFIGS_KEY constant, all reads/writes of the flag,
markConfigsDirty (now a no-op concept), notifyChange wrapper, and all
tests that seeded false manually.

The per-user/role TTL cache (overrideCacheTtl, default 60s) is the
sole caching mechanism. On cache miss, getApplicableConfigs queries
the DB. This is one indexed query per user per TTL window — acceptable
for the config override use case.

* docs: rewrite admin panel remaining work with current state

* perf: cache empty override results to avoid repeated DB queries

When getApplicableConfigs returns no configs for a principal, cache
baseConfig under their override key with TTL. Without this, every
user with no per-principal overrides hits MongoDB on every request
after the 60s cache window expires.

* fix: add tenantId to cache keys + reject PUBLIC principal type

- Include tenantId in override cache keys to prevent cross-tenant
  config contamination. Single-tenant deployments (tenantId undefined)
  use '_' as placeholder — no behavior change for them.
- Reject PrincipalType.PUBLIC in admin config validation — PUBLIC has
  no PrincipalModel and is never resolved by getApplicableConfigs,
  so config docs for it would be dead data.
- Config middleware passes req.user.tenantId to getAppConfig.

* fix: fourth review pass findings

DB methods (config.ts):
- findConfigByPrincipal accepts { includeInactive } option so admin
  GET can retrieve inactive configs (#5)
- upsertConfig catches E11000 duplicate key on concurrent upserts and
  retries without upsert flag (#2)
- unsetConfigField no longer filters isActive:true, consistent with
  patchConfigFields (#11)
- Typed filter objects replace Record<string, unknown> (#12)

Admin handlers (config.ts):
- patchConfigField: serial broad capability check before Promise.all
  to pre-warm ALS principal cache, preventing N parallel DB calls (#3)
- isValidFieldPath rejects leading/trailing dots and consecutive
  dots (#7)
- Duplicate fieldPaths in patch entries return 400 (#8)
- DEFAULT_PRIORITY named constant replaces hardcoded 10 (#14)
- Admin getConfig and patchConfigField pass includeInactive to
  findConfigByPrincipal (#5)
- Route import uses barrel instead of direct file path (#13)

Resolution (resolution.ts):
- deepMerge has MAX_MERGE_DEPTH=10 guard to prevent stack overflow
  from crafted deeply nested configs (#4)

* fix: final review cleanup

- Remove ADMIN_PANEL_REMAINING.md (local dev notes with Windows paths)
- Add empty-result caching regression test
- Add tenantId to AdminConfigDeps.getAppConfig type
- Restore exhaustive never check in principalModel switch
- Standardize toggleConfigActive session handling to options pattern

* fix: validate priority in patchConfigField handler

Add the same non-negative number validation for priority that
upsertConfigOverrides already has. Without this, invalid priority
values could be stored via PATCH and corrupt merge ordering.

* chore: remove planning doc from PR

* fix: correct stale cache key strings in service tests

* fix: clean up service tests and harden tenant sentinel

- Remove no-op cache delete lines from regression tests
- Change no-tenant sentinel from '_' to '__default__' to avoid
  collision with a real tenant ID when multi-tenancy is enabled
- Remove unused CONFIG_STORE from AppConfigServiceDeps

* chore: bump @librechat/data-schemas to 0.0.46

* fix: block prototype-poisoning keys in deepMerge

Skip __proto__, constructor, and prototype keys during config merge
to prevent prototype pollution via PUT /api/admin/config overrides.
2026-03-25 19:39:29 -04:00
Marco Beretta
ccd049d8ce
📁 refactor: Prompts UI (#11570)
* style: enhance prompts UI with new components and improved structure; add CreatePromptButton and AutoSendPrompt; refactor GroupSidePanel and PromptsAccordion

* refactor(Prompts): move button components to buttons/ subdirectory

* refactor(Prompts): move dialog components to dialogs/ subdirectory

* refactor(Prompts): move display components to display/ subdirectory

* refactor(Prompts): move editor components to editor/ subdirectory

* refactor(Prompts): move field components to fields/ subdirectory

* refactor(Prompts): move form components to forms/ subdirectory

* refactor(Prompts): move layout components to layouts/ subdirectory

* refactor(Prompts): move list components to lists/ subdirectory

* refactor(Prompts): move sidebar components to sidebar/ subdirectory

* refactor(Prompts): move utility components to utils/ subdirectory

* refactor(Prompts): update main exports and external imports

* refactor(Prompts): fix class name typo in AutoSendPrompt

* refactor(Prompts): reorganize exports and imports order across components

* refactor(Prompts): reorder exports for better organization and clarity

* refactor(Buttons): enhance prompts accessibility with aria-labels and update translations

* refactor(AdminSettings): reorganize imports and improve form structure for clarity

* refactor(Dialogs): reorganize imports for consistency and clarity across DeleteVersion, SharePrompt, and VariableDialog components

* refactor(Dialogs): enhance prompts accessibility with aria-labels

* refactor(Display): enhance prompt components and accessibility features

* refactor(.gitignore): add Playwright MCP directory

* refactor(Preview): enhance prompt components, improve layout, and add accessibility features

* refactor(Prompts): enhance variable handling, improve accessibility, and update UI components

* refactor(Prompts): enhance loading state handling and improve accessibility in PromptName component

* refactor(Prompts): streamline special variable handling, improve icon management, and enhance UI components

* refactor(Prompts): update AdvancedSwitch component to use Radio for mode selection, enhance PromptName with tooltips, and improve layout in PromptForm

* refactor(Prompts): enhance VersionCard and VersionBadge components for improved UI and accessibility, update loading state handling in VersionsPanel

* refactor(Prompts): improve layout and styling of VersionCard component for better visual alignment and clarity

* refactor(DeleteVersion): update text color for confirmation prompt in DeleteConfirmDialog

* refactor(Prompts): add configurations for always make production and auto-send prompts, update localization strings for clarity

* refactor(Prompts): enhance layout and styling in CategorySelector, CreatePromptForm, and List components for improved responsiveness and clarity

* refactor(Prompts): enhance PromptDetailHeader and ChatGroupItem components, add shared prompt indication, and remove unused PromptMetadata component

* refactor(Prompts): implement prompt group usage tracking, update sorting logic, and enhance related components

* fix(Prompts): security, performance, and pagination fixes

- Fix cursor pagination skipping/duplicating items by including
  numberOfGenerations in cursor condition to match sort order
- Close NoSQL injection vector via otherFilters rest spread in
  GET /all, GET /groups, and buildPromptGroupFilter
- Validate groupId as ObjectId before passing to query (GET /)
- Add prompt body validation in addPromptToGroup (type + text)
- Return 404 instead of 500 for missing group in POST /use
- Combine data + count into single $facet aggregation
- Add compound index {numberOfGenerations, updatedAt, _id}
- Add index on prompt.author for deleteUserPrompts
- Update useRecordPromptUsage to refresh client caches
- Replace console.error with logger.error

* refactor(PromptForm): remove console warning for unselected prompt in VersionsPanel

* refactor(Prompts): improve error handling for groupId and streamline usage tracking

* refactor(.gitignore): add CLAUDE.md to ignore list

* refactor(Prompts): streamline prompt components by removing unused variables and enhancing props structure

* refactor(Prompts): fix sort stability, keyboard handling, and remove dead code

Add _id tiebreaker to prompt group sort pipelines for deterministic
pagination ordering. Prevent default browser scroll on Space key in
PromptEditor preview mode. Remove unused blurTimeoutRef and its
onMutate callback from DashGroupItem.

* refactor(Prompts): enhance groupId validation and improve prompt group aggregation handling

* fix: aria-hidden, API fixes, accessibility improvements

* fix: ACL author filter, mobile guard, semantic HTML, and add useFocusTrap hook

- Remove author filter from patchPromptGroup so ACL-granted editors
  can update prompt groups (aligns with deletePromptGroupController)
- Add missing group guard to mobile HeaderActions in PromptForm
- Replace div with article in DashGroupItem, remove redundant
  stopPropagation and onClick on outer container
- Add useFocusTrap hook for keyboard focus management
- Add numberOfGenerations to default projection
- Deduplicate ObjectId validation, remove console.warn,
  fix aria-labelledby, localize search announcements

* refactor(Prompts): adjust UI and improve a11y

* refactor(Prompts): reorder imports for consistency and clarity

* refactor(Prompts): implement updateFieldsInPlace for efficient data updates and add related tests

* refactor(Prompts): reorder imports to include updateFieldsInPlace for better organization

* refactor(Prompts): enhance DashGroupItem with toast notifications for prompt updates and add click-to-edit functionality in PromptEditor

* style: use self-closing TooltipAnchor in CreatePromptButton

Replace ></TooltipAnchor> with /> for consistency with the rest of the Prompts directory.

* fix(i18n): replace placeholder text for com_ui_global_group translation key

The value was left as 'something needs to go here. was empty' which
would be visible to users as an aria-label in DashGroupItem.

* fix(DashGroupItem): sync rename input with group.name on external changes

nameInputValue was initialized via useState(group.name) but never
synced when group.name changed from a background refetch. Added
useEffect that updates the input when the dialog is closed.

* perf(useFocusTrap): store onEscape in ref to avoid listener churn

onEscape was in the useEffect dependency array, causing the keydown
listener to be torn down and re-attached on every render when callers
passed an inline function. Now stored in a ref so the effect only
re-runs when active or containerRef changes.

* fix(a11y): replace role=button div with layered button overlay in ListCard

The card used role='button' on a div that contained nested Button
elements — an invalid ARIA pattern. Replaced with a hidden button
at z-0 for the card action while child interactive elements sit
at z-10, eliminating nested interactive element violations.

* fix(PromptForm): reset selectionIndex on route change, guard auto-save, and fix a11y

- Reset selectionIndex to 0 and isEditing to false when promptId
  changes, preventing out-of-bounds index when navigating between
  groups with different version counts.
- Track selectedPrompt in a ref so the auto-save effect doesn't
  fire against a stale prompt when the selection changed mid-edit.
- Stabilize useFocusTrap onEscape via useCallback to avoid
  unnecessary listener re-attachment.
- Conditionally render mobile overlay instead of always-present
  button with aria-hidden/pointer-events toggling.

* refactor: extract isValidObjectIdString to shared utility in data-schemas

The same regex helper was duplicated in api/server/routes/prompts.js
and packages/data-schemas/src/methods/prompt.ts. Moved to
packages/data-schemas/src/utils/objectId.ts and imported from both
consumers. Also removed a duplicate router.use block introduced
during the extraction.

* perf(updateFieldsInPlace): replace JSON deep clone with targeted spread

Instead of JSON.parse(JSON.stringify(data)) which serializes the
entire paginated data structure, use targeted immutable spreads
that only copy the affected page and collection array. Returns the
original data reference unchanged when the item is not found.

* perf(VariablesDropdown): memoize items array and stabilize handleAddVariable

The items array containing JSX elements was rebuilt on every render.
Wrapped in useMemo keyed on usedVariables and localize. Also wrapped
handleAddVariable in useCallback and memoized usedCount to avoid
redundant array filtering.

* perf(DashGroupItem): stabilize mutation callbacks via refs

handleSaveRename and handleDelete had updateGroup/deleteGroup mutation
objects in their useCallback dependency arrays. Since mutation objects
are new references each render, the callbacks were recreated every
render, defeating memoization. Now store mutation objects in refs and
call via ref.current in the callbacks.

* fix(security): validate groupId in incrementPromptGroupUsage

The data-schema method passed the groupId string directly to
findByIdAndUpdate without validation. If called from a different
entrypoint without the route-level check, Mongoose would throw a
CastError. Now validates with isValidObjectIdString before the
DB call and throws a clean 'Invalid groupId' error.

* fix(security): add rate limiter to prompt usage tracking endpoint

POST /groups/:groupId/use had no rate limiting — a user could spam
it to inflate numberOfGenerations, which controls sort order for all
users. Added promptUsageLimiter (30 req/user/min) following the same
pattern as toolCallLimiter. Also handle 'Invalid groupId' error from
the data layer in the route error handler.

* fix(updateFieldsInPlace): guard against undefined identifier value

If updatedItem[identifierField] is null/undefined, findIndex could
match unintended items where that field is also undefined. Added
early return when the identifier value is nullish.

* fix(a11y): use React useId for stable unique IDs in ListCard

aria-describedby/id values were derived from prompt name which can
contain spaces and special characters, producing invalid HTML IDs
and potential collisions. Now uses React.useId() for guaranteed
unique, valid IDs per component instance.

* fix: Align prompts panel styling with other sidebar panels and fix test

- Match FilterPrompts first row to Memory/Bookmark pattern (items-center gap-2)
- Remove items-stretch override from PromptsAccordion
- Add missing promptUsageLimiter mock to prompts route test

* fix: Address code review findings for prompts refactor PR

- Fix #5: Gate DeletePrompt in HeaderActions behind canDelete permission
- Fix #8: BackToChat navigates to last conversation instead of /c/new
- Fix #7: Restore useLiveAnnouncer for screen reader feedback on delete/rename
- Fix #1: Use isPublic (set by API) instead of deprecated projectIds for globe icon
- Fix #4: Optimistic cache update in useRecordPromptUsage instead of full invalidation
- Fix #6: Add migration to drop superseded { createdAt, updatedAt } compound index
- Fix #9: Single-pass reduce in PromptVariables instead of triple filter
- Fix #10: Rename PromptLabelsForm internal component to avoid collision with PromptForm
- Fix #14: Remove redundant aria-label from aria-hidden Checkbox in AutoSendPrompt

* fix: Align prompts panel filter row element sizes with other panels

- Override Dropdown trigger to size-9 (36px) to match FilterInput height
- Set CreatePromptButton to size-9 shrink-0 bg-transparent matching
  Memory/Bookmark panel button pattern

* fix(prompts): Shared Prompts filter ignores direct shares, only returns PUBLIC

Folds fix from PR #11882 into the refactored codebase.

Bug A: filterAccessibleIdsBySharedLogic now accepts ownedPromptGroupIds:
- MY_PROMPTS: accessible intersect owned
- SHARED_PROMPTS: (accessible union public) minus owned
- ALL: accessible union public (deduplicated)
Legacy fallback preserved when ownedPromptGroupIds is omitted.

Bug B: getPromptGroup uses $lookup aggregation to populate productionPrompt,
fixing empty text on direct URL navigation to shared prompts.

Also adds getOwnedPromptGroupIds to data-schemas methods and passes it
from both /all and /groups route handlers.

* fix: Add missing canDelete to mobile HeaderActions, remove dead instanceProjectId prop

- Pass canDelete to mobile HeaderActions row (was only on desktop)
- Remove instanceProjectId prop from ChatGroupItem and DashGroupItem
  since global check now uses group.isPublic
- Remove useGetStartupConfig from List.tsx (no longer needed)

* fix: Use runtime ObjectId instead of type-only Types.ObjectId, fix i18next interpolation

- getPromptGroup and getOwnedPromptGroupIds were using Types.ObjectId
  (imported as type-only), which is erased at compile time. Use the
  runtime ObjectId from mongoose.Types (already destructured at line 20).
  This fixes the 404s in PATCH /groups/:groupId tests.
- Fix com_ui_prompt_deleted_group translation to use {{0}} (i18next
  double-brace syntax) instead of {0}.

* chore: Fix translation key ordering, add sideEffects: false to data-provider

- Reorder new translation keys to maintain alphabetical order:
  com_ui_click_to_edit, com_ui_labels, com_ui_live, com_ui_prompt_delete_confirm,
  com_ui_prompt_deleted_group, com_ui_prompt_details, com_ui_prompt_renamed,
  com_ui_prompt_update_error, com_ui_prompt_variables_list
- Add "sideEffects": false to librechat-data-provider package.json to
  enable tree-shaking of unused exports (types, constants, pure functions)

* fix: Reduce prompts panel spacing, align memory toggle with checkbox pattern

- Remove unnecessary wrapper div around AutoSendPrompt in PromptsAccordion,
  reducing vertical space between the toggle and the first prompt item
- Replace Memory panel's Switch toggle with Checkbox+Button pattern
  matching the prompts panel's AutoSendPrompt for visual consistency

* fix: Reduce gap between AutoSendPrompt and first prompt item

Change ChatGroupItem margin from my-2 to mb-2 to eliminate the
doubled spacing (gap-2 from parent + top margin from first item).
Restore wrapper div around AutoSendPrompt for right-alignment.

* fix: Restore prompt name on empty save, remove dead bodyProps from checkGlobalPromptShare

- PromptName: reset newName to name when save is cancelled due to empty
  or unchanged input, preventing blank title in read mode
- checkGlobalPromptShare: remove dead bodyProps config — Permissions.SHARE
  was not in the permissions array so the bodyProps rule was never evaluated.
  Per-resource share checks are handled by canAccessPromptGroupResource.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-22 16:56:22 -04:00
Danny Avila
04e65bb21a
🧹 chore: Move direct model usage from PermissionsController to data-schemas 2026-03-21 15:20:15 -04:00
Danny Avila
a78865b5e0
🔄 refactor: Update Token Deletion Logic to Use AND Semantics
- Refactored the `deleteTokens` method to delete tokens based on all provided fields using AND conditions instead of OR, enhancing precision in token management.
- Updated related tests to reflect the new logic, ensuring that only tokens matching all specified criteria are deleted.
- Added new test cases for deleting tokens by type and userId, and for preventing cross-user token deletions, improving overall test coverage and robustness.
- Introduced a new `type` field in the `TokenQuery` interface to support the updated deletion functionality.
2026-03-21 14:45:53 -04:00
Danny Avila
150361273f
🧽 chore: Resolve TypeScript errors and test failures in agent/prompt deletion methods
Type AclEntry model as Model<IAclEntry> instead of Model<unknown> in
    deleteUserAgents and deleteUserPrompts, wire getSoleOwnedResourceIds
    into agent.spec.ts via createAclEntryMethods, replace permissionService
    calls with direct AclEntry.create, and add missing principalModel field.
2026-03-21 14:30:46 -04:00
Danny Avila
b5c097e5c7
⚗️ feat: Agent Context Compaction/Summarization (#12287)
* chore: imports/types

Add summarization config and package-level summarize handler contracts

Register summarize handlers across server controller paths

Port cursor dual-read/dual-write summary support and UI status handling

Selectively merge cursor branch files for BaseClient summary content
block detection (last-summary-wins), dual-write persistence, summary
block unit tests, and on_summarize_status SSE event handling with
started/completed/failed branches.

Co-authored-by: Cursor <cursoragent@cursor.com>

refactor: type safety

feat: add localization for summarization status messages

refactor: optimize summary block detection in BaseClient

Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist.

chore: add runName to chainOptions in AgentClient

refactor: streamline summarization configuration and handler integration

Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic.

feat(summarization): add staged chunk-and-merge fallback

feat(usage): track summarization usage separately from messages

feat(summarization): resolve prompt from config in runtime

fix(endpoints): use @librechat/api provider config loader

refactor(agents): import getProviderConfig from @librechat/api

chore: code order

feat(app-config): auto-enable summarization when configured

feat: summarization config

refactor(summarization): streamline persist summary handling and enhance configuration validation

Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness.

refactor(summarization): update event handling and remove legacy summarize handlers

Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process.

refactor(summarization): standardize event names in handlers

Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling.

feat(summarization): enhance usage tracking for summarization events

Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes.

feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type

Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure.

test: add end-to-end tests for summarization functionality

Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability.

refactor(summarization): include initial summary in formatAgentMessages output

Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response.

refactor: move hydrateMissingIndexTokenCounts to tokenMap utility

Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application.

refactor(summarization): standardize step event handling and improve summary rendering

Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process.

feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management

Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality.

feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations

Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process.

feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration

Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features.

feat(logging): add on_agent_log event handler for structured logging

Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events.

fix: remove duplicate IBalanceUpdate interface declaration

perf(usage): single-pass partition of collectedUsage

Replace two Array.filter() passes with a single for-of loop that
partitions message vs. summarization usages in one iteration.

fix(BaseClient): shallow-copy message content before mutating and preserve string content

Avoid mutating the original message.content array in-place when
appending a summary block. Also convert string content to a text
content part instead of silently discarding it.

fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling

- Fix SUMMARY else-if branch indentation in Part.tsx to match chain level
- Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary
  re-renders when no summarizing parts exist
- Protect against undefined completeData.summary instead of unsafe spread

fix(agents): use strict enabled check for summarization handlers

Change summarizationConfig?.enabled !== false to === true so handlers
are not registered when summarizationConfig is undefined.

chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope

refactor(Summary): align collapse/expand behavior with Reasoning component

- Single render path instead of separate streaming vs completed branches
- Use useMessageContext for isSubmitting/isLatestMessage awareness so
  the "Summarizing..." label only shows during active streaming
- Default to collapsed (matching Reasoning), user toggles to expand
- Add proper aria attributes (aria-hidden, role, aria-controls, contentId)
- Hide copy button while actively streaming

feat(summarization): default to self-summarize using agent's own provider/model

When no summarization config is provided (neither in librechat.yaml nor
on the agent), automatically enable summarization using the agent's own
provider and model. The agents package already provides default prompts,
so no prompt configuration is needed.

Also removes the dead resolveSummarizationLLMConfig in summarize.ts
(and its spec) — run.ts buildAgentContext is the single source of truth
for summarization config resolution. Removes the duplicate
RuntimeSummarizationConfig local type in favor of the canonical
SummarizationConfig from data-provider.

chore: schema and type cleanup for summarization

- Add trigger field to summarizationAgentOverrideSchema so per-agent
  trigger overrides in librechat.yaml are not silently stripped by Zod
- Remove unused SummarizationStatus type from runs.ts
- Make AppSummarizationConfig.enabled non-optional to reflect the
  invariant that loadSummarizationConfig always sets it

refactor(responses): extract duplicated on_agent_log handler

refactor(run): use agents package types for summarization config

Import SummarizationConfig, ContextPruningConfig, and
OverflowRecoveryConfig from @librechat/agents and use them to
type-check the translation layer in buildAgentContext. This ensures
the config object passed to the agent graph matches what it expects.

- Use `satisfies AgentSummarizationConfig` on the config object
- Cast contextPruningConfig and overflowRecoveryConfig to agents types
- Properly narrow trigger fields from DeepPartial to required shape

feat(config): add maxToolResultChars to base endpoint schema

Add maxToolResultChars to baseEndpointSchema so it can be configured
on any endpoint in librechat.yaml. Resolved during agent initialization
using getProviderConfig's endpoint resolution: custom endpoint config
takes precedence, then the provider-specific endpoint config, then the
shared `all` config.

Passed through to the agents package ToolNode, which uses it to cap
tool result length before it enters the context window. When not
configured, the agents package computes a sensible default from
maxContextTokens.

fix(summarization): forward agent model_parameters in self-summarize default

When no explicit summarization config exists, the self-summarize
default now forwards the agent's model_parameters as the
summarization parameters. This ensures provider-specific settings
(e.g. Bedrock region, credentials, endpoint host) are available
when the agents package constructs the summarization LLM.

fix(agents): register summarization handlers by default

Change the enabled gate from === true to !== false so handlers
register when no explicit summarization config exists. This aligns
with the self-summarize default where summarization is always on
unless explicitly disabled via enabled: false.

refactor(summarization): let agents package inherit clientOptions for self-summarize

Remove model_parameters forwarding from the self-summarize default.
The agents package now reuses the agent's own clientOptions when the
summarization provider matches the agent's provider, inheriting all
provider-specific settings (region, credentials, proxy, etc.)
automatically.

refactor(summarization): use MessageContentComplex[] for summary content

Unify summary content to always use MessageContentComplex[] arrays,
matching the pattern used by on_message_delta. No more string | array
unions — content is always an array of typed blocks ({ type: 'text',
text: '...' } for text, { type: 'reasoning_content', ... } for
reasoning).

Agents package:
- SummaryContentBlock.content: MessageContentComplex[] (was string)
- tokenCount now optional (not sent on deltas)
- Removed reasoning field — reasoning is now a content block type
- streamAndCollect normalizes all chunks to content block arrays
- Delta events pass content blocks directly

LibreChat:
- SummaryContentPart.content: Agents.MessageContentComplex[]
- Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js
- Summary.tsx derives display text from content blocks via useMemo
- Aggregator uses simple array spread

refactor(summarization): enhance summary handling and text extraction

- Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats.
- Modified summarization logic to ensure consistent handling of summary content across different message formats.
- Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method.
- Refined SSE useStepHandler to initialize summary content as an array.
- Updated configuration schema by removing unused minReserveTokens field.
- Cleaned up SummaryContentPart type by removing rangeHash property.

These changes streamline the summarization process and ensure compatibility with various content structures.

refactor(summarization): streamline usage tracking and logging

- Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability.
- Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types.
- Enhanced logging functionality by ensuring the logger correctly handles different log levels.
- Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity.

These changes improve the overall structure and efficiency of the summarization handling and logging processes.

refactor(summarization): update summary content block documentation

- Removed outdated comment regarding the last summary content block in BaseClient.js.
- Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation.

These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic.

refactor(summarization): update summary content structure in tests

- Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling.
- Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests.

These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure.

refactor(summarization): remove legacy E2E test setup and configuration

- Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys.
- Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run.

These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations.

refactor(summarization): enhance E2E tests and error handling

- Added a cleanup step to force exit after all tests to manage Redis connections.
- Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests.
- Improved error handling in the processStream function to capture and return processing errors.
- Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution.

These changes improve the reliability and clarity of the E2E tests for the summarization process.

refactor(summarization): enhance test coverage for maxContextTokens behavior

- Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise.
- Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup.

These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions.

feat(summarization): add comprehensive E2E tests for summarization process

- Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents.
- The tests utilize real AI providers and include functionality to track summaries during and between runs.
- Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit.

These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions.

fix(service): import logger from winston configuration

- Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'.
- This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application.

refactor(summary): simplify Summary component and enhance token display

- Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface.
- Updated the token display logic to use a localized string for better internationalization support.
- Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component.

These changes enhance the clarity and usability of the Summary component while ensuring better localization practices.

feat(summarization): add maxInputTokens configuration for summarization

- Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000.
- Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context.

These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process.

refactor(summarization): simplify maxInputTokens logic in createRun function

- Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value.
- This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization.

These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic.

feat(summary): enhance Summary component to display meta information

- Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content.
- Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details.

These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative.

refactor(summarization): standardize reserveRatio configuration in summarization logic

- Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase.
- Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized.
- Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity.

These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters.

* fix: circular dependency of `~/models`

* chore: update logging scope in agent log handlers

Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application.

* feat: calibration ratio

* refactor(tests): update summarizationConfig tests to reflect changes in enabled property

Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents.

* feat(tests): add markSummarizationUsage mock for improved test coverage

Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling.

* refactor(tests): simplify event handler setup in createResponse tests

Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events.

* refactor(agents): move calibration ratio capture to finally block

Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata.

* refactor(agents): streamline bulk write logic in recordCollectedUsage function

Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures.

* refactor(agents): enhance summarization configuration resolution in createRun function

Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings.

* refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling

Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase.

* refactor(agents): extract agent log handler for improved clarity and reusability

Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application.

* test: add summarization event tests for useStepHandler

Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application.

* refactor(config): update summarizationTriggerSchema to use enum for type validation

Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema.

* test(usage): add bulk write tests for message and summarization usage

Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic.

* refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component

Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability.

* chore(translations): remove unused "Expand Summary" key from English translations

Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure.

* refactor: adjust token counting for Claude model to account for API discrepancies

Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities.

* refactor(agents): implement token count adjustment for Claude model messages

Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model.

* refactor(agents): token counting for media content in messages

Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality.

* chore: fix missing import

* fix(agents): clamp baseContextTokens and document reserve ratio change

Prevent negative baseContextTokens when maxOutputTokens exceeds the
context window (misconfigured models). Document the 10%→5% default
reserve ratio reduction introduced alongside summarization.

* fix(agents): include media tokens in hydrated token counts

Add estimateMediaTokensForMessage to createTokenCounter so the hydration
path (used by hydrateMissingIndexTokenCounts) matches the precomputed
path in AgentClient.getTokenCountForMessage. Without this, messages
containing images or documents were systematically undercounted during
hydration, risking context window overflow.

Add 34 unit tests covering all block-type branches of
estimateMediaTokensForMessage.

* fix(agents): include summarization output tokens in usage return value

The returned output_tokens from recordCollectedUsage now reflects all
billed LLM calls (message + summarization). Previously, summarization
completions were billed but excluded from the returned metadata, causing
a discrepancy between what users were charged and what the response
message reported.

* fix(tests): replace process.exit with proper Redis cleanup in e2e test

The summarization E2E test used process.exit(0) to work around a Redis
connection opened at import time, which killed the Jest runner and
bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect()
for graceful cleanup instead.

* fix(tests): update getConvo imports in OpenAI and response tests

Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability.

* fix(clients): improve summary text validation in BaseClient

Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability.

* fix(config): replace z.any() with explicit union in summarization schema

Model parameters (temperature, top_p, etc.) are constrained to
primitive types rather than the policy-violating z.any().

* refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant

Export from the TS source in packages/api and import in the JS client,
eliminating the static class property that could drift out of sync.

* refactor(agents): eliminate duplicate selfProvider in buildAgentContext

selfProvider and provider were derived from the same expression with
different type casts. Consolidated to a single provider variable.

* refactor(agents): extract shared SSE handlers and restrict log levels

- buildSummarizationHandlers() factory replaces triplicated handler
  blocks across responses.js and openai.js
- agentLogHandlerObj exported from callbacks.js for consistent reuse
- agentLogHandler restricted to an allowlist of safe log levels
  (debug, info, warn, error) instead of accepting arbitrary strings

* fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement

- ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame
  instead of calling setMessages per chunk
- Exhaustive never-check on TStepEvent catches unhandled variants at
  compile time when new StepEvents are added
- ON_SUMMARIZE_COMPLETE error announcement only fires when a summary
  part was actually present and removed

* feat(agents): persist instruction overhead in contextMeta and seed across runs

Extend contextMeta with instructionOverhead and toolCount so the
provider-observed instruction overhead is persisted on the response message
and seeded into the pruner on subsequent runs. This enables the pruner to
use a calibrated budget from the first call instead of waiting for a
provider observation, preventing the ratio collapse caused by local
tokenizer overestimating tool schema tokens.

The seeded overhead is only used when encoding and tool count match
between runs, ensuring stale values from different configurations
are discarded.

* test(agents): enhance OpenAI test mocks for summarization handlers

Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing.

* fix(agents): address review findings for summarization v2

Cancel rAF on unmount to prevent stale Recoil writes from dead
component context. Clear orphaned summarizing:true parts when
ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null
guard and safe spread to agentLogHandler. Handle Anthropic-format
base64 image/* documents in estimateMediaTokensForMessage. Use
role="region" for expandable summary content. Add .describe() to
contextMeta Zod fields. Extract duplicate usage loop into helper.

* refactor: simplify contextMeta to calibrationRatio + encoding only

Remove instructionOverhead and toolCount from cross-run persistence —
instruction tokens change too frequently between runs (prompt edits,
tool changes) for a persisted seed to be reliable. The intra-run
calibration in the pruner still self-corrects via provider observations.
contextMeta now stores only the tokenizer-bias ratio and encoding,
which are stable across instruction changes.

* test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior

Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts.

* refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions

Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability.

* refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient

Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing.

* chore: update dependencies in package-lock.json and package.json files

Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes.

* chore: imports order

* refactor: extract summarization config resolution from buildAgentContext

* refactor: rename and simplify summarization configuration shaping function

* refactor: replace AgentClient token counting methods with single-pass pure utility

Extract getTokenCount() and getTokenCountForMessage() from AgentClient
into countFormattedMessageTokens(), a pure function in packages/api that
handles text, tool_call, image, and document content types in one loop.

- Decompose estimateMediaTokensForMessage into block-level helpers
  (estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens)
  shared by both estimateMediaTokensForMessage and the new single-pass function
- Remove redundant per-call getEncoding() resolution (closure captures once)
- Remove deprecated gpt-3.5-turbo-0301 model branching
- Drop this.getTokenCount guard from BaseClient.sendMessage

* refactor: streamline token counting in createTokenCounter function

Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction.

* refactor: simplify summarization configuration types

Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase.

* chore: import order

* fix: summarization event handling in useStepHandler

- Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale
  frames firing after map reset or component unmount
- Move announcePolite('summarize_completed') inside the didFinalize
  guard so screen readers only announce when finalization actually occurs
- Remove dead cleanup closure returned from stepHandler useCallback body
  that was never invoked by any caller

* fix: estimate tokens for non-PDF/non-image base64 document blocks

Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME
types (e.g. text/plain, application/json), silently underestimating
context budget. Fall back to character-based heuristic or countTokens.

* refactor: return cloned usage from markSummarizationUsage

Avoid mutating LangChain's internal usage_metadata object by returning
a shallow clone with the usage_type tag. Update all call sites in
callbacks, openai, and responses controllers to use the returned value.

* refactor: consolidate debug logging loops in buildMessages

Merge the two sequential O(n) debug-logging passes over orderedMessages
into a single pass inside the map callback where all data is available.

* refactor: narrow SummaryContentPart.content type

Replace broad Agents.MessageContentComplex[] with the specific
Array<{ type: ContentTypes.TEXT; text: string }> that all producers
and consumers already use, improving compile-time safety.

* refactor: use single output array in recordCollectedUsage

Have processUsageGroup append to a shared array instead of returning
separate arrays that are spread into a third, reducing allocations.

* refactor: use for...in in hydrateMissingIndexTokenCounts

Replace Object.entries with for...in to avoid allocating an
intermediate tuple array during token map hydration.
2026-03-21 14:28:56 -04:00
Danny Avila
e4e468840e
🏢 feat: Multi-Tenant Data Isolation Infrastructure (#12091)
* chore: imports

* chore: optional chaining in `spendTokens.spec.ts`

* feat: Add tenantId field to all MongoDB schemas for multi-tenant isolation

  - Add AsyncLocalStorage-based tenant context (`tenantContext.ts`) for
    request-scoped tenantId propagation without modifying method signatures
  - Add Mongoose `applyTenantIsolation` plugin that injects `{ tenantId }`
    into all query filters when tenant context is present, with
    `TENANT_ISOLATION_STRICT` env var for fail-closed production mode
  - Add optional `tenantId` field to all 28 collection schemas
  - Update all compound unique indexes to include tenantId (email, OAuth IDs,
    role names, serverName, conversationId+user, messageId+user, etc.)
  - Apply tenant isolation plugin in all 28 model factories
  - Add `tenantId?: string` to all TypeScript document interfaces

  Behaviorally inert — transitional mode (default) passes through all queries
  unchanged. No migration required for existing deployments.

* refactor: Update tenant context and enhance tenant isolation plugin

- Changed `tenantId` in `TenantContext` to be optional, allowing for more flexible usage.
- Refactored `runAsSystem` function to accept synchronous functions, improving usability.
- Introduced comprehensive tests for the `applyTenantIsolation` plugin, ensuring correct tenant filtering in various query scenarios.
- Enhanced the plugin to handle aggregate queries and save operations with tenant context, improving data isolation capabilities.

* docs: tenant context documentation and improve tenant isolation tests

- Added detailed documentation for the `tenantStorage` AsyncLocalStorage instance in `tenantContext.ts`, clarifying its usage for async tenant context propagation.
- Updated tests in `tenantIsolation.spec.ts` to improve clarity and coverage, including new tests for strict mode behavior and tenant context propagation through await boundaries.
- Refactored existing test cases for better readability and consistency, ensuring robust validation of tenant isolation functionality.

* feat: Enhance tenant isolation by preventing tenantId mutations in update operations

- Added a new function to assert that tenantId cannot be modified through update operators in Mongoose queries.
- Implemented middleware to enforce this restriction during findOneAndUpdate, updateOne, and updateMany operations.
- Updated documentation to reflect the new behavior regarding tenantId modifications, ensuring clarity on tenant isolation rules.

* feat: Enhance tenant isolation tests and enforce tenantId restrictions

- Updated existing tests to clarify behavior regarding tenantId preservation during save and insertMany operations.
- Introduced new tests to validate that tenantId cannot be modified through update operations, ensuring strict adherence to tenant isolation rules.
- Added checks for mismatched tenantId scenarios, reinforcing the integrity of tenant context propagation.
- Enhanced test coverage for async context propagation and mutation guards, improving overall robustness of tenant isolation functionality.

* fix: Remove duplicate re-exports in utils/index.ts

Merge artifact caused `string` and `tempChatRetention` to be exported
twice, which produces TypeScript compile errors for duplicate bindings.

* fix: Resolve admin capability gap in multi-tenant mode (TODO #12091)

- hasCapabilityForPrincipals now queries both tenant-scoped AND
  platform-level grants when tenantId is set, so seeded ADMIN grants
  remain effective in tenant mode.
- Add applyTenantIsolation to SystemGrant model factory.

* fix: Harden tenant isolation plugin

- Add replaceGuard for replaceOne/findOneAndReplace to prevent
  cross-tenant document reassignment via replacement documents.
- Cache isStrict() result to avoid process.env reads on every query.
  Export _resetStrictCache() for test teardown.
- Replace console.warn with project logger (winston).
- Add 5 new tests for replace guard behavior (46 total).

* style: Fix import ordering in convo.ts and message.ts

Move type imports after value imports per project style guide.

* fix: Remove tenant isolation from SystemGrant, stamp tenantId in replaceGuard

- SystemGrant is a cross-tenant control plane whose methods handle
  tenantId conditions explicitly. Applying the isolation plugin
  injects a hard equality filter that overrides the $and/$or logic
  in hasCapabilityForPrincipals, making platform-level ADMIN grants
  invisible in tenant mode.
- replaceGuard now stamps tenantId into replacement documents when
  absent, preventing replaceOne from silently stripping tenant
  context. Replacements with a matching tenantId are allowed;
  mismatched tenantId still throws.

* test: Add multi-tenant unique constraint and replace stamping tests

- Verify same name/email can exist in different tenants (compound
  unique index allows it).
- Verify duplicate within same tenant is rejected (E11000).
- Verify tenant-scoped query returns only the correct document.
- Update replaceOne test to assert tenantId is stamped into
  replacement document.
- Add test for replacement with matching tenantId.

* style: Reorder imports in message.ts to align with project style guide

* feat: Add migration to drop superseded unique indexes for multi-tenancy

Existing deployments have single-field unique indexes (e.g. { email: 1 })
that block multi-tenant operation — same email in different tenants
triggers E11000. Mongoose autoIndex creates the new compound indexes
but never drops the old ones.

dropSupersededTenantIndexes() drops all 19 superseded indexes across 11
collections. It is idempotent, skips missing indexes/collections, and
is a no-op on fresh databases.

Must be called before enabling multi-tenant middleware on an existing
deployment. Single-tenant deployments are unaffected (old indexes
coexist harmlessly until migration runs).

Includes 11 tests covering:
- Full upgrade simulation (create old indexes, drop them, verify gone)
- Multi-tenant writes work after migration (same email, different tenant)
- Intra-tenant uniqueness preserved (duplicate within tenant rejected)
- Fresh database (no-op, no errors)
- Partial migration (some collections exist, some don't)
- SUPERSEDED_INDEXES coverage validation

* fix: Update systemGrant test — platform grants now satisfy tenant queries

The TODO #12091 fix intentionally changed hasCapabilityForPrincipals to
match both tenant-scoped AND platform-level grants. The test expected
the old behavior (platform grant invisible to tenant query). Updated
test name and expectation to match the new semantics.

* fix: Align getCapabilitiesForPrincipal with hasCapabilityForPrincipals tenant query

getCapabilitiesForPrincipal used a hard tenantId equality filter while
hasCapabilityForPrincipals uses $and/$or to match both tenant-scoped
and platform-level grants. This caused the two functions to disagree
on what grants a principal holds in tenant mode.

Apply the same $or pattern: when tenantId is provided, match both
{ tenantId } and { tenantId: { $exists: false } }.

Adds test verifying platform-level ADMIN grants appear in
getCapabilitiesForPrincipal when called with a tenantId.

* fix: Remove categories from tenant index migration

categoriesSchema is exported but never used to create a Mongoose model.
No Category model factory exists, no code constructs a model from it,
and no categories collection exists in production databases. Including
it in the migration would attempt to drop indexes from a non-existent
collection (harmlessly skipped) but implies the collection is managed.

* fix: Restrict runAsSystem to async callbacks only

Sync callbacks returning Mongoose thenables silently lose ALS context —
the system bypass does nothing and strict mode throws with no indication
runAsSystem was involved. Narrowing to () => Promise<T> makes the wrong
pattern a compile error. All existing call sites already use async.

* fix: Use next(err) consistently in insertMany pre-hook

The hook accepted a next callback but used throw for errors. Standardize
on next(err) for all error paths so the hook speaks one language —
callback-style throughout.

* fix: Replace optional chaining with explicit null assertions in spendTokens tests

Optional chaining on test assertions masks failures with unintelligible
error messages. Add expect(result).not.toBeNull() before accessing
properties, so a null result produces a clear diagnosis instead of
"received value must be a number".
2026-03-21 14:28:54 -04:00
Danny Avila
9e0592a236
📜 feat: Implement System Grants for Capability-Based Authorization (#11896)
* feat: Implement System Grants for Role-Based Capabilities

- Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application.
- Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles.
- Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup.
- Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase.

* test: Add unit tests for capabilities middleware and system grant methods

- Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles.
- Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes.
- Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware.
- Utilized mocks for database interactions to isolate tests and improve reliability.

* refactor: Transition to Capability-Based Access Control

- Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management.
- Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions.
- Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement.
- Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability.

* test: Enhance capability-based access tests for ADMIN role

- Updated tests to reflect the new capability-based access control, specifically for the ADMIN role.
- Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks.
- Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware.
- Improved overall test coverage for capability verification, ensuring robust permission management.

* test: Update capability tests for MCP server access

- Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS.
- Updated seeding of capabilities for the ADMIN role to align with the new capability structure.
- Ensured consistency in capability definitions across tests and middleware for improved permission management.

* feat: Add hasConfigCapability for enhanced config access control

- Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections.
- Updated middleware to export the new capability function, ensuring consistent access control across the application.
- Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability.

* fix: Update tenantId filter in createSystemGrantMethods

- Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided.
- This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic.

* fix: account deletion capability check

- Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined.
- This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions.

* refactor: Optimize seeding of system grants for ADMIN role

- Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function.
- This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently.

* refactor: Simplify systemGrantSchema index definition

- Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema.
- This change streamlines the index definition, potentially improving query performance and clarity in the schema design.

* refactor: Reorganize role capability check in roles route

- Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure.
- This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management.

* refactor: Remove unused ISystemGrant interface from systemCapabilities.ts

- Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity.
- This change helps reduce clutter in the file and focuses on relevant capabilities for the system.

* refactor: Migrate SystemCapabilities to data-schemas

- Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files.
- This change centralizes the management of system capabilities, improving code organization and maintainability.

* refactor: Update account deletion middleware and capability checks

- Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management.
- Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues.
- Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification.
- Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity.

* refactor: Extract principal ID normalization into a separate function

- Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability.
- Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication.

* test: Add unit tests for principalId normalization in systemGrant

- Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats.
- Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety.
- Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application.

* feat: Introduce capability implications and enhance system grant methods

- Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks.
- Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic.
- Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null.
- Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations.

* test: Add unit tests for canDeleteAccount middleware

- Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities.
- Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases.
- Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware.

* fix: Add principalType enum validation to SystemGrant schema

Without enum validation, any string value was accepted for principalType
and silently stored. Invalid documents would never match capability
queries, creating phantom grants impossible to diagnose without raw DB
inspection. All other ACL models in the codebase validate this field.

* fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety

When two server instances start simultaneously (K8s rolling deploy, PM2
cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate
upsert, both instances may attempt to insert the same documents, causing
E11000 duplicate key errors that crash server startup.

bulkWrite with ordered:false handles concurrent upserts gracefully and
reduces 17 individual round trips to a single network call. The returned
documents (previously discarded) are no longer fetched.

* perf: Add AsyncLocalStorage per-request cache for capability checks

Every hasCapability call previously required 2 DB round trips
(getUserPrincipals + SystemGrant.exists) — replacing what were O(1)
string comparisons. Routes like patchPromptGroup triggered this twice,
and hasConfigCapability's fallback path resolved principals twice.

This adds a per-request AsyncLocalStorage cache that:
- Caches resolved principals (same for all checks within one request)
- Caches capability check results (same user+cap = same answer)
- Automatically scoped to request lifetime (no stale grants)
- Falls through to DB when no store exists (background jobs, tests)
- Requires no signature changes to hasCapability

The capabilityContextMiddleware is registered at the app level before
all routes, initializing a fresh store per request.

* fix: Add error handling for inline hasCapability calls

canDeleteAccount, fetchAssistants, and validateAuthor all call
hasCapability without try-catch. These were previously O(1) string
comparisons that could never throw. Now they hit the database and can
fail on connection timeout or transient errors.

Wrap each call in try-catch, defaulting to deny (false) on error.
This ensures a DB hiccup returns a clean 403 instead of an unhandled
500 with a stack trace.

* test: Add canDeleteAccount DB-error resilience test

Tests that hasCapability rejection (e.g., DB timeout) results in a clean
403 rather than an unhandled exception. Validates the error handling
added in the previous commit.

* refactor: Use barrel import for hasCapability in validateAuthor

Import from ~/server/middleware barrel instead of directly from
~/server/middleware/roles/capabilities for consistency with other
non-middleware consumers. Files within the middleware barrel itself
must continue using direct imports to avoid circular requires.

* refactor: Remove misleading pre('save') hook from SystemGrant schema

The pre('save') hook normalized principalId for USER/GROUP principals,
but the primary write path (grantCapability) uses findOneAndUpdate —
which does not trigger save hooks. The normalization was already handled
explicitly in grantCapability itself. The hook created a false impression
of schema-level enforcement that only covered save()/create() paths.

Replace with a comment documenting that all writes must go through
grantCapability.

* feat: Add READ_ASSISTANTS capability to complete manage/read pair

Every other managed resource had a paired READ_X / MANAGE_X capability
except assistants. This adds READ_ASSISTANTS and registers the
MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications,
enabling future read-only assistant visibility grants.

* chore: Reorder systemGrant methods for clarity

Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available.

* fix: Wrap seedSystemGrants in try-catch to avoid blocking startup

Seeding capabilities is idempotent and will succeed on the next restart.
A transient DB error during seeding should not prevent the server from
starting — log the error and continue.

* refactor: Improve capability check efficiency and add audit logging

Move hasCapability calls after cheap early-exits in validateAuthor and
fetchAssistants so the DB check only runs when its result matters. Add
logger.debug on every capability bypass grant across all 7 call sites
for auditability, and log errors in catch blocks instead of silently
swallowing them.

* test: Add integration tests for AsyncLocalStorage capability caching

Exercises the full vertical — ALS context, generateCapabilityCheck,
real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB
via MongoMemoryServer. Covers per-request caching, cross-context
isolation, concurrent request isolation, negative caching, capability
implications, tenant scoping, group-based grants, and requireCapability
middleware.

* test: Add systemGrant data-layer and ALS edge-case integration tests

systemGrant.spec.ts (51 tests): Full integration tests for all
systemGrant methods against real MongoDB — grant/revoke lifecycle,
principalId normalization (string→ObjectId for USER/GROUP, string for
ROLE), capability implications (both directions), tenant scoping,
schema validation (null tenantId, invalid enum, required fields,
unique compound index).

capabilities.integration.spec.ts (27 tests): Adds ALS edge cases —
missing context degrades gracefully with no caching (background jobs,
child processes), nested middleware creates independent inner context,
optional-chaining safety when store is undefined, mid-request grant
changes are invisible due to result caching, requireCapability works
without ALS, and interleaved concurrent contexts maintain isolation.

* fix: Add worker thread guards to capability ALS usage

Detect when hasCapability or capabilityContextMiddleware is called from
a worker thread (where ALS context does not propagate from the parent).
hasCapability logs a warn-once per factory instance; the middleware logs
an error since mounting Express middleware in a worker is likely a
misconfiguration. Both continue to function correctly — the guard is
observability, not a hard block.

* fix: Include tenantId in ALS principal cache key for tenant isolation

The principal cache key was user.id:user.role, which would reuse
cached principals across tenants for the same user within a request.
When getUserPrincipals gains tenant-scoped group resolution, principals
from tenant-a would incorrectly serve tenant-b checks. Changed to
user.id:user.role:user.tenantId to prevent cross-tenant cache hits.

Adds integration test proving separate principal lookups per tenantId.

* test: Remove redundant mocked capabilities.spec.js

The JS wrapper test (7 tests, all mocked) is a strict subset of
capabilities.integration.spec.ts (28 tests, real MongoDB). Every
scenario it covered — hasCapability true/false, tenantId passthrough,
requireCapability 403/500, error handling — is tested with higher
fidelity in the integration suite.

* test: Replace mocked canDeleteAccount tests with real MongoDB integration

Remove hasCapability mock — tests now exercise the full capability
chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals,
SystemGrant collection). Only mocks remaining are logger and cache.

Adds new coverage: admin role without grant is blocked, user-level
grant bypasses deletion restriction, null user handling.

* test: Add comprehensive tests for ACL entry management and user group methods

Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions.

* refactor: Update capability checks and logging for better clarity and error handling

Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors.

* refactor: Enhance logging and retry logic in seedSystemGrants method

Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management.

* refactor: Consolidate imports in canDeleteAccount middleware

Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware.

* test: Enhance systemGrant tests for error handling and capability validation

Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs.

* fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites

canAccessResource, files.js, and roles.js all had hasCapability inside
outer try-catch blocks that returned 500 on DB failure instead of
falling through to the regular ACL check. This contradicts the
deny-by-default pattern used everywhere else.

Also removes raw error.message from the roles.js 500 response to
prevent internal host/connection info leaking to clients.

* fix: Normalize user ID in canDeleteAccount before passing to hasCapability

requireCapability normalizes req.user.id via _id?.toString() fallback,
but canDeleteAccount passed raw req.user directly. If req.user.id is
absent (some auth layers only populate _id), getUserPrincipals received
undefined, silently returning empty principals and blocking the bypass.

* fix: Harden systemGrant schema and type safety

- Reject empty string tenantId in schema validator (was only blocking
  null; empty string silently orphaned documents)
- Fix reverseImplications to use BaseSystemCapability[] instead of
  string[], preserving the narrow discriminated type
- Document READ_ASSISTANTS as reserved/unenforced

* test: Use fake timers for seedSystemGrants retry tests and add tenantId validation

- Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds
  of real setTimeout delays per test run
- Add regression test for empty-string tenantId rejection

* docs: Add TODO(#12091) comments for tenant-scoped capability gaps

In multi-tenant mode, platform-level grants (no tenantId) won't match
tenant-scoped queries, breaking admin access. getUserPrincipals also
returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-21 14:28:54 -04:00
Danny Avila
0412f05daf
🪢 chore: Consolidate Pricing and Tx Imports After tx.js Module Removal (#12086)
* 🧹 chore: resolve imports due to rebase

* chore: Update model mocks in unit tests for consistency

- Consolidated model mock implementations across various test files to streamline setup and reduce redundancy.
- Removed duplicate mock definitions for `getMultiplier` and `getCacheMultiplier`, ensuring a unified approach in `recordCollectedUsage.spec.js`, `openai.spec.js`, `responses.unit.spec.js`, and `abortMiddleware.spec.js`.
- Enhanced clarity and maintainability of test files by aligning mock structures with the latest model updates.

* fix: Safeguard token credit checks in transaction tests

- Updated assertions in `transaction.spec.ts` to handle potential null values for `updatedBalance` by using optional chaining.
- Enhanced robustness of tests related to token credit calculations, ensuring they correctly account for scenarios where the balance may not be found.

* chore: transaction methods with bulk insert functionality

- Introduced `bulkInsertTransactions` method in `transaction.ts` to facilitate batch insertion of transaction documents.
- Updated test file `transactions.bulk-parity.spec.ts` to utilize new pricing function assignments and handle potential null values in calculations, improving test robustness.
- Refactored pricing function initialization for clarity and consistency.

* refactor: Enhance type definitions and introduce new utility functions for model matching

- Added `findMatchingPattern` and `matchModelName` utility functions to improve model name matching logic in transaction methods.
- Updated type definitions for `findMatchingPattern` to accept a more specific tokensMap structure, enhancing type safety.
- Refactored `dbMethods` initialization in `transactions.bulk-parity.spec.ts` to include the new utility functions, improving test clarity and functionality.

* refactor: Update database method imports and enhance transaction handling

- Refactored `abortMiddleware.js` to utilize centralized database methods for message handling and conversation retrieval, improving code consistency.
- Enhanced `bulkInsertTransactions` in `transaction.ts` to handle empty document arrays gracefully and added error logging for better debugging.
- Updated type definitions in `transactions.ts` to enforce stricter typing for token types, enhancing type safety across transaction methods.
- Improved test setup in `transactions.bulk-parity.spec.ts` by refining pricing function assignments and ensuring robust handling of potential null values.

* refactor: Update database method references and improve transaction multiplier handling

- Refactored `client.js` to update database method references for `bulkInsertTransactions` and `updateBalance`, ensuring consistency in method usage.
- Enhanced transaction multiplier calculations in `transaction.spec.ts` to provide fallback values for write and read multipliers, improving robustness in cost calculations across structured token spending tests.
2026-03-21 14:28:53 -04:00
Danny Avila
8ba2bde5c1
📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830)
* chore: move database model methods to /packages/data-schemas

* chore: add TypeScript ESLint rule to warn on unused variables

* refactor: model imports to streamline access

- Consolidated model imports across various files to improve code organization and reduce redundancy.
- Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path.
- Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact.
- Enhanced test files to align with the new import paths, maintaining test coverage and integrity.

* chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas

* test: update agent model mocks in unit tests

- Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality.
- Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication.
- Ensured consistency in agent mock implementations across test files.

* fix: update types in data-schemas

* refactor: enhance type definitions in transaction and spending methods

- Updated type definitions in `checkBalance.ts` to use specific request and response types.
- Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety.
- Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods.
- Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness.

* refactor: streamline model imports and enhance code organization

- Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy.
- Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact.
- Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods.

* feat: implement loadAddedAgent and refactor agent loading logic

- Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution.
- Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`.
- Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`.
- Enhanced type definitions and improved error handling in the agent loading process.
- Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage.

* refactor: enhance balance handling with new update interface

- Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase.
- Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety.
- Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities.
- Enhanced overall code clarity and maintainability by refining type definitions related to balance operations.

* feat: add unit tests for loadAgent functionality and enhance agent loading logic

- Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks.
- Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality.
- Improved test coverage for agent loading, including handling of non-existent agents and user permissions.

* chore: reorder memory method exports for consistency

- Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.
2026-03-21 14:28:53 -04:00
Danny Avila
58f128bee7
🗑️ chore: Remove Deprecated Project Model and Associated Fields (#11773)
* chore: remove projects and projectIds usage

* chore: empty line linting

* chore: remove isCollaborative property across agent models and related tests

- Removed the isCollaborative property from agent models, controllers, and tests, as it is deprecated in favor of ACL permissions.
- Updated related validation schemas and data provider types to reflect this change.
- Ensured all references to isCollaborative were stripped from the codebase to maintain consistency and clarity.
2026-03-21 14:28:53 -04:00
Danny Avila
38521381f4
🐘 feat: FerretDB Compatibility (#11769)
* feat: replace unsupported MongoDB aggregation operators for FerretDB compatibility

Replace $lookup, $unwind, $sample, $replaceRoot, and $addFields aggregation
stages which are unsupported on FerretDB v2.x (postgres-documentdb backend).

- Prompt.js: Replace $lookup/$unwind/$project pipelines with find().select().lean()
  + attachProductionPrompts() batch helper. Replace $group/$replaceRoot/$sample
  in getRandomPromptGroups with distinct() + Fisher-Yates shuffle.
- Agent/Prompt migration scripts: Replace $lookup anti-join pattern with
  distinct() + $nin two-step queries for finding un-migrated resources.

All replacement patterns verified against FerretDB v2.7.0.

* fix: use $pullAll for simple array removals, fix memberIds type mismatches

Replace $pull with $pullAll for exact-value scalar array removals. Both
operators work on MongoDB and FerretDB, but $pullAll is more explicit for
exact matching (no condition expressions).

Fix critical type mismatch bugs where ObjectId values were used against
String[] memberIds arrays in Group queries:
- config/delete-user.js: use string uid instead of ObjectId user._id
- e2e/setup/cleanupUser.ts: convert userId.toString() before query

Harden PermissionService.bulkUpdateResourcePermissions abort handling to
prevent crash when abortTransaction is called after commitTransaction.

All changes verified against FerretDB v2.7.0 and MongoDB Memory Server.

* fix: harden transaction support probe for FerretDB compatibility

Commit the transaction before aborting in supportsTransactions probe, and
wrap abortTransaction in try-catch to prevent crashes when abort is called
after a successful commit (observed behavior on FerretDB).

* feat: add FerretDB compatibility test suite, retry utilities, and CI config

Add comprehensive FerretDB integration test suite covering:
- $pullAll scalar array operations
- $pull with subdocument conditions
- $lookup replacement (find + manual join)
- $sample replacement (distinct + Fisher-Yates)
- $bit and $bitsAllSet operations
- Migration anti-join pattern
- Multi-tenancy (useDb, scaling, write amplification)
- Sharding proof-of-concept
- Production operations (backup/restore, schema migration, deadlock retry)

Add production retryWithBackoff utility for deadlock recovery during
concurrent index creation on FerretDB/DocumentDB backends.

Add UserController.spec.js tests for deleteUserController (runs in CI).

Configure jest and eslint to isolate FerretDB tests from CI pipelines:
- packages/data-schemas/jest.config.mjs: ignore misc/ directory
- eslint.config.mjs: ignore packages/data-schemas/misc/

Include Docker Compose config for local FerretDB v2.7 + postgres-documentdb,
dedicated jest/tsconfig for the test files, and multi-tenancy findings doc.

* style: brace formatting in aclEntry.ts modifyPermissionBits

* refactor: reorganize retry utilities and update imports

- Moved retryWithBackoff utility to a new file `retry.ts` for better structure.
- Updated imports in `orgOperations.ferretdb.spec.ts` to reflect the new location of retry utilities.
- Removed old import statement for retryWithBackoff from index.ts to streamline exports.

* test: add $pullAll coverage for ConversationTag and PermissionService

Add integration tests for deleteConversationTag verifying $pullAll
removes tags from conversations correctly, and for
syncUserEntraGroupMemberships verifying $pullAll removes user from
non-matching Entra groups while preserving local group membership.

---------
2026-03-21 14:28:49 -04:00
Danny Avila
0736ff2668
✨ v0.8.4 (#12339)
* 🔖 chore: Bump version to v0.8.4

- App version: v0.8.4-rc1 → v0.8.4
- @librechat/api: 1.7.26 → 1.7.27
- @librechat/client: 0.4.55 → 0.4.56
- librechat-data-provider: 0.8.400 → 0.8.401
- @librechat/data-schemas: 0.0.39 → 0.0.40

* chore: bun.lock file bumps
2026-03-20 18:01:00 -04:00