Commit graph

544 commits

Author SHA1 Message Date
Danny Avila
eb1475b4b3 fix: harden scheduled run lifecycle 2026-08-18 07:48:58 -04:00
Danny Avila
fa98a47a01 refactor: rebuild scheduled chats on durable agent triggers 2026-08-18 07:48:58 -04:00
Danny Avila
84a34f53b3 test: preserve scheduled init reconciliation option 2026-08-18 07:46:18 -04:00
Danny Avila
7e3f5e3e0d fix: generation-fence abort recovery evidence 2026-08-18 07:46:18 -04:00
Danny Avila
fad98f680a fix: finish scheduled chat lifecycle hardening 2026-08-18 07:46:18 -04:00
Danny Avila
62ba45a3de fix: one manager-owned owner-lease span across every abort delivery path
Seventh external review round; four lifecycle-fence gaps, closed by making the
owner-lifecycle lease a single manager-owned, heartbeat-held span.

- Fail-closed acknowledgements. The pre-ACK hook registered a one-shot lease and
  the transport ACKed even when it failed; a same-replica owned abort likewise
  swallowed registration failure. The hook now acquires a HELD owner lease
  (heartbeat until the owner's catch releases it via releaseOwnerLease) and a
  rejection SUPPRESSES the acknowledgement — the stopping side stays retryable
  behind its retention lease, and every resignal re-drives the handler. The hook
  also stopped gating on `job.createdAt === generationId`: during a replacement
  handoff the store holds the replacement while the abort targets the
  predecessor, and that gate silently skipped exactly the generation being
  acknowledged (the store job is owner identity, never a generation gate). A
  local owned abort acquires the same held lease before tripping its provider;
  post-CAS the trip cannot be withheld, so acquisition failure downgrades
  delivery and the retention handoff keeps the user fenced.

- Committed-but-lost-reply disambiguation. A thrown abort transition released
  the contender lease as if nothing had happened, but a Lua CAS can commit and
  lose its reply — an aborted job invisible to active-set scans whose provider
  was never signalled, with no fence left. The throw path now re-reads the exact
  generation: only a job still live under the caller's identity proves no
  commit; committed or ambiguous outcomes hand the fence to the deterministic
  stop lease (kept on the contender lease if even that fails) before rethrowing.

- Replacement handoff covered end to end. A LOCAL replacement abort acquires the
  predecessor's held owner lease before the trip (failure reports the receipt
  undelivered, engaging retention). Failed-handoff retention is no longer a
  swallowed one-shot: it heartbeats with the durable acknowledgement proof as
  its renewal predicate — acquisition failures keep retrying for as long as the
  fence is needed, and the retainer stands down (without clearing the shared
  field) once the remote owner ACKs and thereby holds its own lease.

- A LOCAL resignal delivery hands off to the owner lease BEFORE clearing the
  retained stop lease, and keeps the stop lease when that handoff fails.

Barrier tests: hook rejection suppressing the ACK, commit-then-lost-reply
retention with its provably-uncommitted counterpart, local-resignal owner
handoff ordering, pre-ACK owner lease held past the store TTL until release
(and provably stopped after), and failed-handoff retention retrying on its
heartbeat — verified fail-before/pass-after by stashing the fixes.
2026-08-18 07:46:18 -04:00
Danny Avila
291ed90765 fix: heartbeat-backed owner-lifecycle leases close the settlement handoff races
Sixth external review round: the remaining P1 interleavings were one structural
problem — lease handoffs that were not atomic — resolved as the requested
consolidated lifecycle-lease pass.

- Quiesce reads leases BEFORE the active-job scan. The admission-lease -> durable
  -job handoff is only atomic against a reader in the OPPOSITE order of the
  writer: writers hold the lease strictly until the job is active-set visible,
  so leases-first shows every interleaving either the lease or the job.
  Jobs-first allowed a request to create its job after the scan and release its
  lease before the count — hiding both, cascading, and letting the new
  generation persist into a deleted account.

- The abort acknowledgement is fenced by the owner-lifecycle lease. Redis ACKed
  the moment the owner's AbortController tripped; the stopping side released its
  lease on that ACK while the owner's asynchronous abort-catch persistence was
  still ahead. The transport now awaits a manager-installed pre-ACK hook that
  registers a DETERMINISTIC owner lease (exactly one owner exists per
  generation, and determinism is what lets the signal-time registrant and the
  owner's catch-side release agree across processes) before the acknowledgement
  is persisted or published; an owned same-replica abort bridges to the same
  lease before tripping its local controller. Both generation-owner catches
  (fresh turn, resume) release it once their writes land.

- A failed replacement handoff no longer orphans the predecessor. The atomic
  replacement removes it from active storage, and an unconfirmed handoff
  terminalizes the replacement too — leaving nothing a quiesce could discover
  while the predecessor's provider may still be generating. Its owner lease is
  now retained at the point the receipt fails delivery; the owner replica renews
  it through the pre-ACK fence when the signal finally lands.

- Leases HEARTBEAT while held. The five-minute store TTL only bounds a crashed
  holder; live persistence — a stalled save, a long deferred title — must never
  outlive its own fence. holdUserFinalization registers and renews every minute
  until released; the controllers' completion/error/admission leases all hold.
  The undelivered-Stop retention moved to a deterministic `stop` lease that
  every resignal attempt renews and the first successful one clears (no more
  opaque leases accumulating to TTL), and a THROWN abort transition releases the
  contender lease instead of leaking it.

- The user-document abort-fence mutations now invalidate the auth user-doc
  cache, matching every other user-doc write.

Barrier tests, each verified to fail without its fix: quiesce lease-scan
ordering (plus the observed-lease defer), pre-ACK fence ordering at the
transport, owned-abort owner-lease bridging, replacement-handoff predecessor
retention, held-lease heartbeat past the TTL, and failed-then-successful
resignal reaping the retained stop lease.
2026-08-18 07:46:18 -04:00
Danny Avila
b34bbd83ef fix: lease-token lifecycle fences — same-generation isolation, admission fence, undelivered-Stop retention, legacy abort enrollment
Fifth external review round; four P1s, handled as the requested consolidated
lifecycle-fence pass.

- Lease tokens. Marker fields were (streamId, createdAt), shared by every
  contender on the same generation — completion and Stop, or two racing Stops —
  so a losing contender's cleanup erased the winner's still-live marker. Every
  registrant now carries a unique lease token in the field and may only ever
  clear its own lease; unqualified legacy clears cannot touch qualified entries.

- Admission fence. Authentication can pass before the deletion barrier goes up,
  and the durable createJob is several async steps later — a deletion quiesce in
  that window saw neither an active job nor a marker and could cascade before
  the admitted request created its job. The controller now registers an
  admission lease and THEN rereads the deletion barrier: the ordering guarantees
  either this request observes the barrier (403, lease released, slot/claim
  cleanup) or the quiesce observes the lease and defers. Held until createJob is
  durable; released on every refusal and initialization-error path. Fail closed
  when the lease itself cannot be registered (503 retryable).

- Undelivered-Stop retention. abortJob released its lease in a finally even when
  delivery AND publication had provably failed — the job reads terminal
  (invisible to active-set scans), a user Stop writes no durable abort fence,
  and the remote owner keeps generating and will persist its abort-catch writes
  whenever the signal finally lands. The lease is now retained in exactly that
  case, and each resignal attempt heartbeats a fresh lease so the fence outlives
  the TTL for as long as delivery is still being driven. The abort-winning
  turn's own loser-side pending saves are additionally fenced in the controller
  catch (best-effort — those writes are already in flight).

- Legacy abort enrollment. abortMiddleware (assistants abort route fallback for
  non-assistants endpoints) awaited abortJob and then spent usage and saved the
  stopped response AFTER the abort's lease was released. Both writes now run
  inside `beforePublish`, between the abort CAS and publication, covered by the
  same lease as every other abort.

Barrier tests, each verified to fail without its fix: same-generation lease
isolation (store), racing two-Stop loser cleanup (manager, stale-read forced
CAS race), undelivered-Stop lease retention, resignal heartbeat, admission
refusal with lease-before-reread ordering plus release-on-durable-create, and
legacy-abort persistence inside beforePublish.
2026-08-18 07:46:18 -04:00
Danny Avila
2b10412c12 fix: deletion-fence marker protocol — generation-scoped, atomic, fail-closed, all terminal paths
One consolidated pass over the finalization-marker mechanism, per the fourth
external review round. The invariant it establishes: NO persistence-owning
terminal CAS runs without a durable, generation-qualified marker covering the
window it opens, and every consumer treats a pending terminal as unsettled.

- Generation-scoped markers. Entries were keyed (userId, streamId), so a
  Stop-superseded generation finishing late could clear the marker its
  replacement registered on the same conversation. Marker fields are now
  qualified by the generation's createdAt; clears must present the same
  identity, and an unqualified legacy clear cannot drop a qualified entry.
- Atomic Redis registration. HSET-then-EXPIRE loses the fresh marker when the
  user's existing hash expires between the two commands (or the process dies
  there); registration is now a single Lua script carrying both.
- Fail closed everywhere. Registration failure (after one retry) now REFUSES
  the terminal CAS instead of proceeding uncovered: the completion claim throws
  into the error path, the error path skips completeJob and leaves the job
  ACTIVE — deletion-visible by itself, recovered by the stale-running reaper —
  and abortJob returns a new retryable `fence_unavailable` failure the Stop
  route answers with 503 and the deletion quiesce treats as an unacknowledged
  stop (fence kept). The previous round's log-and-proceed is gone.
- Every terminal path enrolled. abortJob now owns its window (register before
  the abort CAS, clear in its finally — the Stop route's checkpoint prune and
  partial save run inside beforePublish, between CAS and publication); the
  interactive and background generation-error paths register before their
  completeJob; the resume controller's error finalization registers before its
  completeJob. A lost or thrown claim releases the marker after pending saves
  flush instead of holding the user's deletion behind the TTL.
- Every pending terminal unsettled. settleAbortFence defers on
  terminalPersistencePending for ALL statuses — including `aborted`, whose
  route-side persistence the previous guard missed.

Also: a direct Redis regression for scheduled stale-pause retention (the P2
test gap), and the PR description no longer claims to carry every commit.
2026-08-18 07:46:18 -04:00
Danny Avila
5955a6b2a0 fix: fence terminal response persistence from deletion; retain stale-pause evidence
Two blockers from the third external review round.

Terminal persistence visible to account deletion:

- The finalization marker was registered only when post-terminal TITLE work was
  possible, but every persistence-owning terminal CAS opens the same window: the
  claim drops the job out of the active set BEFORE the response save (and the
  background user-message/convo saves), so a deletion quiesce landing there saw
  neither an active job nor a marker and could cascade while the admitted request
  could still recreate messages. Both controllers now register the marker before
  every persistence-owning claim — the fresh-turn path and the HITL resume — and
  release it only after their pending saves (and any post-terminal title) have
  landed, on success and failure paths alike. The TTL bounds a crash.
- settleAbortFence no longer clears a complete/error fence while
  `terminalPersistencePending` is set: terminal at the CAS is not settled while
  the owner is still persisting. The job facade now surfaces the flag.
- The marker trio is REQUIRED by the runtime store contract (assertJobStoreV2
  refuses a store without it at configure time, keeping the failure loud and
  deterministic) while remaining optional on the legacy public IJobStore type for
  source compatibility. The silent degrade path from the previous round is gone —
  it was not deletion-safe.

Stale-pause recovery retains scheduled evidence:

- Three crash/timeout recovery paths — ApprovalLifecycle.failStalePausePersistence
  and the InMemory/Redis stale-pause cleanups — unconditionally stamped
  `completedAt`, putting a scheduled fire's failed-pause error terminal on the
  short completed TTL. A Mongo outage longer than that TTL erased the evidence and
  the reconciler recovered the run as `interrupted` instead of `error`. All three
  now follow the controller-observed path from the previous round: scheduled jobs
  omit `completedAt` (retained-evidence TTL) and stamp the error outcome.

The PR description now explicitly narrows the deletion guarantee for the remote
OpenAI-compatible/Responses paths (tracked in issue 14594).
2026-08-18 07:46:18 -04:00
Danny Avila
6e964d5897 feat: Scheduled Chats — agent-centric scheduled runs creating real conversations
Squash of the full review-hardened branch (PR 14540, supersedes 14373) onto
latest dev, preserving the exact verified tree. History prior to this commit
lived on the pre-squash branch; every invariant below survived 25 Codex review
rounds plus two external audit rounds (R26) with regression tests that fail
without their fixes.

Feature:
- Schedules CRUD + side-panel UI (cadence dialog, run cards, Run Now), roles/
  permissions (SCHEDULES:USE), interface.schedules availability, per-user limits
  and capacity slots, timezone-aware cadence with DST-conservative floors and
  misfire grace.
- Engine: single-process claim/fire loop with leases, loopback POST dispatch
  (signed schedule-fire JWT claims, per-occurrence idempotency key inside the
  route's clientRequestId charset), overlap/balance/capacity/duplicate skip
  policies with auto-disable streaks (too_many_failures, insufficient_balance),
  reconciliation from retained terminal-job evidence, erasure sweep.
- Scheduled runs create real conversations through the resumable agents chat
  path: HITL pauses surface on the card (requires_action), resumes re-apply the
  fire boundary's admission policy (revision fence, enabled, global kill switch,
  SCHEDULES:USE, availability) before continuing a billed generation.

Correctness invariants (the audit surface):
- Settlement discipline: every persistence-producing write happens-before a
  run's terminal outcome write; Stop/complete/pause race through single-winner
  terminal CAS claims (dev's TerminalJobClaim substrate) with retained,
  completedAt-less evidence for scheduled fires plus an owner-intended outcome
  stamp (scheduleOutcome) the reconciler prefers over re-derived success —
  round-tripped through the Redis hash mapper.
- Swallowed generation failures (client error content parts) classify to
  error/skipped_balance instead of success on both initial and resumed paths;
  stale stamps are refreshed evidence-first when persistence plus the Mongo
  outcome write both fail.
- Abort honesty: delivery judged by generation ownership and the CAS's actual
  from-status; republication escalates to the transport's acknowledged variant;
  the Stop route settles only genuinely paused runs.
- Account deletion: one-way barrier (deletionRequestedAt) with auth-cache
  tombstone-before-stamp, boundary rechecks across all auth strategies, quiesce
  of scheduled + interactive work with durable per-stream abort fences
  (positive-evidence acknowledgement only), owner-side finalization markers for
  post-terminal billed writes, deferred-deletion sweep (explicitly ensured
  partial index) that completes cascades autonomously. Remote OpenAI-compatible/
  Responses requests are documented as outside the quiesce and tracked in issue
  14594.
- Store compatibility: finalization markers optional on the legacy IJobStore
  contract with coherent degradation (registration fails -> synchronous title
  fallback; count reads 0).
2026-08-18 07:46:18 -04:00
Danny Avila
547bd8c4bf
🧵 feat: Persist View-Only Subagent Threads (#14957) 2026-08-18 07:41:42 -04:00
Danny Avila
7d62be2ad3
🕸️ feat: Run Saved Agent Teams as Subagents (#14944)
* feat: Add graph subagent integration

* style: Sort response usage test imports

* fix: Preserve lazy graph runtime context

* fix: Use isolated graph input helper

* test: Align graph integration fixtures

* fix: Preserve lazy graph runtime capabilities

* fix: Bound lazy graph metadata preload

* fix: Harden lazy graph resolution lifecycle

* fix: Coalesce lazy graph member resolution

* fix: Snapshot initialized graph members only

* fix: Preserve lazy agent runtime context

* fix: Preserve batched lazy context preparation

* fix: Preserve graph member capability bounds

* fix: reconcile graph subagents with execution profiles

* style: align graph subagent types with formatter
2026-08-17 18:02:52 -04:00
Danny Avila
aa35cd42b1
📬 feat: Add Durable Agent Trigger Delivery (#14925)
* feat: wire trusted agent trigger dispatch

* feat: add durable agent trigger delivery

* fix: annotate trigger envelope byte limit

* test: isolate trigger startup in server specs

* fix: fence trigger delivery during account deletion

* test: isolate trigger service in user controller specs

* fix: close trigger deletion admission race

* fix: harden account deletion fences

* fix: close durable trigger review gaps

* fix: require offline stale-fence recovery

* fix: type trigger lane sequence ids

* fix: fence admin user deletion triggers

* fix: make trigger deletion recovery durable

* fix: harden offline user deletion

* fix: serialize trigger lane publication

* style: sort trigger delivery imports

* fix: recover orphaned trigger publications

* fix: preserve trigger recovery ordering

* fix: fence trigger publication during purge

* fix: defer remote trigger deletion fences

* fix: close durable delivery cleanup races

* fix: drain CLI generation owners before deletion
2026-08-17 09:25:08 -04:00
Danny Avila
f8f118ef29
🛰️ feat: Execute Generic Agent Trigger Deliveries (#14921)
* feat: add generic agent trigger dispatch seam

* refactor: harden trigger dispatch contract

* fix: annotate envelope depth alias

* style: sort trigger envelope imports

* fix: reject unknown trigger dispatch modes

* fix: reject unknown trigger envelope versions

* refactor: validate complete trigger envelopes

* feat: add agent trigger execution host

* fix: enforce trigger delivery contracts

* fix: harden trigger admission path

* fix: finish trigger cancellation handling

* fix: retry strict steer rollout gaps

* fix: retry paused trigger steers

* fix: parallelize trigger admission setup
2026-08-17 08:45:49 -04:00
Danny Avila
57ea1137f6
🛡️ feat: Let Admins Restrict Stateful Workspace Scopes (#14910)
* feat: let admins restrict stateful workspace scopes

* fix: enforce stateful scope policy across agent paths

* fix: close stateful scope policy activation gaps
2026-08-17 01:29:19 -04:00
Danny Avila
7d850c308a
🧠 feat: Add Live Reasoning Labels (#14893)
* feat: add live reasoning labels

* fix: Stabilize reasoning label checks

* fix: Address reasoning label review findings

* chore: Bump Agents SDK for reasoning labels

* fix: Reset reused reasoning step evidence

* fix: Reconcile cleared reasoning labels

* fix: Fence reasoning label resets

* fix: Reset reasoning ownership before gap labels

* fix: Preserve THINK type through label reset

* test: Expect run-global reasoning revision
2026-08-16 18:11:55 -04:00
Danny Avila
fb8ae881cf
⏱️ feat: Show Run-Step Durations On Tool Cards (#14892)
* ⏱️ feat: Show Run-Step Durations On Tool Cards

Surfaces how long each tool call took, derived from the `closed_at` /
`created_at` pair already carried by `on_run_step_closed` — the same event
#14871 and #14873 use for the terminal status. No new event, no new SDK
surface.

The duration is stamped onto the content part at the same three sites as
`runStepStatus`, so it survives a reload and a resumable reconnect rather
than living only on the live React message:

- `callbacks.js`, on the aggregated part before the event is forwarded
- `RedisJobStore`, in the host-authored replay reconstruction branch
- `useStepHandler`, on the live message

Rendering lands in the shared `ProgressText`, which nine tool cards already
use, rather than in each card: one place decides whether a duration is shown
and how it reads, and the cards only forward the number. That keeps this from
adding a tenth independent state derivation to a component family whose
label/announcement/progress split is already the subject of AI-1810.

The value is deliberately absent rather than zero whenever it would be a
guess — no `created_at`, non-finite input, or a negative elapsed time from
two clocks that disagree, which is now reachable because a step can be opened
in one process and closed in another after a checkpoint resume. Sub-second
durations are suppressed as noise, and it renders only on a settled,
non-error card, where the slot is not already carrying the cancelled icon or
the error suffix.

For assistive technology the compact form (`3.5s`) is hidden and paired with
a spoken equivalent ("took 3.5 seconds"), both inside the button, so the
accessible name carries the duration without an `aria-live` region
re-announcing it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🎨 style: Sort Imports In Touched Files

The import-sort gate runs against the files a PR changes, so pre-existing
drift in `ProgressText.tsx` and `RedisJobStore.ts` surfaced on this branch.
Both were already unsorted on `dev`; this is the sorter's output, with no
semantic change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🐛 fix: Accept Partial Timestamps In Run-Step Duration Helper

`getReportableRunStepDurationMs` declared its parameter as
`Pick<RunStepClosedEvent, 'created_at' | 'closed_at'>`, where `closed_at` is
required. That contradicted the function's own purpose: every guard inside it
exists precisely to handle stamps that may be missing.

The Redis replay branch reconstructs closures from persisted JSON and holds
nothing stronger than "might be a number", so it failed to typecheck against
the narrower signature.

Widened to an exported `RunStepTimestamps` shape with both stamps optional,
rather than asserting at the call site — an assertion would move the decision
about what is trustworthy somewhere it cannot be enforced, which is the thing
the helper exists to centralize. Callers holding a fully-typed event still
pass, since a required field satisfies an optional one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🐛 fix: Suppress Duration When Failure Arrives As errorSuffix Alone

At every call site `error` carries cancellation while failure travels
through `errorSuffix` with `error` false, so gating the duration on
`!error` alone rendered "· 3.5s" beside "· failed" — and announced it.
The gate now checks both terminal-failure channels.

The original test pinned only the `error: true` path, which is why this
survived; the failed-via-suffix path is now pinned separately, both the
visible and the announced half.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🧩 refactor: Persist Raw Run-Step Durations, Threshold At Render Only

The three stamp sites filtered through the 1-second reportability
threshold before persisting, baking a presentation rule into stored
data: a 900ms step stored nothing, making "fast" indistinguishable from
"not derivable" and unrecoverable if the display rule ever changes.

Stamp sites now persist the raw `getRunStepDurationMs` value — absent
only when genuinely not derivable — and the renderer alone decides what
is worth showing, which `ProgressText` already did. Rendering is
unchanged. `getReportableRunStepDurationMs` is removed; it existed only
to serve the write-time filter, and a test now pins that sub-threshold
durations survive to storage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🐛 fix: Suppress Duration On Backgrounded Bash And Code Cards

A backgrounded call's run step closes when dispatch returns the handle,
so the stamped duration is the dispatch time. Rendering it beside
"Running/Finished in background" misstated a detached task's runtime as
seconds — and violated the "settled card only" rule, since the card is
still tracking the detached run. Scope is exactly the two cards that
parse background handles.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🌍 fix: Format The Sub-10s Decimal For The Active Locale

The fractional seconds value was interpolated as a raw JS number, which
hardcodes the en-US decimal point into every language — "1.4s" where
the locale writes "1,4 s" — and translators cannot fix a number
formatted in code. The value is now formatted via Intl.NumberFormat
with i18n.language, following MessageTimestamp's pattern of threading
the language into the util; plural-key selection stays on the numeric
value. A malformed language tag falls back to the plain number.

Also documents the two accepted limits of the derivation, so they read
as decisions rather than oversights: positive clock skew is
undetectable from a single stamp pair, and the value is wall-clock
elapsed, so a step held open across a suspension (checkpoint resume,
HITL approval wait) includes that time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🐛 fix: Persist A Durable `backgrounded` Marker Through Harvest; Localize Minute Digits

Codex round 3, both findings confirmed.

**Background origin survived only as transient state.** The dispatch
handle in `tool_call.output` and the live status-marker attachment are
both gone once the harvester patches the settled task's stdout over the
handle — so the round-2 suppression (`backgroundHandle == null`) came
back on after harvest or reload, showing dispatch time as the task's
runtime. Following the same rule as e4bd15d (persist facts, decide at
render): the harvest patch now stamps `backgrounded: true` onto the
tool call in the same atomic write that erases the handle — on the heal
path too, which re-applies over full-row saves that reverted the part.
The cards gate on handle-or-marker; the dispatch duration itself stays
stored.

**Minute-branch digits bypassed locale formatting.** The seconds branch
went through Intl.NumberFormat while minutes interpolated raw numbers,
so Arabic/Persian locales flipped to ASCII digits above one minute. All
interpolated values now flow through the (renamed) formatDurationValue;
an ar-EG test pins the localized digits.

data-schemas cannot be installed in this environment (same npm ci 403 as
packages/api), so message.ts/harvest.ts are syntax-checked with
resolution off and otherwise verified by review; CI runs their real
typecheck and suites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🧪 test: Assert The `markBackgrounded` Stamp In Harvest Expectations

The successful-harvest test's exact `toHaveBeenCalledWith` object did
not include the newly forwarded `markBackgrounded`, so the API suite
would fail on it. All three harvest-call expectations now assert
`markBackgrounded: true` — the exact-object one of necessity, the two
`objectContaining` ones deliberately, since the durable stamp (on the
best-effort file-failure path and the reapply heal alike) is now part
of the behavior under test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🎨 style: Wrap Harvest Spec Expectation Per Prettier

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-16 16:17:02 -04:00
Danny Avila
06bf324cf0
🛤️ feat: Per-Agent Code Execution Routing With Stateful Session Scopes (#14848)
* feat: route code execution per agent profile

* chore: sort execution profile imports

* test: preserve stateful environment literal types

* fix: isolate stateful code environments by user

* fix: preserve per-agent code routing end to end

* fix: route code priming by execution profile

* fix: isolate code profile lifecycle state

* fix: preserve mixed-profile code resources

* fix: complete stateful skill routing
2026-08-16 09:42:15 -04:00
Danny Avila
eb3b353712
📡 fix: Publish App-Level MCP Tool Catalogs Without a Reserved Revision (#14858)
* 📡 fix: Publish App-Level MCP Tool Catalogs Without a Reserved Revision

Shared MCP servers advertised no tools to agents, so every turn failed with
"configured to use MCP tools, but none are available" (#14857).

`replaceAppServerTools` returned false whenever a publication carried no
`publicationRevision`, but only `refreshChangedTools` reserves one. Every other
app-level publisher — the first-connect snapshot, reinitialization, on-demand
catalog reads, the retained-catalog restore — was silently dropped. The agent
path fails closed on that drop: the skipped write returns null, so reinitialize
yields no tools and the turn 503s.

Startup hid it. `connectAppServers()` defers the initial refresh and calls
`refreshToolList()` itself, which does reserve, so a boot that reaches its MCP
servers looks healthy. Only a lazily created app connection — the server not yet
up when LibreChat boots, a dropped connection, a cold cache — takes the
unreserved path.

`ConnectionsRepository` now reserves before its own `tools/list`, matching the
list_changed path; a failed reservation publishes unordered rather than failing
the connection. Publishers with no pre-fetch reservation point have already
fetched by the time they reach the cache, so they take the next revision at write
time instead of being discarded. `mergeAppTools` still publishes at revision 0 and
stays deferential to a live catalog.

* 📡 fix: Bind App Catalog Ordering to the Fetch That Produced It

Addresses review feedback on the previous commit: allocating a revision at
publish time lets a slow `tools/list` of an old catalog outrank a newer one that
reserved after it started, and it would let the retained-catalog restore — which
republishes deliberately pre-mutation data — outrank a live catalog.

Ordering now travels with the data. `fetchToolsSnapshot` reserves before its
first page and returns the ticket on the snapshot, so every app-level publisher
reads the revision belonging to the read it is publishing rather than one
allocated at an unrelated moment. `fetchOrderedToolsSnapshot` carries the
refresh's revision when it defers to one, since that is whose catalog it returns.

With the reservation at the single point where app-level tools are read, no
publisher can forget it, so `replaceAppServerTools` goes back to refusing an
unordered write: a publication that lost its ticket fetched at an unknown time
and cannot be ordered.

A failed reservation is reported as `orderingUnavailable` rather than swallowed,
which keeps the list_changed path retrying instead of publishing a catalog that
would be silently dropped, and leaves inspection unaffected by a transient cache
outage.

`MCPServerInspector.getToolFunctions` becomes `getToolCatalog` and returns the
revision with the tools, so there is no variant that quietly discards ordering.

* 📡 fix: Retry an Empty App Catalog That Could Not Reserve Ordering

Review follow-up. The no-tools-capability branch destructured the reservation
result and dropped `orderingUnavailable`, publishing without a revision when the
revision store was transiently unavailable. That write is rejected in silence,
and unlike the snapshot branch this one returned without reaching
`refreshToolList()`, so whatever the server last advertised stayed in place until
the connection was recreated or the cache expired.

Both branches now route an unreservable catalog through the same retry path.

* 📡 fix: Serve Tools Whose Shared Catalog Write Could Not Be Ordered

Review follow-up. Only the shared catalog write needs ordering; the tools
themselves were just read from the server and are correct to serve. Discarding
them because the write could not be ordered is what turns a cache failure into a
server that appears to have no tools at all, which is the reported symptom.

`updateMCPServerTools` now returns the tools it built when the publication has no
reserved revision, instead of null. A superseded write still discards — there
another replica holds something newer.

Reinitialization also asks the connection to republish under backoff when its
snapshot could not reserve ordering, so the shared catalog does not stay cold
until something else triggers a refresh.

* 📡 fix: Surface a Discarded App Catalog Instead of Debug-Logging It

#14857 went a release without a diagnostic because the only trace of a dropped
app-level catalog was a debug line no deployment runs. Operators saw agents fail
every turn with nothing in the logs to explain it, and the reporter had to read
the source to find the cause.

A publication discarded because it cannot be addressed or ordered means this
server's tools are unavailable to every agent that selected them, and serving an
unpublished catalog means every request re-fetches it. Both are warnings now. A
superseded write stays at debug: concurrent replicas produce it routinely and the
winner already holds newer tools.

Tests pin the level, so a later refactor cannot quietly make the failure silent
again.

* 🧪 test: Pin the Reinitialize Path's Catalog Ordering

Reinitialization is the path an agent falls back to when the shared catalog is
cold, so it is where #14857 surfaced as "configured to use MCP tools, but none
are available". Nothing pinned that it forwards the ordering its snapshot was
fetched with, nor that it asks the connection to republish a catalog it could
not order.

Both assertions fail against the pre-fix source.
2026-08-15 12:48:23 -04:00
Anubhav Anand
a2ad0aa0c8
🤐 feat: Allow Promptless Sends When Files Are Attached (#13717)
* ✨ feat: Allow sending file attachments without a text message

When an agent asks the user to upload a document, the user could attach
the file but still had to type a placeholder message ("OK", "Here is
the file") before the send button enabled and the submit guard let the
message through.

Attachments now count as submittable content:

- New isSubmittableMessage(text, fileCount) util: non-whitespace text
  OR at least one attached file.
- ask() in useChatFunctions uses it instead of bailing on empty text,
  so an empty draft with attached files submits.
- SendButton receives the attached file count and enables accordingly.
- ChatForm only marks the text field as required when no files are
  attached, so react-hook-form validation no longer blocks handleSubmit.

Submitting an empty draft with no attachments is still rejected at all
three layers.

Fixes #13646

* Address review: support replayed file-only turns + drop empty vision text

- ask(): count replayed attachments (overrideFiles) in the submittable
  check and skip it entirely for regenerate, so a file-only message can
  be regenerated or saved-and-resubmitted instead of being rejected as
  empty.
- formatVisionMessage(): omit the text content part when the message text
  is empty. Anthropic rejects empty text content blocks with HTTP 400,
  and an empty block adds nothing for other providers; image-only sends
  now format cleanly. Added formatMessages tests for with-text and
  image-only (Anthropic + other) cases.

* Address review: keep attachment-only turns valid for providers, answer mode, and titles

- formatMessage: substitute minimal text when a user turn carries files but no
  inline content, so Anthropic does not reject an empty user message for RAG or
  code-environment attachments.
- assistants chatV1: send the same stand-in for attachment-only Threads
  messages, which reject an empty body. The persisted message keeps empty text.
- ChatForm: attachments no longer make an empty draft submittable in answer
  mode, where submitText consumes the click without answering or sending.
- agents request: seed title generation from attachment filenames when the turn
  has no text, so immediate-mode titles are not invented from an empty string.
- useChatFunctions.regenerate.spec: mock the utils barrel over the real module
  so new exports resolve.

* Cover the agents path for attachment-only turns

AgentClient formats its payload with the SDK's formatMessage, not the local
one, so the earlier guard missed the endpoint the feature actually targets: an
attachment-only turn still reached Anthropic as an empty user message. Apply
the same stand-in after the file-context and quote merges, so a turn that
already gained inline content is untouched.

* Carry filenames on freshly attached files

The fresh-file submission mapping copied only file_id, filepath, type, and
dimensions, so the attachment-only title fallback read an undefined filename
and produced nothing. Include filename, and cover it with a test that submits
an empty draft with one attachment.

* Address review: cover assistants v2, fresh agent attachments, editor, and title fallback

- agents client: the current turn has no files during buildMessages, so read
  the resolved attachments from message_file_map instead. The previous guard
  only ever fired for persisted historical turns.
- assistants chatV2: the default assistants endpoint routes here, so it needs
  the same stand-in body chatV1 got.
- assistants title: fall back to filenames, then the response, and keep the
  default title rather than saving an empty one.
- EditMessage: retained attachments make an empty edit submittable, matching
  the composer, so the overrideFiles replay path is reachable from the UI.

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2026-08-15 12:47:31 -04:00
Danny Avila
88747f0ad8
🩺 fix: Render Stopped Run Steps From Explicit Status (#14871)
* 🩺 fix: Render Stopped Run Steps From Explicit Status

Tool calls decided "still running" vs "stopped" with a whole-message
heuristic:

  const cancelled = !isSubmitting && progress < 1 && !hasError;

That inference cannot tell which step actually stopped. An aborted step
keeps spinning while `isSubmitting` is still true, and when submitting
ends, every unfinished part flips to "Cancelled" at once regardless of
which one died.

`@librechat/agents` v3.4.6+ emits `on_run_step_closed`, a terminal
per-step signal carrying `status` and timestamps — including for steps
swept at end-of-run because the caller aborted. The pinned 3.5.1 already
ships it; nothing consumed it.

- `StepEvents.ON_RUN_STEP_CLOSED` plus `RunStepClosedEvent` /
  `RunStepStatus` types mirroring the SDK payload.
- `PartMetadata.runStepStatus` — a dedicated field, since `status` is
  already claimed by activity-label and question-form parts.
- Server handler forwards the event without the visibility gating the
  other step handlers apply: a step whose open reached the client must
  get its close, or the client is left inferring again.
- `useStepHandler` writes the terminal status onto the tool call part.
- Both decision points (`ToolCall`, the shared `useToolCallState`) prefer
  explicit status, keeping the heuristic as fallback for messages saved
  before this and endpoints that do not emit the event.

Threaded through the five cards sharing `useToolCallState`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🩹 fix: Address Codex Review On Run Step Closure Rendering

- Persist the terminal status server-side. The handler emitted the
  closure without folding it into `contentParts`, so the status existed
  only on the live React message: a reload or resumable reconnect
  dropped it and fell back to the very heuristic this fixes. Now stamped
  onto the aggregated tool-call part (via `stepMap`, falling back to the
  event's own index) before forwarding.

- Honor terminal status independently of output parsing. Gating on
  `hasError` meant a `failed` step with unparseable output rendered as
  "cancelled", while a `failed`/`cancelled` step whose output did parse
  as an error was not terminal at all and shimmered indefinitely when no
  completion event arrived. A closed step now forces progress complete
  and reports `failed` as an error state on its own authority.

- Pass the status to the second `BashCall` branch, which rendered the
  same updated component without it.

- Reuse `Agents.RunStepClosedStatus` in `PartMetadata` instead of
  redeclaring the union, so a future SDK status cannot diverge between
  the event and the persisted part.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* ♿ fix: Replay Closed Status On Redis Resume, Announce Failures

- Apply closure events during Redis reconstruction. The stamp added in
  the previous commit mutates only the originating process's in-memory
  `contentParts`; a resumable reconnect landing on another replica
  rebuilds from `RedisJobStore.getContentParts`, whose allowlist omits
  `on_run_step_closed`. The status was therefore absent from the sync
  snapshot and, being snapshot-covered, never redelivered as pending —
  so multi-replica resume fell back to the whole-message heuristic.
  Handled as a host-authored event alongside `on_steer_applied` and
  `on_activity_label`, since the SDK aggregator has no notion of it.

- Announce terminal failures in the live region. Forcing terminal
  progress for a closed step meant a `failed` tool reached the
  `aria-live` region through `getFinishedText()`, which only special-
  cased cancellation and otherwise announced "completed function" —
  telling screen-reader users the opposite of what the card showed. A
  regression introduced by the previous commit; error states now
  announce failure before any completion string.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

* 🎯 fix: Resolve Closed Steps By ID, Never By Index

The steer and HITL offset wrappers clone and shift only `ON_RUN_STEP`
and `ON_AGENT_UPDATE`; every other event passes through untouched. A
stored `on_run_step_closed` therefore carries the SDK's unshifted index,
while the part it belongs to was rebuilt at the shifted one. Any run
containing a steer insertion or HITL resume would stamp the status onto
an earlier tool card, or none — leaving the real card on the fallback
heuristic while mislabeling a different one.

- Redis reconstruction builds a step ID -> index map from the replayed
  `on_run_step` payloads (which carry the shifted index) and resolves
  closures against it, mirroring what the live callback does via
  `stepMap`.

- The live handler drops its `?? data.index` fallback for the same
  reason. Skipping is the safe failure: a missing status degrades to the
  old heuristic, whereas a misplaced one actively mislabels the wrong
  card.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-15 10:14:16 -04:00
Ravi Kumar L
bc6392d05b
🪢 fix(langfuse): mark provider-backed agent traces (#14833)
* fix(langfuse): mark provider-backed agent traces

* fix(langfuse): mark stored response traces

* test(langfuse): isolate provider marker setup
2026-08-14 10:28:25 -04:00
Danny Avila
abc669ab58
🩹 fix: Restore the @librechat/api Build and Remove Legacy Code (#14808)
* 🧹 chore: Remove Dead Legacy Agent Controller

`_LegacyAgentController` has been unreachable since the resumable path became
the only route: it is unreferenced, unexported, and untested. It had also
drifted out of compilability against the live file — line 2009 called
`attachConversationCreatedAt(req, { userId, conversationId, isNewConvo })`
against the 3-argument signature declared at line 97, which would await
`undefined` and then throw dereferencing `resolved.createdAt`.

Keeping it was not free. It carried a third independent copy of the response
message-id wiring (`getReqData`, `onStart`, four `updateMetadata` calls), so
every change to how a generation identifies its response row had a dead third
site to keep in step, and no test to say whether it had been kept in step.

Removing the block leaves `createCloseHandler` and the `sendEvent`,
`clientRegistry`, `requestDataMap` and `handleAbortError` imports with no
remaining callers, so those go too. `AgentController` was a three-line
passthrough to `ResumableAgentController`; the real controller is now exported
directly, which also matches the `[ResumableAgentController]` prefix every log
line in the file already uses. `server/routes/agents/chat.js` binds the export
to its own local name and passes the same five arguments, so the route is
unchanged.

No behavior change: 379 lines removed, 2 added.

* 🩹 fix: Remove Duplicated Anchor Block Breaking the `@librechat/api` Build

`dev` does not build. `packages/api/src/agents/activityPhases/runtime.ts`
carries two byte-identical 98-line copies of the same block (former lines
516-613 and 614-711), so rolldown fails to parse it:

    [PARSE_ERROR] Identifier `AnchorFields` has already been declared

The duplicated block is the anchor-construction work from #14805:
`AnchorFields`, `laterDefinedIndex`, `foldedAgentIds`, `boundedAnchor` and
`mergeAnchors`. #14807 was squashed from a branch that predated #14805 and
re-included that commit, so both copies landed. Only the `type` produced an
error — the four function declarations simply redeclare.

This removes the first copy. The two blocks were verified byte-identical
before the cut, and the resulting file has no duplicate top-level
declarations, is missing nothing that #14805 introduced, and retains
everything new to #14807 (`ResolvedPosition`, `resolvePosition`).

Verified: `tsdown` builds, `tsc --noEmit` clean, `config/circular-deps.mjs`
green across all five graphs (it was reporting `✗ @librechat/api` purely
because the build it shells out to was failing), and the 68 tests in
`activityPhases/runtime.spec.ts` pass.

Carried here rather than in a separate PR because this PR's checks cannot go
green until it lands: the failed `packages/api` build cascades into e2e, MCP
list_changed, bombadil and the Docker image jobs.
2026-08-13 19:57:32 -04:00
Danny Avila
df6e15a0de
🔖 feat: Bound Parent Activity Phases With an Exclusive End Index (#14768)
* 🧭 fix: Finalize Parent Activity Phases at Run Completion

* 🧭 fix: Preserve Activity Phase Boundaries

* 🎨 fix: Format Activity Phase Boundary Check

* 🧭 fix: Ignore Late Label Artifacts at Phase Completion

* 🧭 fix: Preserve Logical Activity Phase Membership

* 🩹 fix: Narrow Optional Activity Phase Marker

* fix activity phase tail boundaries

* fix activity phase test lint

* fix straddling activity phase batches

* preserve activity phase boundaries at scale

* fix persisted activity phase final boundary

* fix resumed activity phase edge cases

* fix sparse activity phase grouping

* fix sparse activity phase tail scan

* fix resumed activity phase text fallback

* fix sparse activity phase completion scans

* avoid sparse activity phase runtime scans

* stabilize sparse activity phase resumes

* support activity phases on current ts target

* preserve sparse phase reservations

* finalize activity phase boundary handling

* avoid sparse phase start scans

* fix activity phase final text bounds

* tighten activity phase summary boundaries

* format activity phase boundary checks

* leave final commentary outside activity phases

* recognize lane-tagged final activity text

* rebase retained activity boundaries on resume

* bound activity phase collection work

* correct resumed phase activity count

* resolve late reasoning before phase completion

* preserve lane-tagged final answers

* assert durable activity phase bounds in e2e

* preserve empty finalized activity phases

* ignore empty reasoning at phase completion

* format phase completion guard

* fix(api): retain overflow reasoning anchors

* perf(api): index overflow reasoning anchors

* perf(api): skip empty reasoning index scans

* fix(api): reconcile completion boundaries efficiently
2026-08-12 23:43:35 -04:00
Danny Avila
1a3e2aebcb
🛰️ fix: Attach Request-Scoped MCP Servers (#14780)
* fix: attach request-scoped MCP servers

* fix: satisfy MCP static checks

* fix: format MCP runtime hint
2026-08-12 23:43:02 -04:00
James Todaro
e696b07619
🧾 fix: Honor Disabled Transactions on the Token-Count Fallback Path (#14774)
`AgentClient.recordTokenUsage` had no `transactions` parameter, so the setting
never reached `createTransaction`, whose guard reads it from the object it is
handed. `transactions?.enabled === false` saw `undefined` and the write went
ahead.

This path is reached only from `BaseClient`'s fallback branch, when the provider
returns no usable stream usage, so the bulk path masked it wherever usage is
reported. Where it is not, the setting had no effect at all.
2026-08-12 22:31:13 -04:00
Danny Avila
dccef82254
🪶 chore: Aggregate Empty MCP Tool Logs (#14767)
* fix: aggregate empty MCP tool logs

* fix: retain server names in MCP tool logs
2026-08-12 22:29:49 -04:00
Danny Avila
ee8c0abe2d
🪝 feat: Execute Agent Plugin Command Hooks (#14755)
* 🪝 feat: Execute Agent Plugin Command Hooks

Implement the missing PluginHookExecutor boundary so deployment plugins'
ai.librechat/hooks/hooks.json documents execute instead of loading inert:

- Command executor runs handlers as child processes outside the API
  process: Claude-shaped JSON payload on stdin, exit 0 + JSON stdout as
  sanitized hook output, exit 2 blocks with stderr as the reason, minimal
  allowlisted environment plus PLUGIN_ROOT/PLUGIN_DATA, abort-signal kill
- Plugin loading carries the parsed hooks document on the contribution and
  threads hookCapabilities from startup, gated on the operator opt-in
  DEPLOYMENT_PLUGIN_HOOKS (off by default: parsed-but-inert with warning)
- Runs register every ready plugin hook onto the per-run HookRegistry after
  internal policy hooks, with once-per-conversation SessionStart dedup

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Harden Plugin Hook Execution Boundary

Address CI and Codex/Copilot review findings on #14755:

- Break the agents -> plugins import cycle: the run seam now reads a
  PluginHookSource wired at startup (mirrors the tool-approval registry)
- Tighten plugin ask decisions to deny unless the run has HITL wiring,
  so an un-resumable interrupt can never strand OpenAI-compatible callers
- Scope cross-run dedup keys by authenticated user and handler identity:
  caller-supplied conversation ids cannot collide across principals, and
  sibling SessionStart handlers all fire; once handlers persist across runs
- Replace a literal NUL byte in source with an escape (file diffed binary)
- Kill the whole detached process group on abort, not just the shell
- Map exit 2 on events without a decision channel to preventContinuation
- Reserve PLUGIN_ROOT/PLUGIN_DATA against allowlist overrides, quote
  PowerShell args, cap captured output by bytes with one-pass decoding,
  and serialize payloads inside the executor's error boundary
- Fix import ordering flagged by the static checks

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Close Plugin Hook Policy and Namespace Gaps

Address the second Codex review round on #14755:

- Drop updatedInput from plugin command outputs: hooks in one dispatch all
  receive the original arguments, so a plugin rewrite would reach the tool
  without the approval policy re-evaluating it (host-only now)
- Translate Claude tool aliases (Bash/Write/Edit/Read) to LibreChat runtime
  names in matchers, with reverse payload mapping, so Claude-authored guards
  fire instead of planning ready and never matching
- Key once-only state by declaration position as well as handler contents,
  so sibling declarations with identical handlers stay independent
- Thread sessionStartSource through createRun and mark the HITL resume
  rebuild as 'resume', so SessionStart matchers see the real lifecycle

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Translate Regex-Form Claude Tool Aliases

Address the third Codex review round on #14755: alias translation now
substitutes word-bounded tokens, covering regex matchers like ^Bash$ and
^(Write|Edit)$ that the exact-token pass left registered against Claude
names and silently never firing. A regex whose alias sits inside a
character class or escape is rejected as unmapped so it fails loudly at
plan time instead of never running.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Scope Alias Translation and Reuse Load-Time Plans

Address the fourth Codex review round on #14755:

- Add the WebSearch -> web_search alias so Claude-authored web-search
  guards fire against the LibreChat built-in
- Apply alias translation only to tool-name events; a StopFailure matcher
  like ^Bash failed$ stays untouched and keeps matching the error text
- Reuse each plugin's load-time hook plan at run registration instead of
  re-planning up to 512 handlers on every chat turn

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Translate Aliased Tool Inputs and Harden Hook Domains

- Present aliased tool inputs under Claude field names (file_path,
  old_string, new_string, including nested edits), so Write/Edit/Read
  guards see the fields they check instead of silently allowing
- Derive the alias table from canonical tool-name definitions
  (BashExecutionToolDefinition, CREATE_FILE_TOOL_NAME, Tools.web_search)
  instead of a parallel hand-authored table
- Reject matchers naming Claude built-ins with no runtime equivalent
  (Task, Glob, Grep, WebFetch, ...) as unmapped at plan time instead of
  registering guards that never fire
- Replace per-event Sets and Stop special-cases with an exhaustive
  EVENT_TRAITS record over HookEvent, so new engine events demand
  explicit semantics at compile time
- Move cross-run once-state behind a PluginHookOnceStore seam with a
  least-recently-marked memory default: active conversations refresh
  their keys each turn, so capacity eviction can no longer re-fire a
  conversation that is still in use; the seam admits a shared-cache
  store for multi-replica deployments
- Gate portable-only command handlers at plan time on Windows via a new
  supportsHandler capability (commandWindows or shell powershell
  required) instead of spawning bash that cannot exist
- Kill Windows hook process trees with taskkill /t on abort
- Require declaration indices on execution requests, stamped from the
  plan instead of defaulted at execution time

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Keep Group SIGKILL Escalation Armed After Wrapper Exit

An aborted hook whose descendant ignores SIGTERM could leak that
descendant: the wrapper shell's exit fired close, which cancelled the
scheduled group SIGKILL. The escalation timer is now never cancelled —
it is unref'd and killTree already tolerates a vanished process group,
so a redundant late sweep is harmless while a surviving descendant is
reliably killed at the grace deadline. killGraceMs is configurable on
CommandExecutorOptions, with a regression test driving a trap-protected
descendant past the wrapper's exit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Scope Once Retention by Conversation and Reject Clear Source

- Restructure the once store around conversation scopes: registration
  touches the scope every run, so rarely-matching once handlers keep
  their keys while the conversation is active; eviction removes whole
  idle conversations (capacity counts conversations, not keys)
- Reject SessionStart matchers naming the clear lifecycle source at
  plan time — no LibreChat run-construction path emits clear, so the
  handler would plan ready and never fire; wildcard warning text now
  reflects the sources that actually occur
- Make the SIGKILL-escalation regression test real: the surviving
  descendant redirects its stdio away from the captured pipes so the
  wrapper's close fires while it is still alive, exercising the
  window a close-time cancellation would leak

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Bound Alias Tokens by Tool-Name Characters and Host Shells

- Translate Claude aliases (and reject unsupported built-ins) only when
  delimited by characters that cannot appear in a runtime tool name:
  action tool names preserve hyphens, so an alias embedded in a longer
  name like deploy-Bash-v2_action_example_com stays the literal tool
  name instead of being rewritten into a matcher that never fires
- Reject PowerShell-only command handlers on POSIX hosts at plan time
  (and skip them at runtime): bash cannot run PowerShell syntax, so the
  guard would fail open; a handler with both variants still runs its
  portable command
- Handle rejected asynchronous once-store calls: a failed touch logs
  instead of raising an unhandled rejection during run construction,
  and a failed markOnce lookup fails open per the store's documented
  over-fire direction

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Probe Group Liveness Before Cancelled or Delivered SIGKILL

The never-cancelled escalation timer could signal a recycled
process-group id when an aborted hook's whole tree exits early in the
grace window. Escalation now probes the group with signal 0: close
cancels the timer only when the group is verifiably empty, and the
deadline re-probes before delivering the group SIGKILL, so surviving
descendants are still reaped while a fully-dead group never receives a
blind late signal. The residual probe-to-signal race is documented as
irreducible without pidfd support.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Gate Windows Escalation on Root-Process Liveness

Windows taskkill /t walks the tree from the root process, so once Node
observes the root's exit an escalation pass can reap nothing and a late
forced taskkill could only hit a recycled PID. The liveness gate is now
platform-aware in one helper: POSIX probes the process group with
signal 0, Windows checks the root's observed exit state, and both the
close-time cancellation and the deadline delivery consult it — no
platform retains a blind late signal. Orphaned SIGTERM-ignoring
descendants on Windows are documented as the platform limitation they
are without Job Objects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Scope Payload Namespace to Declarations and Reap Stray Workers

- Reverse name/input translation now applies only to declarations whose
  matcher actually required Claude-alias translation: the plan records
  requiresToolNameTranslation per entry, so a native-authored matcher
  like ^create_file$ receives native tool names and fields instead of
  Claude-shaped payloads its guard never expected
- Coordinate the two dedup layers via a shouldExecute gate on the
  executor: a declaration suppressed by spent once-state declines
  before claiming the per-input dedup slot, so an identical handler
  under an overlapping matcher can still claim it and fire its own
  independent once-key instead of being permanently shadowed
- Reap process groups that outlive a successful hook: a backgrounded
  worker left running after normal wrapper exit gets the same
  term-then-escalate sequence an abort uses, since unsupported async
  handlers mean no lifecycle owns such processes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🧰 chore: Vendor Pocock Codebase-Design and Architecture Skills

Adds mattpocock/skills engineering/codebase-design and
engineering/improve-codebase-architecture (MIT, license included) under
.claude/skills so future sessions share the deep-module vocabulary
(module, interface, depth, seam, adapter, leverage, locality) and the
architecture-review process. Force-added past the /.claude/ gitignore
deliberately; relocate if project skills should live elsewhere.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 refactor: Extract Process-Tree Reaping Into a Reaper Module

Tree lifecycle — five of the last seven review findings — lived as
event-handler wiring inside runCommand with its invariants in comments.
It now sits behind a two-method seam: createReaper(child, graceMs)
exposes reap() and onClose(), hiding the term-grace-escalate state
machine, the per-platform liveness gates, the recycled-id guards, and
the clean-exit sweep. The executor shrinks to capture-and-parse, and
the reaper is unit-tested directly with real process trees through its
own interface instead of only via whole-executor integration runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Scope Translation Per Alternative and Sweep at Root Exit

- Track which runtime tool names alias translation produced, so a
  mixed-namespace matcher like Bash|create_file presents Claude-shaped
  payloads only for bash_tool invocations while the natively-authored
  create_file alternative keeps native names and fields; a capability
  omitting the produced-names list keeps declaration-wide translation
- Sweep the process tree at root exit as well as close: a backgrounded
  descendant holding the captured pipes delays close until it dies, so
  the exit-time sweep terminates it promptly instead of stalling the
  hook until its timeout aborts
- Pass the primary agent's resolved model and identity into the plugin
  hook context, so SessionStart payloads carry model and agent_type
  instead of always omitting them

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Default Wildcard Declarations to the Document Namespace

- Matcherless (or wildcard) tool-payload declarations now inherit the
  hook document's Claude namespace: with no alternatives to carry
  namespace evidence, the plan marks them for declaration-wide reverse
  translation, so a wildcard guard inspecting standard Claude names and
  fields sees Write/file_path instead of silently failing open on
  native payloads; PostToolBatch entries translate the same way
- Recognize aliases delimited by regex metacharacters: dots leave the
  tool-name boundary class (runtime names never contain them — action
  ids underscore domain dots), so ^Bash.*$ translates to ^bash_tool.*$
  instead of registering a guard that never fires
- Expand Claude's ${CLAUDE_PLUGIN_ROOT} spelling in hook commands and
  export it in the child environment alongside PLUGIN_ROOT
- Scope SessionStart once-keys by lifecycle source, so a startup firing
  no longer suppresses the conversation's resume rebuild

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Normalize Claude Structured Hook Output

Stock Claude hooks return decisions under hookSpecificOutput
(permissionDecision/permissionDecisionReason), surface context there,
and use continue:false plus the legacy approve/block decisions — none
of which the sanitizer's native field names recognized, so a guard that
works in Claude silently allowed in LibreChat. Parsed JSON now passes
through a dialect normalizer first: hookSpecificOutput fields map to
decision/reason/additionalContext, continue:false becomes
preventContinuation, approve becomes allow, and block becomes deny on
events that block by denying. Native fields win when both dialects
appear, and the ask-to-deny gate applies to the Claude dialect too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Validate Native Decisions and Slim Once Keys

- Strip malformed native output fields before the dialect merge, so a
  placeholder like {"decision":null} can no longer suppress a valid
  Claude permissionDecision into a silent allow; only recognized
  decision tokens take precedence
- Preserve the caller's working directory in hook payloads: cwd now
  reports the run's session context instead of the plugin installation
  path, which commands already receive as PLUGIN_ROOT and which the
  executor still uses as each process's working directory
- Store a compact sha256 digest instead of the full serialized handler
  in once keys: declarations may carry 32 KB commands and 256 args, and
  the previous key embedded them in every retained conversation scope

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

* 🪝 fix: Validate Decisions Per Event Channel and Control Post-Tool Blocks

- Accept native decision tokens only from the target event's own
  vocabulary: "continue" is valid on Stop but malformed on a tool
  event, where it previously survived validation, blocked the Claude
  dialect merge, and was then dropped by sanitization into a silent
  allow
- Translate a structured "block" on events with no deny channel
  (PostToolUse, PostToolUseFailure, and the other prevent-trait events)
  into preventContinuation with the block reason as stopReason, instead
  of discarding it and returning a reason that controls nothing
- Document why LibreChat runs supply no payload cwd: tool paths address
  a remote code-execution sandbox rather than the API host where hook
  commands run, so no host directory describes the run

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWXQZD2WzeAsvee4eRdWWc

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-12 13:21:15 -04:00
Danny Avila
88e08c91e8
🧷 fix: Preserve Elicitation Answers Across Aborts (#14745)
* fix: preserve elicitation answers across aborts

* chore: sort stream type imports

* fix: guard malformed resolved answers

* fix: close abort answer race gaps

* fix: retain exact answers across pauses

* fix: retain answers across resumed pauses

* test: satisfy HITL fixture types

* fix: retain legacy answers through approvals

* fix: preserve answers in reconnect snapshots

* test: type legacy answer fixture

* fix: bind legacy answers to paused content

* test: guard optional resume content

* fix: resume questions without streamed content

* fix: Preserve legacy answers through abort filtering

* test: Narrow reconstructed abort fixture

* test: Type abort transform fixture explicitly

* fix: Isolate answers with missing ask content

* test: Type missing-content ask fixture
2026-08-12 07:32:48 -04:00
Danny Avila
236ee6c1ab
🧭 fix: Re-Anchor Parent Activity Phase Bounds (#14741)
* test: cover parent activity phase finalization

* test(e2e): stabilize parent phase coverage

* fix(agents): reanchor parent activity phase bounds

* fix(agents): preserve delayed tools in activity phases

* test(agents): keep phase slice bounds typed

* fix(agents): preserve sparse activity phase bounds

* test(e2e): read structured phase replies
2026-08-11 10:16:57 -04:00
Danny Avila
7347cfc195
🍡 feat: Batched User Questions With A Single Bounded Answer Form (#14737)
* feat: support batched user questions

* test: align batched question fixtures

* fix: harden batched question lifecycle

* test: submit batched HITL answers in e2e

* fix: address batched question review findings

* fix: preserve invoke return typing
2026-08-11 01:06:16 -04:00
Danny Avila
09cbd54f48
🪆 fix: Rebase Activity Phase Bounds over Sparse Content (#14729)
The aggregator writes content parts at provider-source indexes, which can
skip slots and leave holes in contentParts. Array.prototype.map preserves
those holes and the Map constructor iterates them as undefined, so
rebaseActivityPhaseBounds threw "Iterator value undefined is not an entry
object" at the end of every run with sparse content — deterministic with
parent phase summaries enabled, on both the completion and resume paths.

Build the identity map with an index loop that skips nullish slots. Holes
must stay out of the map: one undefined key would falsely match every hole
in previousParts as a retained part and corrupt the rebased bound.
2026-08-10 15:05:10 -04:00
Danny Avila
a3cec67e08
🪆 feat: Add Parent Activity Phase Summaries (#14721)
* feat: add activity phase summaries

* fix: preserve activity phase lifecycle semantics

* fix: satisfy activity phase type checks

* fix: simplify activity phase status mapping

* style: format activity phase changes

* fix: rebase activity phase bounds after shaping

* fix: link activity phase trace ancestry

* fix: reconcile activity phase bounds

* style: format activity phase reconciliation test

* style: align activity phase assertion

* fix: retain reasoning across commentary

* fix: preserve activity phase boundary state

* fix: detect renderable phase children

* test: type parallel phase assertion

* chore: bump agents SDK for activity phases

* fix: retain unphased lane reasoning

* fix: preserve tool group expansion across phases

* style: format phase expansion regression

* fix: preserve phase interaction state efficiently

* perf: skip sparse phase segment holes

* perf: partition phase segments with offsets

* fix: preserve phase boundaries and cursor state

* test: align activity phase regressions with CI

* test: keep phase context mock hoist-safe
2026-08-10 13:41:37 -04:00
Danny Avila
1bccc2bc18
📡 fix: Refresh MCP Tools After List-Changed Notifications (#14686)
* fix(mcp): handle dynamic tool list changes

Co-authored-by: Pascal Garber <pascal@artandcode.studio>

* test(mcp): fix CI validation

* fix(mcp): keep dynamic tool catalogs live

* fix(mcp): harden dynamic catalog lifecycle

* test(mcp): use typed startup connection

* test(mcp): isolate dynamic e2e fixtures

* fix(mcp): refresh tools after reconnect

* fix(mcp): close dynamic catalog cache gaps

* test(mcp): update OAuth connection mocks

* fix(mcp): preserve app snapshot ownership

* style(mcp): sort connection imports

* fix(mcp): close review race conditions

* fix(mcp): preserve cache ownership edges

* fix(mcp): harden recovery lifecycle

* fix(mcp): guard tool-less app refresh

* fix(mcp): fence distributed cache races

* fix(mcp): retire stale connection state

* fix(mcp): keep tool snapshots authoritative

* fix(mcp): fence stale app tool publications

* style(mcp): sort repository test imports

* test(mcp): mock empty startup publication

* fix(mcp): preserve app publication generations

* fix(mcp): harden publication recovery races

* fix(mcp): address tool catalogs by runtime config

* fix(mcp): load scoped catalogs for assistant writes

* fix(mcp): harden catalog publication recovery

* fix(mcp): serialize forced connection replacement

* fix(mcp): serialize ordinary creation with replacements

* fix(mcp): harden catalog fallback boundaries

* fix(mcp): close lifecycle fencing gaps

* fix(mcp): preserve catalog authority on failures

* fix(mcp): compensate failed catalog mutations

* fix(mcp): fence catalog refresh ordering

* style(mcp): sort agent loader imports

* fix(mcp): cancel stale connection creation

* fix(mcp): fence catalog coordination

* fix(mcp): close catalog race windows

* fix(mcp): harden cross-pod catalog fencing

* fix(mcp): close catalog lifecycle edges

* style(mcp): sort assistant imports

* fix(mcp): reject stale recovery authority

* fix(mcp): restore static catalog on every startup

* fix(mcp): order app catalog publications

* style(mcp): sort catalog revision imports

* fix(mcp): separate catalog allocation and commit fences

---------

Co-authored-by: Pascal Garber <pascal@artandcode.studio>
2026-08-08 13:50:21 -04:00
Marco Beretta
39f5f9d846
⚡ perf: Agent List and Model Selector at Scale (#14601)
* perf: cut serial round trips from the agent list query path

The agent list was the slowest path on first page load. Three separate
problems compounded:

- `getListAgentsHandler` chained its reads: two ACL lookups, the avatar
  refresh cache probe and the viewer skill scope all resolved serially
  ahead of the list query, and `attachOwnerContacts` added two more hops
  after it. The four independent reads now resolve together, and the
  avatar refresh runs alongside the list query instead of before it -
  refreshed paths reach the response through `urlCache`, not through
  whatever the list query happened to read. Serial hops per request drop
  from 7 to 4 on a warm cache.

- The avatar refresh loaded the user's whole accessible agent set (up to
  MAX_AVATAR_REFRESH_AGENTS) to discover which entries were S3-backed.
  Scoping the query to `avatar.source` means deployments on any other
  file strategy match nothing instead of walking the full set.

- `fetchAllAgentPages` walked cursor pages at the server's default size
  of 100, and callers consume the flattened result, so every extra page
  was a serial round trip for no benefit. It now requests the server
  maximum. Measured over a 2,860 agent account: 29 requests / 1.65s
  before, 3 requests / 0.29s after.

Also parallelizes the conversation file reads in `initializeAgent`. The
convo file refs and the execute_code thread walk share no inputs, and the
two code-file lookups depend only on `threadFileIds`, so the chain of six
serial reads on every turn collapses to two. This one is time to first
token the user waits through.

* perf: virtualize the model selector agent list

Opening the agents submenu with a large agent set froze the tab and could
kill it outright. With ~10k accessible agents the submenu blocked for over
15 seconds and took the heap from 96MB to 911MB. Four per-row costs were
being multiplied by the full list, which rendered unwindowed:

- `useIsActiveItem` allocated a MutationObserver per row (10,016 of them
  for one dropdown). Replaced with an Ariakit store subscription, which
  needs no observer at all and returns a boolean so a row only re-renders
  when its own active state flips.

- `useFavorites` ran per row, opening a jotai subscription, a query
  subscription and a mutation each time. Hoisted to one call per endpoint.

- Each row rescanned `endpoint.models` to recover `isGlobal`, a field the
  parent had already discarded from the array it was mapping. The parent
  now passes it down from a lookup map.

- The list itself is now windowed above 100 rows. Ariakit's composite only
  knows about mounted rows, so arrow-keying to the window edge previously
  found no next item and let focus escape the nested menu, closing it;
  `handleBoundaryNavigation` scrolls the next index in, waits for it to
  mount, then moves the composite onto it. Navigation inside the window is
  left to Ariakit.

Open drops from >15s to 96ms, mounted rows from 10,028 to ~18, DOM nodes
from 123,346 to ~1,000, and the heap no longer grows. Verified in browser:
arrow keys track 1:1 to index 238 and back, and click selection works.

* perf: serve the model selector from the shared VIEW agent query

The model selector asked for EDIT-scoped agents whenever the marketplace
is enabled, while `useAgentsMap` and `useMentions` asked for VIEW. Since
the cache key includes the params, that was two distinct entries, so first
page load ran the paginated walk twice and held two copies of the whole
agent list in memory. Measured against a 10k agent account: 22 list handler
invocations per page load, now 11.

Collapsing the two by asking for the same permission everywhere would have
changed what the selector shows - under the marketplace the EDIT scope is
what makes it "My Agents", with discovery handled by the marketplace entry.
So the list endpoint now marks each row with `isEditable`, resolved from an
ACL read folded into the existing parallel batch (no extra serial hop), and
the selector filters the shared VIEW response instead of refetching. A
VIEW-scoped list for a user with 2861 visible / 361 editable agents returns
exactly 360 rows flagged editable, matching what the EDIT query returned.

`AgentSelect` deliberately keeps its own EDIT query: it reads `skills` and
`skills_enabled`, which `sanitizeViewerSkillScope` strips from VIEW-scoped
responses. It also only mounts when the builder panel is open, so it is not
part of the first-load cost.

The field is set unconditionally rather than omitted when false so that a
client talking to an older server sees `undefined`, keeps every agent, and
degrades to showing too many rather than none.

* fix: address review findings on the agent list at scale

Three issues from review, all confirmed against the code before fixing.

Avatar refresh no longer runs alongside the list query. `updateAgent` writes
through `findOneAndUpdate` on a `timestamps: true` schema, so refreshing an
avatar advances `updatedAt` — the field `getListAgentsByAccess` sorts and
cursors on. A write landing after the first page's snapshot moved that agent
ahead of the returned cursor, dropping it from every later page and silently
truncating the caller's flattened list. This was a regression introduced when
the two were parallelized; serializing them costs nothing on the common path,
because a cache hit returns without issuing any query, so only the
once-per-30-minutes miss pays for the ordering. The new test asserts the write
lands before the list snapshot and fails against the parallel version.

The virtualized list no longer inserts a focusable grid into the combobox.
`List` spreads its props onto `Grid`, whose defaults are `role="grid"`,
`containerRole="row"` and `tabIndex={0}`; inside Ariakit's listbox that added a
tab stop ahead of any row and put grid/row semantics between the listbox and its
options. All three are now neutralized so focus and ARIA stay with the combobox
items.

The list also resets to the top when the filter changes. `Grid` keeps its scroll
offset across prop changes and clamps an out-of-range offset to
`totalRowsHeight - height`, the end of the shorter list. Scrolling deep and then
searching landed on the tail: measured at row 626 of 667 matches, with only
those rows mounted and reachable by keyboard. Keying the list on the search
value restores row 0.

* fix: declare option position and set size for the virtualized model list

Once the model list is windowed, only the mounted slice exists in the listbox,
so a screen reader infers position and total from ~19 elements instead of the
real set — announcing "3 of 19" partway through 10,014 agents.

Model rows now carry aria-posinset and aria-setsize. The marketplace entry and
any model specs share the same numbering, because they are options in the same
listbox: declaring the values on some options while leaving others to be
inferred from the DOM would make the set internally inconsistent. Both are
omitted entirely when the list is short enough to render unwindowed, where the
DOM holds every option and the implicit values are already correct.

Verified against a 10,014 agent account: the marketplace entry reports 1 of
10015, the first models 2 and 3, and after scrolling to row 4999 the leading
mounted model reports 5001 of 10015 with 19 options in the DOM.

* 🩹 fix: Address Follow-Ups on the Agent List at Scale

Corrects residual issues in the agent-list perf work, all inside its own scope.

- Forward `idOnTheSource` through `PermissionService.findAccessibleResources`
  so `getUserPrincipals` skips the user-document read. The list handler resolves
  three permission sets per request and each was paying its own `User.findById`;
  the auth strategies already normalize the field to a value or null.
- Gate the editable-set lookup on its own predicate instead of borrowing
  `canReturnSkillConfig`. The two answer unrelated questions and only coincide
  today, so redefining the skill flag would have marked every agent editable.
- Log mapping failures in the list response instead of swallowing them.
- Apply the walk page size after the caller's params in `fetchAllAgentPages`.
  A caller limit only changed page size, never what the flattened walk returned,
  so `defaultAgentParams`' `limit: 10` would have turned one request into 301.
- Carry `isEditable` on the agent rows the create and update mutations write
  into the list cache. Mutation responses omit the field, so those rows lost it.
- Document `isEditable` as list-only, ACL-derived, and fail-open on absence.
- Restore the truthiness guard on the thread walk in `initializeAgent`. Widening
  it to `!= null` made an empty `parentMessageId` issue a full-conversation read
  against an anchor that can never match.
- Await `getConvoFiles` directly rather than calling `.then()` on it, restoring
  tolerance for synchronous test doubles.
- Correct the avatar-refresh comment: the projection was never full documents,
  and the real reason to filter is that an unfiltered budget is self-reinforcing.

Tests: both new `initialize` tests and both new backend tests are
mutation-verified; the concurrency test fails under either serialization order.

* fix: preserve ACL isEditable when merging agent mutation responses

Mutation responses omit list-only isEditable. Inferring true from write
success promoted VIEW-only rows into the editable subset for MANAGE_AGENTS
callers who can PATCH agents their ACL marks non-editable.

* fix: sort imports in agent mutations test

ESLint import-order check failed on the isEditable cache-preservation test.

* 🧷 fix: Carry isEditable Onto Duplicated Agent List Rows

`useDuplicateAgentMutation` prepended the raw duplicate response to the cached
list, and mutation responses omit the list-only `isEditable` field. The row
survived the "My Agents" filter only by failing open on `undefined`, so it would
disappear the moment a consumer read the flag strictly.

Duplicating grants the caller ownership, so the new row is editable outright;
this is the create case rather than the merge case `mergeAgentListRow` handles.
Last cache write on this path that did not carry the field.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-08-07 21:04:55 -04:00
Danny Avila
5ff46d8c67
🛟 fix: Stop Agents When Code Resources Cannot Recover (#14651)
* fix: Block Agents When Code Resources Cannot Recover

* fix: Preserve Resource Recovery Failures Across Agent Paths

* fix: Centralize Fatal Agent Initialization

* chore: sort agent imports
2026-08-07 00:33:28 -04:00
Dustin Healy
6d2f29266c
🔑 feat: Refresh-Capable Google Admin OAuth Sessions (#13832)
* 🔑 feat: Refresh-Capable Google Admin OAuth Sessions

Google admin sessions cannot be refreshed today. Three gaps add up to that:
passport.authenticate('googleAdmin', ...) in api/server/routes/admin/auth.js
never sets access_type=offline, so Google omits the refresh_token from its
token response; createOAuthHandler in api/server/controllers/auth/oauth.js
only forwards a refresh token into the admin exchange payload when the user's
provider is 'openid' AND OPENID_REUSE_TOKENS is enabled; and
/api/admin/oauth/refresh is openid-only, calling openid-client.refreshTokenGrant
against the configured OIDC issuer. OpenID admins refresh transparently
because all three are in place for them.

This PR closes all three. The googleAdmin authenticate call now passes
accessType: 'offline' and prompt: 'consent' so Google issues a refresh token
on consent; the chat-side googleLogin is untouched. The shared socialLogin
verify callback now passes the IdP refreshToken through as passport's third
argument (info), landing on req.authInfo, with the two-argument call shape
preserved when no refresh token is present so existing strategy tests stay
valid. createOAuthHandler reads req.authInfo?.refreshToken for non-OpenID
admin providers and forwards it into the exchange code; the OpenID branch
and its OPENID_REUSE_TOKENS gate are unchanged. /api/admin/oauth/refresh
now accepts an optional provider field ('openid' | 'google', default 'openid').
The new Google branch POSTs grant_type=refresh_token to
https://oauth2.googleapis.com/token, decodes the returned id_token for the sub
claim, looks up the admin user by googleId, enforces tenant scope and
ACCESS_ADMIN, and mints a fresh LibreChat JWT in the same response shape
/oauth/exchange returns. It is gated on GOOGLE_CLIENT_ID and
GOOGLE_CLIENT_SECRET being set (returns 503 GOOGLE_NOT_CONFIGURED otherwise);
unknown provider values return 400 INVALID_PROVIDER.

* 🔁 fix: Harden Google admin refresh against bot review findings

Five validated findings from the initial bot pass:

socialLogin.js: mirror the OpenID migrate-or-reject pattern on the email
fallback. When an existing user is found by email and the stored provider
id is empty, persist the refreshed sub so the refresh path can later bind
to it. When the stored id is present and differs, reject as AUTH_FAILED
to prevent identity-swap, matching the existing OpenID behavior in
packages/api/src/auth/openid.ts.

oauth.js: scope the non-OpenID admin refresh-token forwarding to
provider === 'google'. The previous else branch would have forwarded a
Discord refresh token (passport-discord supplies one) into the admin
exchange payload even though /api/admin/oauth/refresh only accepts
openid or google, leaving the admin client with a token it could not
refresh.

admin/auth.js (refreshGoogleAdminSession): drop id_token from the
mandatory-fields check. Google's OAuth refresh response is documented to
include id_token only conditionally, so the previous mandatory check
broke refresh whenever Google omitted it. Decode id_token when present
(fast path); when absent, call Google's userinfo endpoint with the
access token to read sub. Wrap tokenResponse.json() in try/catch and
return IDP_INCOMPLETE on parse failure instead of a generic 500.
Tighten access_token to a typeof string check.

admin/auth.js (refreshGoogleAdminSession): reuse serializeUserForExchange
for the response user so the Google refresh shape matches /oauth/exchange
and the OpenID branch exactly (full _id, id, email, name, username, role,
avatar, provider, openidId). The previous Google-specific subset dropped
fields the admin client relies on for later provider-specific refreshes
and disambiguation.

Tests cover each fix: socialLogin's migration and rejection cases, the
oauth.js Discord-gating case, the userinfo fallback path on missing
id_token, CLAIMS_INCOMPLETE when both id_token and userinfo are absent,
IDP_INCOMPLETE on a non-JSON token body, and the full response shape on
the happy path.

* 🧪 fix: Add updateUser to appleStrategy test mock for socialLogin migration

The shared socialLogin verify callback now invokes `updateUser` when the
email-fallback path discovers a same-provider user with an empty provider
id, persisting the refreshed sub. The Apple strategy test's `~/models`
mock did not stub `updateUser`, so the migration path hit
`TypeError: updateUser is not a function` and failed the
`should handle existing user and update avatarUrl` case in CI shard 1/3.

* 🧹 refactor: Move Google admin refresh into TypeScript @librechat/api helper

Per repo guidance (CLAUDE.md): all new backend code must be TypeScript in
/packages/api, and /api is a thin JS wrapper. The previous commit landed the
Google admin refresh flow as ~120 lines of new JS inside
api/server/routes/admin/auth.js, which violates that. This commit extracts
the flow into a new TS helper at packages/api/src/auth/googleRefresh.ts and
reduces the route handler to a thin dep-wiring wrapper.

The helper exports applyGoogleAdminRefresh(deps, options) with the same
shape as the OpenID applyAdminRefresh: callers pass findUsers, getUserById,
canAccessAdmin, and mintToken as deps so the package stays free of /api
model imports and capability/session helpers. The route handler now builds
those deps from the existing model + capability + token modules and calls
the helper, mapping AdminRefreshError to the documented HTTP responses.

While moving the code, the helper now guards getUserById with
Types.ObjectId.isValid before the direct-lookup branch, matching the
OpenID admin path at packages/api/src/auth/refresh.ts. Without this guard
a malformed user_id from the admin client would hit Mongoose findById's
CastError and surface as a 500 INTERNAL_ERROR instead of falling through
to the documented sub-based lookup.

Tests move with the code: packages/api/src/auth/googleRefresh.spec.ts now
owns the helper's behavior (token endpoint, userinfo fallback, ObjectId
guard, USER_ID_MISMATCH/TENANT_MISMATCH/USER_NOT_FOUND/FORBIDDEN, rotated
refresh-token pass-through, GOOGLE_NOT_CONFIGURED, IDP_INCOMPLETE on
non-JSON body, CLAIMS_INCOMPLETE when both id_token and userinfo miss).
The route-level api/server/routes/admin/auth.refresh.test.js drops the
duplicated end-to-end Google cases and keeps a smaller surface: route
delegates to applyGoogleAdminRefresh with the right deps + options, maps
AdminRefreshError to HTTP status/code, falls through to 500 for unknown
errors, and rejects unknown providers with INVALID_PROVIDER.

* 🔁 fix: Tighten Google admin refresh and limit social-login changes

Brutal-review findings on top of the upstream feature work.

socialLogin.js: the migrate-or-reject pattern from the previous commit
applied to every provider's chat-side verify callback, not just the admin
flow. Gate both branches on `options.existingUsersOnly` so the chat-side
googleLogin / facebookLogin / etc. keep their pre-existing email-fallback
behavior unchanged. Tests follow: restore the original `should fallback to
finding user by email` chat-side case and re-add the migration and
mismatch-reject cases as admin-only by passing `{ existingUsersOnly: true }`
to socialLogin in those tests.

googleRefresh.ts: add a defense-in-depth `isEmailAllowed(user)` dep that
the helper invokes before `canAccessAdmin`. Mirrors the
`isEmailDomainAllowed` check the initial Google admin login already runs,
so a deployment that removes a domain from `registration.allowedDomains`
after issuance can no longer mint fresh JWTs for that admin via refresh.
The route handler wires it up with `resolveAppConfigForUser` +
`isEmailDomainAllowed`, falling back to `baseOnly` config for users
without a tenantId.

googleRefresh.ts: drop the unreachable `?? ''` defensive coalescing in
`fetchGoogleTokenset`. The `GOOGLE_NOT_CONFIGURED` guard upstream already
narrows `clientId`/`clientSecret` to non-empty strings; the function
takes a narrowed `GoogleAdminRefreshConfiguredOptions` shape and
`applyGoogleAdminRefresh` constructs that shape after the guard.

* 🔒 fix: Apply brutal-review hardening to Google admin refresh

Tighten the Google OAuth refresh flow against all outstanding code review
findings: enforce JWT aud claim verification against the configured clientId
(ISSUER_MISMATCH on mismatch), reject ambiguous googleId matches (limit:2 in
findUsers, USER_ID_MISMATCH when multiple rows match), scope the authInfo
refresh-token carrier to the Google provider only, add TOCTOU re-read defense
after the admin googleId migration write in socialLogin, deduplicate
canAccessAdmin/mintToken closures via buildAdminRefreshClosures shared by both
OpenID and Google refresh paths, document rotation semantics on
AdminExchangeResponse.refreshToken, standardise all log prefixes to
[admin/oauth/refresh], and expand test coverage for all new paths.

* 🔒 fix: Reject refresh for users migrated off the Google provider

The interactive Google admin login path in socialLogin.js already rejects
a user whose provider field is not 'google', returning AUTH_FAILED. Without
a matching guard in the refresh path, a user migrated to OpenID could use
an unexpired Google refresh token to keep minting admin JWTs indefinitely.

Add a PROVIDER_MISMATCH check after resolving the user in both the direct
getUserById branch and the findUsers fallback branch of resolveAdminUser,
mirroring the provider gate the interactive path enforces.

* 🔒 fix: Add ban check and fix domain allowlist on admin OAuth refresh

Two gaps in the /api/admin/oauth/refresh route:

Add middleware.checkBan to the route chain before preAuthTenantMiddleware,
matching the gate that /login/local and createOAuthHandler already apply.
Without it a banned admin could keep minting JWTs until their IdP refresh
token expired.

Replace getAppConfig({ baseOnly: true }) in the non-tenant isEmailAllowed
closure with getAppConfig({ role: user.role }), which includes DB-layer
overrides from the admin panel. baseOnly returns only YAML-derived config,
so any allowedDomains list maintained entirely through the admin panel was
silently inert on this path. Extract isEmailAllowedForUser as a shared
helper, move it into buildAdminRefreshClosures so both Google and OpenID
refresh paths enforce domain policy consistently, and add isEmailAllowed
to AdminRefreshDeps in the TS package so applyAdminRefresh can invoke it.

* 🔒 fix: Harden admin OAuth refresh against user bans, tenant scope gaps, and cross-tenant migration

Post-identity-resolution ban check: the initial checkBan middleware fires before the
refresh token is exchanged and req.user is populated, so it can only evaluate IP bans.
After applyGoogleAdminRefresh/applyAdminRefresh resolves the user identity, we now
synthesize req.user and re-run checkBan against the resolved user's id before emitting
the JWT, so a user-level ban is enforced even from a fresh IP.

Domain allowlist now includes userId: the getAppConfig call in isEmailAllowedForUser
was passing only role, missing user and group-level allowedDomains overrides that the
initial OAuth callback's checkDomainAllowed enforces via userId. Both branches now
pass userId so buildPrincipals takes the full user+group+role resolution path. The
tenant branch is also inlined (replacing resolveAppConfigForUser) to accept userId,
wrapped in tenantStorage.run for correct Mongoose scoping and cache-key resolution.

Cross-tenant email-fallback migration: the Passport verify callback fires before
tenantContextMiddleware, so findUser({email}) is unscoped and can return a same-email
user from another tenant. Writing googleId onto that document permanently corrupts
the other tenant's account. Migration is now blocked for users with a tenantId;
single-tenant users are unaffected.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-08-07 00:10:34 -04:00
Danny Avila
56175af0b5
🎟️ fix: Reconcile MCP OAuth Readiness Across Pods (#14629)
* fix: stabilize MCP OAuth readiness across pods

* fix: harden MCP readiness review findings

* fix: resolve CI type check and terminal OAuth polling

* fix: address MCP OAuth readiness review

* fix: align MCP OAuth readiness state

* test: stabilize MCP OAuth readiness assertion

* fix: reject stale MCP OAuth callbacks

* fix: close distributed MCP OAuth readiness gaps

* style: sort Redis MCP test imports

* fix: preserve MCP OAuth polling across rolling pods

* fix: finalize distributed MCP OAuth readiness

* fix: preserve runtime-detected MCP OAuth

* fix: report runtime MCP OAuth readiness

* fix: preserve live MCP OAuth classification

* style: sort MCP connection imports
2026-08-05 19:42:26 -04:00
Danny Avila
489bc02d4a
🧭 fix: Fail Closed When Expected MCP Tools Are Unavailable (#14646)
* fix: fail closed when expected mcp tools are unavailable

* test: strengthen MCP handoff coverage

* fix: clarify unavailable MCP tool guidance

* fix: preserve MCP discovery for empty catalogs
2026-08-05 17:30:57 -04:00
Dustin Healy
3f0a1ec8d9
🛡️ fix: Run message-filter PII patterns on a linear-time regex engine (ReDoS) (#14554)
* 🛡️ fix: Run message-filter PII patterns on a linear-time regex engine

The messageFilter.pii middleware compiled admin-configured customPatterns with the native RegExp engine and ran them synchronously against every message on the shared event loop, so a catastrophic-backtracking pattern such as (a+)+$ could stall the entire process (native RegExp takes tens of seconds at roughly 32 characters) and take the instance down for every user.

Compile these patterns with RE2JS, a linear-time RE2 port with no native addon, so catastrophic backtracking is impossible regardless of the pattern rather than something the code tries to detect. Patterns using features RE2 does not support, such as backreferences, fail to compile and are dropped and logged exactly as an invalid pattern already is. The filter only tests for a match, so this is a drop-in engine swap with no behavior change for valid patterns.

* 🛡️ fix: Reject RE2-incompatible messageFilter patterns at config load

The customPatterns regex was validated with native RegExp at config load, but the runtime now compiles it with a linear-time engine (RE2) that does not support backreferences or lookaround. Such a pattern passed validation, then failed to compile and was silently dropped at request time, quietly removing PII protection after upgrade.

Reject backreferences and lookaround during config validation with an explicit message, and document RE2 syntax in the example config instead of "JavaScript-flavor". The runtime engine remains the authoritative boundary and still drops-and-logs anything this load-time check misses.

* 🧹 test: Use direct MessageFilterPiiConfig annotations in the PII specs

The added ReDoS cases satisfy the exported MessageFilterPiiConfig type directly, so the `as unknown as` assertions were unnecessary. Annotate the config objects directly, matching the repo's type-safety guidance.

* 🧹 fix: Reject named backreferences in messageFilter patterns at config load

Extend the config-load check to also reject named backreferences (\k<name>), which are valid JavaScript regex but unsupported by the linear-time runtime engine, so they surface at load rather than being dropped at request time. Together with the existing numeric-backreference and lookaround checks this covers the RE2-incompatible construct set; the runtime engine remains authoritative.

* 🛡️ fix: Preserve Unicode whitespace matching in messageFilter starter patterns

RE2's \s is ASCII-only, so after the engine swap the built-in api-key and Bearer starters no
longer matched a secret separated by non-ASCII whitespace (e.g. a non-breaking space), which
native RegExp did match. Broaden the whitespace classes to [\s\p{Zs}] so those patterns keep
their original coverage, and add a regression test for a non-breaking-space separator.

* 🛡️ fix: Validate messageFilter patterns with the RE2 engine at config load

Replace the syntax blacklist (numeric/named backreferences, lookaround) with authoritative
validation: config load now compiles each custom pattern with the same linear-time engine the
runtime uses, so any RE2-incompatible construct (including control escapes like \cA) is rejected
at load with a clear error instead of being silently dropped at request time.

The validator is swappable and defaults to native RegExp so browser builds add no engine; the
server wires the RE2-backed check at startup via configureMessageFilterRegexValidator in both
entry points.

* 🛡️ fix: Match the full whitespace set in messageFilter starter patterns

RE2's `\s` omits the vertical tab and `\p{Zs}` omits U+2028, U+2029, and
U+FEFF, so a separator built from one of those characters slipped past the
`api-key` and `Bearer` starter patterns and reached the model. Broaden the
starter whitespace class to the full JavaScript whitespace set so those
separators are covered again.

* fix: fail closed when messageFilter.pii compiles to zero patterns

DB and admin config overrides bypass the RE2 schema validation (it only
runs at YAML load), so an override whose only pattern is RE2-incompatible
was dropped at compile time, left zero patterns, and let the request
through. compile() now returns a failClosed flag when a config declared
patterns but every one failed to compile; the middleware returns 400 and
findPiiMatchInMessages returns a distinct misconfigured match that the
OpenAI and Responses controllers surface with an admin-facing message.

* 🛡️ fix: Fail closed when any messageFilter.pii custom pattern drops

compile() previously set failClosed only when every pattern dropped (patterns.length === 0 && dropped > 0). With the default starters present, a single RE2-incompatible custom override incremented dropped but left patterns.length > 0, so the filter silently enforced only the surviving subset and text matching only the dropped rule passed.

failClosed now keys off dropped > 0, so any dropped custom pattern blocks with the misconfigured 400. YAML patterns are RE2-validated at load, so dropped stays 0 for valid configs and only unvalidated DB or admin overrides can trip it. Reframed the two keeps-others-active specs to assert fail-closed and added a default-starters partial-drop regression.

* 🧹 fix: Correct the misconfigured JSDoc and drop redundant casts in the PII specs

The misconfigured flag now means any configured custom pattern failed to compile, not that every pattern failed, so its JSDoc on PiiMatch is updated to match. The partial-drop regressions now use direct MessageFilterPiiConfig annotations instead of as-unknown-as casts, keeping the specs type-checked, consistent with the rest of the suite.
2026-08-05 13:42:18 -04:00
Danny Avila
b11978017d
🧱 fix: Enforce Agent Runtime File Trust Boundaries (#14577)
* fix: secure agent runtime file metadata

* chore: sort agent resource test imports

* fix: Align Agent Tool Resource Types

* fix: Rehydrate Agent Image Resources

* fix: preserve remote agent file authorization
2026-08-02 14:18:28 -04:00
Danny Avila
db6ba5392a
🪢 fix: Bind MCP OAuth Secrets to Trusted Endpoints (#14578)
* fix: bind MCP OAuth secrets to trusted endpoints

* fix: bind stored MCP OAuth clients during refresh

* fix: address MCP OAuth review findings

* fix: bind stored MCP OAuth credentials

* fix: make MCP OAuth credentials generation-safe

* test: update MCP OAuth uninstall binding fixtures

* fix: harden MCP OAuth credential persistence

* fix: scope MCP OAuth refresh single-flight

* style: sort MCP OAuth token imports
2026-08-02 13:38:58 -04:00
Danny Avila
7e74f8eb8c
🪪 fix: Strip Unresolved Header Placeholders at Final Resolution (#14595)
Unresolved {{LIBRECHAT_USER_*}} header templates leaked literally to
upstream providers when user context was missing at resolution time
(e.g. async title generation racing client disposal), letting a gateway
trust LibreChat's own template syntax as an account identity.

resolveHeaders now takes an opt-in stripUnresolved flag that blanks any
resolvable-but-unresolved LIBRECHAT_USER/BODY/OPENID placeholder, enabled
at every final resolution boundary (resolveConfigHeaders, model fetches,
Google init, summarization overrides, azureAssistants init). Staged
passes that resolve again later with more context are left untouched, as
is the async-resolved {{LIBRECHAT_GRAPH_ACCESS_TOKEN}} and unknown names.

titleConvo now resolves headers from the req captured at entry instead of
re-reading this.options.req, which disposeClient nulls concurrently.

Fixes #14580
2026-08-02 06:38:06 -04:00
Danny Avila
59395a6bf0
🪢 refactor: Move Agent Execution Seam Before Initialization (#14581)
* refactor: move agent execution seam before initialization

* refactor: type librechat agent request extensions

* refactor: read envelope values from descriptors

* fix: preserve envelope types and validation errors

* fix: bound agent envelope traversal
2026-08-01 18:25:34 -04:00
Danny Avila
3191f6975a
🏷️ fix: Skip Title Generation for Preempt-Incomplete Turns (#14571)
Some checks failed
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / pack (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
GitNexus Index / index (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Helm Chart Tags / Ignore non-main push (push) Has been cancelled
Sync Helm Chart Tags / Sync chart tags (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / publish-npm (push) Has been cancelled
GitNexus Index / post-index (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
2026-08-01 09:01:13 -04:00
Danny Avila
3dc5532111
🏢 fix: Preserve Tenant Context for Partial Response Saves on Disconnect (#14567) 2026-08-01 08:18:32 -04:00
Danny Avila
6f45a9e32e
🔗 fix: Normalize MCP Tool Keys at Every Producer, Resolve Raw Names via Aliases (#14553)
* 🔗 fix: Normalize MCP Tool Keys at Every Producer, Resolve Raw Names via Aliases

Tool keys had two spellings that could diverge for any server whose name
contains characters outside [a-zA-Z0-9_.-]: the tool cache (and registry
inspector) built keys with the RAW server name, while runtime instances
are named with normalizeServerName(serverName). Three code comments
already asserted "tool keys embed the normalized server name" - no
producer honored it. For a special-character server that meant:

- definitions-only mode shipped raw def names the model echoed back,
  but the executor's tool map held the normalized instance name, so
  every call failed with "Tool not found";
- per-tool tool_options (defer_loading / allowed_callers /
  run_in_background / describe_intent) were persisted under raw keys
  that never matched the definition names the option passes resolve
  against, so builder settings were silently inert;
- tool-key parsing against normalized candidate lists silently fell
  back to last-delimiter splitting, which mis-parses delimiter-bearing
  tool names.

The reconciliation is one contract enforced in three moves:

1. PRODUCERS NORMALIZE. The tool cache (packages/api/src/mcp/tools.ts)
   and the registry inspector build keys with the normalized server
   name, matching the instance names MCP.js has always assigned. The
   builder's tool ids, agent.tools entries, tool_options keys, and
   definition names all flow from these keys, so every model-facing
   name now agrees. The cache STORE stays keyed by the raw config name.

2. CONFIG LOOKUPS RESOLVE ALIASES. New shared helpers in data-provider
   (buildServerNameAliases, normalizeMCPToolKey) map a parsed
   normalized name back to the raw config name that the registry,
   config maps, tool cache, and plugin-auth rows are keyed by. Applied
   in the definitions loader closure, handleTools grouping,
   createMCPTool's parsing fallback, getUserMCPAuthMap, and the MCP
   tools endpoint - matching both spellings so legacy raw keys keep
   resolving.

3. LEGACY DATA HEALS AT ONE BOUNDARY. initializeAgent rewrites
   raw-keyed agent.tools entries and tool_options keys to the
   normalized form (normalizeAgentToolKeys) before anything consumes
   them, so agents persisted under the old convention load their tools
   AND have all four per-tool options honored. Placeholder and
   server-pin tokens stay raw - they are config-identity references,
   not model-facing names.

Servers whose names are already in the safe character set (the common
case) produce byte-identical keys before and after; the fast path
allocates nothing. Stale Redis-cached raw keys self-heal via the
existing reconnect-on-missing path within one cache cycle.

* 🧯 fix: Deterministic Alias Collisions + Raw Names in Definition Metadata

Two review findings on the normalization contract:

- Two configured server names that normalize to the same segment (e.g.
  'Sales Force' and 'Sales:Force' -> 'Sales_Force') produce inherently
  ambiguous tool keys; the alias map silently resolved last-wins, so a
  tool selected from one server could execute against the other's
  config. buildServerNameAliases now resolves collisions to the FIRST
  configured name deterministically, and resolveMCPServerContext warns
  once per colliding pair per process so the operator can rename one
  server. A collision-resistant identifier would change every existing
  tool key, so detection + stable routing is the right treatment here;
  startup-time config validation can follow separately.

- The definitions loader resolved parsed (normalized) server names to
  raw only inside the ToolService closure, while the definition
  metadata (serverName -> mcpRawServerName) kept the normalized value.
  Server instructions are keyed by raw config names, so a
  special-character server's instructions were silently omitted in
  definitions-only mode. loadToolDefinitions now takes rawServerNames,
  resolves the boundary against both spellings, and stores the RAW
  name in definition metadata - consistent with the instance path.

* 🧯 fix: Heal Stale Caches, Skill Allowed-Tools, and Builder Selectors

Three review findings on the normalization rollout, all in the
transition class:

- Stale cache entries (P1): the definitions-only loader treats the
  per-server tool map as authoritative and never reconnects on a
  per-key miss, so a pre-change raw-keyed Redis entry would make a
  special-character server's tools vanish for up to the cache TTL.
  getMCPServerTools now heals legacy raw-keyed entries to the
  normalized format at read time (keys and function names), covering
  every consumer with no coordinated invalidation; safe names return
  the map untouched.

- Skill allowed-tools: a skill declaring a raw MCP key in
  allowed-tools bypassed the initialize-boundary heal (the union runs
  after it) and would neither dedupe against healed agent tools nor
  match the normalized tool map. The primes' allowedTools now pass
  through the same normalizeAgentToolKeys heal before unioning.

- Builder selectors: matchesMcpServer and useVisibleTools parsed tool
  ids against raw server names only, so an attached special-character
  server rendered as an unselected orphan card. Both now accept the
  normalized spelling and resolve it back to the raw map key, keeping
  legacy raw ids working.

* 🧯 fix: Fail Closed on Normalized Server-Name Collisions

Escalation of the collision finding: a deterministic first-wins alias
plus a warning still let the tools listing publish BOTH colliding
servers, so a tool selected under the shadowed second server would
silently execute against the first server's configuration (their
model-facing keys are identical, so routing cannot ever distinguish
them).

- findShadowedServerNames identifies later-configured names whose
  normalized form an earlier different name claimed.
- getMCPTools excludes shadowed servers from the published listing
  entirely (with a warn naming the collision), so their tools are
  never selectable - nothing ambiguous can be picked.
- Server creation reserves both spellings: a generated slug may not
  collide with a raw config name OR the normalized form its tool keys
  would carry.

Collision-resistant model-facing IDs remain out of scope: changing
normalizeServerName's output would rewrite every existing tool key
(agent documents, caches, instance names) for ALL servers to handle a
misconfiguration that is now blocked from exposure instead.

* ✅ fix: Dedupe Reserved Server-Name Spellings at Creation

The reservation list appended normalized forms unconditionally, which
duplicated every safe name (raw === normalized) and broke the
route-level contract test pinning the exact list. Dedupe via a Set so
safe names contribute one entry, while special-character names still
reserve both spellings; adds the special-character reservation case.

* 🧯 fix: Never Heal a Shadowed Server's Keys; Align Authorization Tie-Break

Persisted references were the remaining collision vector: an agent or
skill saved with the shadowed later server's raw key was HEALED into
the shared normalized key, authorized through a last-wins map, and
routed first-wins - authorized as one server, executed as another.

- normalizeAgentToolKeys now refuses to rewrite keys of shadowed
  servers (findShadowedServerNames): rewriting would produce exactly
  the first server's key. Left raw, the key cannot match the
  normalized-keyed tool map and the tool fails visibly - broken beats
  misrouted. Covers agent.tools, tool_options, and skill
  allowed-tools through the shared heal.

- filterAuthorizedTools (agents/v1.js) builds its normalized-to-raw
  map via the shared buildServerNameAliases instead of a last-wins
  Map constructor, so authorization resolves a colliding key to the
  SAME first server execution routes to.

* 🧯 fix: Direct Identity Wins Over Aliases; Heal Client Forms and Degraded Contexts

Four review findings on the normalization edges:

- Alias hijack (P1): a user-DB server named exactly like an operator
  server's normalized form ('foo' vs YAML 'foo!') had its tools
  rerouted to the operator server by unconditional alias resolution.
  Resolution is now DIRECT-FIRST everywhere: the parsed name is tried
  as-is, and only when nothing resolves is it treated as a normalized
  spelling (definitions loader, handleTools grouping, createMCPTool
  fallback). buildServerNameAliases seats identity entries before
  derived ones so a literal name owns its slot regardless of config
  order, findShadowedServerNames and the collision warning derive from
  the same construction, and getUserMCPAuthMap fetches auth under both
  spellings so either owner finds its rows.

- Builder double-match: a normalized name containing the delimiter
  ('foo mcp bar' -> 'foo_mcp_bar') also suffix-matched a server named
  'bar', selecting both cards and making removal strip the wrong tool.
  matchesMcpServer now resolves the token ONCE against the full
  configured list (longest boundary, both spellings) when the caller
  supplies it; selection and removal share the resolution.

- Builder legacy ids: an agent saved with raw-keyed ids showed its
  tools unchecked while the runtime heal kept them active, and
  selection updates never replaced the legacy entries. McpSection maps
  legacy raw ids to their current normalized ids when deriving and
  rewriting this server's selection.

- Degraded context: a transient ensureConfigServers failure returned
  an entirely empty context, leaving normalized keys unresolvable for
  the request. resolveMCPServerContext now keeps the name lists (they
  derive from the config snapshot alone) and degrades only the
  lazy-init configs.

* 🧯 fix: Collision Detection Sees Accessible Servers; Shadowed Refs Fail Closed End to End

Round follow-ups on the collision design, all in the
DB-server-visibility class:

- The legacy-key heal detected collisions against operator-config
  names only, so healing could still produce a key that direct-first
  resolution routes to an invisible user-DB server. initializeAgent
  gains an optional getAccessibleMcpServerNames dep (wired through
  ToolService for controllers that mock it, directly elsewhere),
  consulted ONLY when a configured name needs normalization - zero
  cost for safe-name deployments. The heal then sees the full
  accessible set and skips shadowed servers' keys.

- Wildcard and legacy raw tokens bypassed catalog filtering, letting a
  shadowed server's instances join a run under the same normalized
  names as the winner's. filterAuthorizedTools rejects tools of
  shadowed servers at authorization (its merged map sees DB + config),
  and handleTools skips them at execution.

- The builder migrated only tool selection, not tool_options: legacy
  raw option keys showed disabled while the runtime honored them, and
  toggles could not clear them. McpSection now migrates option keys to
  the current normalized ids (existing normalized entries win).

- A transient ensureConfigServers failure degraded to an EMPTY server
  context, leaving normalized keys unresolvable for the request.
  resolveMCPServerContext keeps the name lists (derived from the
  config snapshot alone) and degrades only the lazy-init configs.

* 🧯 fix: Complete the Collision Audit at Every Gate; Safer Heal Semantics

Round follow-ups hardening the collision audit:

- Execution guards now consult the FULL accessible set: the caller's
  heal threads its already-fetched names through loadTools, and
  handleTools fetches them itself when a configured name needs
  normalization (never for safe-name deployments) - so a cross-tier
  collision (user-DB 'foo' vs operator 'foo!') fails closed at eager
  execution instead of joining the run under one normalized name.

- Healing is SKIPPED when the collision audit cannot complete
  (transient lookup failure, or no dep): un-healed raw keys still
  resolve through the direct-first candidates, so skipping is safe
  while rewriting against an incomplete audit is not.

- The audit lookup is gated on the agent actually carrying
  delimiter-bearing keys (tools, tool_options, or skill
  allowed-tools), so non-MCP agents never pay a registry round-trip
  even on specially named deployments.

- normalizeAgentToolKeys gives the CURRENT (normalized) entry
  precedence when both spellings carry options, matching the builder's
  migration semantics instead of letting insertion order decide.

- The builder's toCurrentToolId resolves entries boundary-exactly
  against every configured server (longest match, both spellings), so
  a raw suffix shared with a LONGER server name can no longer reassign
  that server's selection or options while another dialog is open.

* 🧯 fix: Shared Collision Audit for Definitions Loading; Fail Closed on Audit Failure

Round follow-ups closing the remaining audit gaps:

- The definitions-only loader now consumes the same collision audit as
  eager loading: shadowed servers' entries (wildcards included) are
  dropped before definitions are emitted, so the default execution
  path can never resolve a shadowed server's normalized function name
  to another server. The audit names thread from initializeAgent's
  heal; the loader self-fetches only when a configured name needs
  normalization.

- resolveCollisionAuditNames centralizes the audit-resolution policy
  (threaded set > self-fetch when needed > incomplete on failure), and
  BOTH loaders now fail closed under an incomplete audit: any
  normalization-sensitive reference (its own name needs normalizing,
  or it equals the normalized form of a configured special-character
  name) is skipped with a warning instead of being audited against
  operator names alone. isNormalizationSensitiveName lives in
  packages/api as a pure helper so test mocks use the real predicate.

- normalizeAgentToolKeys collapses duplicate ids after healing
  (order-preserving): a document carrying both spellings converges on
  one key, never two instances with the same function name.

* 🧯 fix: Thread the Audit Everywhere; Identity-Aware Alias Fallback

Round follow-ups on audit plumbing:

- The OpenAI-compatible and Responses tool loaders now forward the
  already-resolved accessibleMcpServerNames instead of discarding it,
  so the definitions loader neither repeats the registry lookup nor
  fails closed on a transient second lookup after the first succeeded.

- The skill-only path threads its audit: when the baseline agent has
  no MCP keys but a primed skill's allowed-tools fetched the complete
  set, that set (not the operator-only list) reaches the loader, so
  the collision remains visible and the shadowed reference stays
  rejected end to end.

- OAuth discovery iterates the collision-FILTERED tool list, so a
  request can no longer emit an OAuth prompt, wait out the connection
  timeout, and reconnect a server whose definitions were deliberately
  rejected.

- The definitions loader's alias fallback is identity-aware: when the
  parsed name IS a known accessible server, a null tool fetch means
  temporarily unavailable (OAuth pending, missing user variables,
  disconnected) and no longer reroutes to the raw alias - previously
  the aliased operator server's definitions could be emitted under the
  unavailable DB server's names.

* 🧯 fix: Legacy-Key Definition Lookup; Retain Audit for Deferred Execution

- createMCPTool resolves tool definitions by BOTH spellings: the key as
  persisted plus the canonical normalized key built from the resolved
  server name. Assistants and direct tool calls persisted before the
  rollout bypass the agent-boundary heal and arrive with raw keys, while
  availableTools is now indexed canonically - previously every such call
  missed the index, burned a reconnect, and returned the unavailable
  stub permanently via the negative cache.

- The initialized agent retains accessibleMcpServerNames (the COMPLETE
  collision audit this initialization resolved), buildAgentToolContext
  copies it into every per-agent tool context, and loadToolsForExecution
  threads it into the eager loader as bare options. Deferred/event-driven
  execution therefore reuses the snapshot instead of repeating the merged
  registry read - a transient failure there could fail-closed a tool the
  same turn already advertised from the successful first audit.

- MCP.spec.js keeps @librechat/api pure helpers REAL (requireActual
  spread) so normalization paths are exercised rather than mirrored.

* 🧯 fix: Parse Legacy Keys Against Both Server-Name Spellings

createMCPTool's boundary candidates were normalized-only, so a legacy
raw key whose server name contains the delimiter (foo_mcp_bar!) missed
the suffix match and fell to the generic last-delimiter split - the
canonical rebuild then produced a key that could never hit the index
and the persisted call stubbed out. The candidate list now carries the
RAW resolved name (and raw config names on the parse-only path) next
to the normalized spellings.

* 🧯 fix: Honest Audit Completeness; Shadowed-Server Form-Key Guard

- resolveAllMcpConfigs tolerates ensureConfigServers failures, so the
  merged registry read can silently omit config-only servers while the
  audit still reported complete: true - a foo/foo! collision would go
  unseen and a persisted key could route to the wrong server. Both
  audit consumers now union the snapshot-derived raw config names back
  in (resolveCollisionAuditNames unions the caller's rawServerNames;
  the initializeAgent heal unions configRawServerNames), keeping the
  completeness label honest without an extra read: operator names come
  from the registry-independent config snapshot, user-DB names from the
  merged read that fails loudly into the existing incomplete path.

- The client tool_options migration now mirrors the runtime heal's
  fail-closed rule for SHADOWED servers: when the dialog's server has
  lost its normalized slot to another catalog name, legacy raw keys
  stay raw instead of being rewritten onto the winning server's key,
  where a later save would apply the wrong server's per-tool settings.
  The dialog's own server joins the alias construction so a stale
  catalog map can't misread as a collision.

* 🧯 fix: Heal Legacy Assistant MCP Tool Names on Save

The assistants create/update controllers look tools up in the cached
definitions by exact key, and the cache is now normalized-keyed - an
assistant saved before the convention resubmits its raw-suffixed MCP
name on every edit, so any save silently removed the tool.

healMcpToolNames pre-heals the payload's tool list: a delimiter-bearing
string that misses the cache resolves through the configured raw names
(longest-suffix, boundary-exact) and rewrites to the normalized key
only when that key actually exists in the cache. SHADOWED raw names
stay raw and fail closed, mirroring the runtime heal; the config read
happens only when a delimiter-bearing name actually misses, and read
failures propagate (write path) rather than silently dropping tools.
v2's update loop also stops re-reading the tool cache per iteration.

* 🧯 fix: Full-Audit Shadow Set + Dedupe in the Assistant Key Heal

- The assistant-save heal built its shadow set from operator config
  names alone, so a cross-tier collision (user-DB `foo` owning the
  normalized slot of operator `foo!`) looked unshadowed and the legacy
  key healed into the shared normalized name - which direct-first
  execution then binds to the DB server. The shadow set now comes from
  resolveCollisionAuditNames' full accessible audit, and an incomplete
  audit skips healing outright (every rewrite candidate is
  normalization-sensitive by construction, so raw-and-fail-closed is
  the only safe answer).

- Healed string entries dedupe order-preserving: a payload carrying
  both spellings of the same tool collapses to one entry instead of
  expanding into duplicate function definitions the provider rejects.
2026-08-01 07:39:24 -04:00
Danny Avila
e7f1838515
⚡ feat: Reliable Interrupt & Steer Escalation and Recovery (#14558)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* feat: surface interrupt-steer escalation on waiting messages

The interrupt & steer feature shipped reachable only through the
composer chord, the send-button hovercard, and the composer button; a
message already waiting (queued for after the run, or steered and
parked at the next tool boundary) had no path to it. Both waiting
surfaces now carry one:

- Queued rows get an icon-only ZapOff escalation button beside the
  existing Steer primary. It routes through sendQueuedNow, which now
  takes a preempt option on its live-run path. The tooltip teaches the
  composer chord, derived through resolveComposerKeyDown so a rebound
  or yielded chord is never advertised.
- In-flight steer bubbles get an "Interrupt now" overflow entry with
  the same race rules as Edit: reclaim first, and only a `reclaimed`
  outcome resubmits (via retrySteer with preempt, swapping the chip
  for an interrupting one). `applied` and run-ended-mid-reclaim
  outcomes stop at the existing informational toasts, so the words can
  never land twice. Not offered on a steer already preempting.
- Every during-run overflow menu gains an "Always interrupt instead"
  toggle for steerInterruptsByDefault, next to the existing steer/queue
  default toggle. MenuEntry supports disabled for the new entries.

Only one interrupt can be unresolved at a time: while one preempt is
pending (or the run is paused on approval, where the server 409s),
every escalation control disables instead of racing the same seal.

Ten new tests across both surfaces; 381 green in the affected suites.

* fix: lock escalation across its reclaim window, keep the paused control visible, label as steer

Codex round 1, all three findings.

P2, escalation race. The single-interrupt invariant had a window between
clicking "Interrupt now" and the reclaim resolving, where no preempt
chip existed for the chip-derived gate to see: two bubbles escalated
back-to-back could both resubmit. A shared escalating flag (Jotai,
per-conversation) now covers the window and disables every escalation
control on both surfaces, and a fresh recheck before resubmitting
catches an interrupt armed elsewhere meanwhile (composer chord, queued
row); those words re-home to the queue with an informational toast
instead of breaking the invariant.

P2, unreachable paused state. canSteer is defined as
hasRealConvoId && !pausedOnApproval, so gating the button on canSteer
removed it exactly when it was meant to render disabled; the test only
passed on an impossible stub combination. The render gate is now
duringRunActive && (canSteer || pausedOnApproval), and the test uses the
real invariant.

P2, label semantics. "Interrupt & send now" borrowed the name of the
hard-abort action; this one preserves the partial answer and steers.
Renamed to "Interrupt & steer now" (com_ui_interrupt_steer_now).

Both behavior fixes counterfactually verified; 384 tests green across
the affected suites.

* fix: disable bubble escalation while the run cannot accept a steer

Codex round 2, one P2. Answer mode (ask_user_question) sets
duringRunActive false while pausedOnApproval stays false, since that
flag only detects approval-bearing tool calls. The bubble's escalation
entry stayed enabled there, so clicking it cancelled a healthy waiting
steer and the preempt resubmission bounced off RUN_PAUSED, degrading
the words to the queue. The entry now also disables on
!duringRunActive, matching the queued-row control's gate.

Counterfactually verified: reverting the gate fails the new
answer-mode test.

* fix: recheck live run state after the reclaim, not just at the click

Codex round 3, one P2, and it is the round-1 recheck principle applied
one level deeper: the entry-time disable cannot see a run that pauses
(tool approval, answer mode) while the reclaim round-trip is in flight,
and the .then closure held the render's stale steering controls, so the
resubmit would fire into a RUN_PAUSED rejection after the reclaim had
already surrendered the steer's boundary slot.

The escalation continuation now reads the LIVE controls through a
latest-ref: if the run can no longer accept a steer, the words re-home
to the queue with an informational toast instead of resubmitting, and
the resubmit itself also goes through the live controls.

Counterfactually verified: reading the stale closure instead of the ref
fails the new mid-reclaim pause test.

* refactor: make escalation one atomic server-side arm, in place

Codex round 4: four P2s, every one an interleaving of the same window —
escalation as reclaim-then-repost is a compound, non-atomic operation
whose continuation must revalidate the world (FIFO position lost, ref
assigned too late, no run fence, competing bubble actions). Rounds 1-3
patched that window with a lock and rechecks; round 4 shows the window
itself is the defect, so this removes it instead of guarding it again.

Escalation is now POST /chat/steer/arm: the server flips preempt on the
EXISTING queued item in one atomic store op (new IJobStore.armSteer; a
decode-patch-encode LSET Lua on Redis, an in-place mutation in memory),
fenced to the validated generation and refused once the queue closes.
The handler mirrors the steer POST's preempt contract exactly: durable
flag gated on the owner's recorded capability, volatile requestPreempt
fire-and-forget because the durable flag is the truth resume/handover
re-arm from.

By construction this resolves all four findings: FIFO survives (the
item never moves; the whole queue still drains in instruction order at
the seal), there is no continuation to hold stale controls, the store
op is fenced to the original run, and a competing Edit/Queue/Cancel
either beats the arm (armed:false, chip untouched) or operates on the
armed item, whose cancel already disarms.

The client escalation entry becomes one mutation: armed:true relabels
the chip in place (same steerId, same position), PREEMPT_UNSUPPORTED
and lost races toast honestly, and the round 1-3 machinery — the
escalating lock atom, the latest-ref, the post-reclaim rechecks and
their two toast strings — is deleted rather than extended.

Verified: 7 new handler tests on the real in-memory manager (including
FIFO preservation and the stale-generation fence), 2 Redis integration
tests against real Redis (in-place arm keeps order and every field;
missing/stale/closed all refuse), client suites 396 green.

* fix: decide capability inside the atomic arm, neutralize the lost-race toast

Codex round 5, both findings, both edges of the new arm design rather
than its mechanism.

P2, capability TOCTOU. A HITL resume on a rolling deploy rewrites
preemptCapable for the SAME generation, so the handler's read could go
stale between validation and the flag flip, arming a steer the live
owner cannot seal. armSteer now returns armed | missing | incapable,
with the owner's live capability part of the same atomic predicate as
the generation fence (HGET preemptCapable inside the Lua; the flat job
field, not a metadata blob — the in-memory store reads the same field).
The handler's pre-check is deleted rather than kept alongside; the
store predicate is the single source. New handler test rewrites the
capability after queueing and expects PREEMPT_UNSUPPORTED with the item
left unflagged; the Redis guards test now asserts the incapable refusal
against real Redis.

P2, ambiguous toast. armed:false covers injected, cancelled, re-homed,
and run-over alike, so telling the user the message "already reached
the agent" claimed one specific outcome. The lost-race branch now uses
a neutral message (com_ui_steer_arm_lost_race) and defers to the events
for what actually happened.

* fix: flip the escalation lock synchronously before the arm request

Codex round 6, one P2. Round 4 deleted the escalating flag along with
the reclaim continuation it guarded, but that left the one-interrupt
gate blind during the arm request's own round trip: the chip-derived
check cannot see an arm until its response relabels the chip, so on a
slow connection two bubbles could both arm before either response
landed. Double-arm is harmless server-side now (the run seals once and
drains the whole queue in order), but every escalation control
advertises "one interrupt at a time" by disabling, and the controls
must tell the truth.

The per-conversation escalating flag returns as a pure UX gate: set
synchronously at click, before the mutation, cleared on settlement, and
folded into interruptPending on both surfaces. Unlike its round 1-3
ancestor there is no continuation behind it to guard and no recheck to
pair with it.

Counterfactually verified: without the synchronous set, the two-bubble
race test arms twice. 207 tests green across the Chat Input suites.

* test(e2e): cover escalation of waiting messages through the real seal

Three mock-harness tests on E2E_SLOW_REPLY, a 160-chunk stream with no
tool boundary, so an in-thread steer part can ONLY come from a genuine
mid-stream seal — which makes each test a behavioral proof rather than
a UI check:

- Queued row escalation: the ZapOff button turns a waiting queued
  message into a preempt-armed steer (202 echoes preempt: true) that
  seals and injects, where the sibling steering.spec test proves the
  unescalated path waits for run end instead.
- Bubble in-place arm: an ordinary steer (202 with no preempt echo)
  waits as a bubble, POST /chat/steer/arm answers armed: true, the
  bubble relabels in place (same single bubble, same text, escalation
  no longer offered on reopen), and the armed steer seals mid-stream.
- Always-interrupt toggle: flipped from a waiting row's overflow menu,
  plain Enter now produces a preempt: true steer that seals in the SAME
  run, and the menu offers the way back. An afterEach clears the
  localStorage preference so a mid-test failure cannot leak
  preempt-by-default into the rest of the serial suite.

All three verified locally through the full harness (real backend, mock
LLM, seeded DB): 3 passed in 27s.

* feat: dedicated escalation arrow + shortcut, menu split into actions and preferences

The escalation was still half-hidden: the bubble only offered it inside
the overflow menu, and the tooltip taught the composer chord, which does
a different thing (interrupts with typed text, not this chip). Three
changes make it a first-class command:

- A shared EscalateNowButton (circular arrow, ghost-bordered like the
  composer's interrupt control) is always visible on BOTH surfaces:
  beside each queued row's Steer primary and on every waiting steer
  bubble next to its menu. It disappears once a steer is interrupting.
- A dedicated registry shortcut, escalateSteer (Cmd/Ctrl+Shift+.),
  editing-allowed and rebindable like every other action. Deliberately
  NOT an Enter chord: the composer owns every Enter chord, and the
  yield design rests on no default binding using Enter besides submit.
  Its handler clicks the newest enabled arrow control (bubbles beat
  queued rows), so the shortcut can never diverge from the button, and
  the arrow's tooltip teaches THIS command via the registry display.
- The overflow menus separate one-off actions from sticky behavior
  changes: Edit, Cancel, Queue, then a smaller "Preferences" section
  holding the queueing and always-interrupt toggles, each with the
  standard InfoHoverCard reusing the Settings panel's descriptions.
  "Interrupt & steer now" leaves the menu entirely.

386 client tests green, including a menu-structure test locking the
order and the absence of the escalation entry; bubble escalation tests
drive the visible arrow. The e2e spec's bubble test now clicks the
arrow, and a fourth test drives the dedicated shortcut end to end
through a real mid-stream seal.

* style: bind the escalation arrow to its message (variant A anatomy)

Two same-weight circles in a row read as one control group, leaving the
arrow's ownership ambiguous, and a floating arrow stops meaning anything
once several messages stack. The shared control now carries variant A's
anatomy: a thin divider binds a small SOLID arrow (filled, inverted) to
the message region on its left, and the menu ellipsis stays a bare
glyph, so the two affordances can no longer blur together — and the
divider+arrow pairing repeats cleanly per chip at N messages.

* chore: drop the unused within import CI lint caught

* fix: advertise the escalation shortcut only while the control is live

Codex on the e2e head, one P2: the tooltip appended the chord hint even
while the button was disabled, advertising a shortcut that does nothing
during an approval pause. The flagged control (InterruptNowButton) was
since replaced by the shared EscalateNowButton, which inherited the
pattern; the successor now omits the chord whenever the control is
disabled, matching the rule the during-run hovercard already follows.

* fix: harden steer escalation lifecycle and recovery

* test(e2e): disambiguate accessible steer preferences

* test: align abort persistence coverage with prerequisites

* chore(i18n): remove obsolete steer race message

* chore: normalize imports across steering changes

* test: exercise stream integration on Redis Cluster

* test: scope HITL checkpoints to generation

* test: fix cluster cleanup and locale policy

* fix: keep escalation visible during ask pauses

* fix: fence recovery downgrade and stale predecessors

* fix: require generation owner abort acknowledgement

* fix: validate delayed preempt arms

* test: align final escalation fixtures

* fix: preserve in-memory predecessor abort handoff

* fix: restore controls for recovered queued messages

* test: cover recovered queue controls

* fix: close final steering review gaps
2026-07-31 20:07:56 -04:00
Danny Avila
60ca751a7f
🧠 fix: Preserve Deferred Tool Schemas Across HITL Resume (#14552)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* 🧠 fix: Preserve deferred tool schemas across HITL resume

* 🧪 test: Harden deferred tool resume regression

* 📦 chore: bump @librechat/agents to v3.3.10
2026-07-31 14:06:13 -04:00