mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 14:08:21 +00:00
54 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
bd9108a87a
|
🧯 ci: Stop Diagnostic Artifact Uploads From Failing or Stalling Test Jobs (#18068)
On 2026-11-11 the artifact service answered every request with an HTML page. In Playwright run 43092661470 eight memory shards failed in "Upload Playwright HTML report" after retrying for 9 to 10.5 minutes; in six of them every test had passed. Uploads that no job downloads (Playwright reports, traces and screenshots, the Lighthouse report, the advisory quality reports) are now `continue-on-error` with a three minute step timeout, so a failed upload stays out of the job result and a stalled one cannot run the job into its own timeout. The Jest result uploads were already `continue-on-error` and gain the same bound. Build outputs and mutation shards that a later job downloads stay fatal. merge-gate.test.mjs holds the gated workflows to that split, and workflow.test.mts holds Change Quality to it. |
||
|
|
d434417089
|
🦠 chore: Upgrade Node.js and Undici to Security Releases (#18066)
* 🦠 chore: Upgrade Node.js and Undici to Security Releases
* test: Wait for Restored Composer Readiness and Reuse Wrapping Config
---------
Co-authored-by: Lia <lia@librechat.ai>
|
||
|
|
9f0646c00d
|
🚦 ci: Make Dev Workflows Ready for a Merge Queue (#18054)
* 🚦 ci: Make Dev Workflows Ready for a Merge Queue * ♻️ ci: Evaluate Merge Gate Expressions With the Shared Workflow Evaluator * 🧺 ci: Share the Installed Tree in Merge Gate Tests the Way ci-setup Requires |
||
|
|
d32403f851
|
🍃 ci: Provision the mongod Binary Before Any Test Starts (#18053)
Some checks failed
Codegraph E2E Vote Gate / Gate (push) Waiting to run
Codegraph E2E Votes / vote (shard 1/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 10/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 2/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 3/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 4/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 5/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 6/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 7/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 8/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 9/10) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Playwright E2E Tests / Codegraph select (push) Waiting to run
Playwright E2E Tests / e2e () (push) Blocked by required conditions
Playwright E2E Tests / MCP list_changed (replica count 1) (push) Blocked by required conditions
Playwright E2E Tests / MCP list_changed (replica count 2) (push) Blocked by required conditions
Playwright E2E Tests / MCP Apps (browser and server) (push) Waiting to run
Playwright E2E Tests / e2e (push) Blocked by required conditions
Main Promotion Policy Tests / Main promotion denial and fast-forward tests (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
* 🍃 ci: Provision the mongod Binary Before Tests Start mongodb-memory-server downloads its ~122 MB mongod the first time a test starts a server, inside a 15 s beforeAll. Only backend-review.yml cached ~/.cache/mongodb-binaries (the same step pasted four times), and a cache alone still leaves the download inside the first hook on a miss. Integration Tests downloaded on every run: 6 failures in 274 runs with "Exceeded timeout of 15000 ms for a hook" in secrets.integration.spec.ts, followed by "spawn ETXTBSY" in the suites that started mongod while the timed-out download was still writing it. Add .github/actions/mongodb-binary: restore the cache with the existing key scheme, ask the installed mongodb-memory-server for the binary it wants (MongoBinary.getPath), make sure that binary sits in the home cache every working directory resolves, run it once, and save the cache right away. The step fails the job when no runnable binary can be put in place. Use it in every job that starts mongodb-memory-server, after dependencies are installed and before tests, and add the action to the path filters of the workflows that list their own file. * 🧪 test: Cover the mongod Provisioning Script Without the Network Run the real script against a stand-in resolver that keeps the properties it relies on: home cache first, download when missing, and a memoised path until the cache is cleared. Integration Tests runs the suite before it uses the action. * 🧪 test: Run mongod Provisioning Against the Real Resolver The suite replaced mongodb-memory-server with a hand-written stand-in, so it checked the script against a description of the resolver rather than the installed one. It now runs the script against the real library and replaces only the network: MONGOMS_DOWNLOAD_MIRROR points at a local server that serves an archive whose bin/mongod is a shell script, with the .md5 the library verifies it against. Recovery also no longer charges a broken copy that was already on disk to the download: with a broken binary in both the home cache and the root node_modules/.cache, the first attempt removed one, the retry met the other, and the step failed without ever downloading. Existing copies are now discarded one at a time and the download keeps its own two attempts. * 🧪 test: Reach the Exit-Status Check Behind the Library's System Binary Probe A system binary that prints nothing is rejected by the library's own version probe, so the case never reached this script's check or the rule that a file the job named is not deleted. Add one that answers the probe and exits non-zero. |
||
|
|
97f7fcdbac
|
🥅 ci: Run Playwright After Merges to Dev and Guard Specs Under Test Selection (#18049)
* 🥅 ci: Run Playwright After Every Merge to Dev The mock e2e suite ran on pull requests, nightly on main and on demand, so when a spec broke on dev no run on dev could go red: the vote workflow that runs the suite after merges cannot fail by design. Add a push trigger for dev with the pull request path filter. A push runs the pull request profile with no lane narrowed (10 memory shards, 2 Redis transport shards, both MCP lanes and MCP Apps), and the gate job reports it. Pushes take their own concurrency group that never cancels: the run in progress finishes, and GitHub keeps one pending run per group, so merges that land meanwhile collapse into one run of the newest commit. Pull request, nightly and manual runs keep their per-ref cancelling group. The matrix test now evaluates the workflow's own expressions per event rather than matching their text. * 🔭 ci: Always Run Source-Scanning Guard Specs Under Test Selection Codegraph test selection narrows a pull request run to the test files an import graph reaches. A spec that reads source with fs has no import edge to what it reads: toolCapabilityGates.spec.js scans four source trees, was not selected when #17864 changed one of the files it counts, and failed on dev afterwards. List every such spec, with the paths it reads, in .github/guard-specs.json. Both selectors now add a workspace's guards to a FILES decision and turn a NONE into a guards-only run when the change touches a path one of them reads, renames a file, or edits the list. A list that cannot be read runs the workspace FULL. FULL decisions and every earlier fail-open path are unchanged. The rule keys off the decision, not the event. A selection with no file left in the checkout gets no guards, so the test job still runs FULL on a stale selection instead of running only the guards. guard-specs.test.mjs fails when a spec in a gated workspace reads the repository through fs without being listed, when a listed path is missing, and when a workflow would not run for a path one of its guards reads. That last check adds the five backend and one frontend paths the triggers lacked. The test also runs both real select steps against canned codegraph responses. |
||
|
|
af3840b5fc
|
🥐 chore: Standardize Bun-First Workspace Typechecks (#17651)
* 🥐 chore: Standardize Bun-First Workspace Typechecks
* fix: Preserve Bun 1.4.3 Checker Compatibility
* chore: Pin Bun 1.4.4 for Local and CI Typechecks
---------
Co-authored-by: Lia <lia@librechat.ai>
|
||
|
|
53ae22a6a3
|
💉 chore: Upgrade Node.js to Security Release 24.22.1 (#17332)
Some checks are pending
Publish `@librechat/client` to NPM / pack (push) Waiting to run
Publish `@librechat/client` to NPM / publish-npm (push) Blocked by required conditions
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* 💉 chore: Upgrade Node.js to Security Release 24.22.1
* test: Enable Enter Submission in Native Acceptance
* test: Use the Machine Picker in Native Acceptance
---------
Co-authored-by: Lia <lia@librechat.ai>
|
||
|
|
e1b4e09d7c
|
🥟 ci: Type-Check Workspaces with bun check (#17009)
Some checks failed
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Main Promotion Policy Tests / Main promotion denial and fast-forward tests (push) Has been cancelled
* ci: Type-Check Workspaces with bun check Make every workspace tsconfig valid under TypeScript 7 semantics while staying clean on the installed tsc 5.9: explicit types arrays (TS 7 no longer auto-includes node_modules/@types), es2017 target for data-provider, rootDir for api specs that import client contracts, and noUncheckedSideEffectImports off for the CSS and matchMedia mock imports. Fix the four client type errors newer DOM and Intl libs expose, and drop the fetch.preconnect copy in useRum that only existed to satisfy the leaked @types/bun globals. Switch the client typecheck script and the CI type-check steps to bun check (Bun 1.4.3); tsc --noEmit stays available as typecheck:tsc. * chore: Fall Back to tsc When bun check Is Unavailable Add scripts/typecheck.mts, which runs bun check when the installed Bun provides it and the repository's tsc otherwise, so contributors without Bun can still type-check. The client typecheck script and the static-checks TypeScript gate both resolve their checker through it, keeping local runs on the same checker CI uses whenever Bun is present. * ci: Type-Check the Chat Package Through the Shared Checker * ci: Run the Type Check When Only the Shared Checker Changes A change to scripts/typecheck.mts alone started neither frontend review nor the static-checks TypeScript gate, so a broken resolver could merge unexercised. Add it to the frontend-review triggers and to every typecheck project's paths. The suppression-baseline scenarios copy static-checks.mts into synthetic roots; copy its new typecheck.mts import alongside i18n.mts so the runner loads there. Add scenarios for the checker choice and its fallback. |
||
|
|
c60dac37d0
|
🏰 ci: Gate Main Promotions on Approved Dev Commits (#16570)
Some checks failed
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Main Promotion Policy Tests / Main promotion denial and fast-forward tests (push) Has been cancelled
Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
ba44443fdb
|
📒 ci: Upload Per-Test Jest Results From Review Workflows (#16110) | ||
|
|
44af4d3789
|
📖 fix: Verify Agents OpenAPI contracts and disable external reporting (#16028) | ||
|
|
69f0dd2a12
|
🔌 feat: OpenAPI spec and docs for the public Agents API (#15928)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
* 🔌 feat: OpenAPI spec and docs for the public Agents API * 🔧 fix: address review on Agents API OpenAPI spec and docs: - adapter: mark JSON request bodies as required - openapi.js: move gate, config read and spec serving into packages/api as an injected router factory; leave /api as wiring only - docs: compute URLs in-browser and use a relative servers url so docs work under a base path (e.g. /chat) - build: copy the generated spec into dist so the multi-stage image ships it - agents: reuse the enforced list envelope for the docs so the AgentList schema cannot drift from the runtime cursor and strictness * 🔧 fix: align Agents API OpenAPI contract with enforced runtime Second review round on the public Agents API spec. Three cases where the published contract described something the server does not do, so generated clients would fail against the live API: - Upload form: document the `purpose` field (enum: file_search, execute_code, context) that POST /agents/{id}/files actually requires, instead of the internal-only `tool_resource`; requests following the old form were rejected with 400 because `purpose` was absent. - Auth: drop the `apiKeyBearer` scheme from the management and skill endpoints. They gate on OIDC/M2M via requireAgentManagementAuth and deliberately reject the API-key fallback; the API key belongs to the out-of-scope OpenAI-compatible endpoints and returns when those are documented. - Skill update: document the 409 conflict PATCH /skills/{id} returns on a stale expectedVersion. The agent update path collapses its internal 409 to 400, so it correctly keeps no 409. * 🔧 fix: serve docs on experimental server and document the 401 Third review round on the public Agents API spec. - Experimental server: mount routes.openapi in api/server/experimental.js before its /api 404 handler. Without it, npm run backend:experimental returns 404 for /api/docs and /api/openapi.json even when openapi.enabled is true, because that server never mounted the router. - Auth failure: document the 401 that requireAgentManagementAuth returns on a missing, expired, malformed, or unbound token. Its body is { "error": "Unauthorized" }, a flatter shape than the management error schema, so add a dedicated UnauthorizedError schema and a shared 401 response to the agent and skill contracts rather than reusing the standard error schema. * 🔧 fix: document skill-update, auth-conflict, upload-limit, and watch-copy gaps * 🔧 fix: added error shapes for permission denied and internal server errors in API responses docs * fix: cover the auth-layer 500 shape and pin the account-deletion code * fix: update OpenApiRouterDeps documentation and simplify request handling in isEnabled function * 🔧 fix: gate the Swagger asset mount behind the OpenAPI toggle * 🔧 fix: accept the malformed-JSON 400 envelope in the contract * fix: accept the tenant-isolation 403 envelope in the contract * fix: add description for content property in SkillFileUpdateRequest to clarify byte limit |
||
|
|
345c7aea7d
|
🎨 ci: Gate Frontend Jest on Codegraph Selection (Stage 1.5) (#15145)
* 🎨 ci: Gate Frontend Jest on Codegraph Selection (Stage 1.5)
* ci: a malformed FILES decision runs FULL, never skips (Codex)
* ci: dev-push runs never cancel each other (Codex P2)
* ci: workflow-file push baseline, cancellable gated jobs, pull-requests read (Codex r4)
* ci: selected paths must live under their workspace, else FULL (Codex r5)
* ci: drop stale selected paths, run FULL when none exist (Codex r6)
|
||
|
|
f7b65cb7ae
|
🎭 ci: Gate Playwright Lanes and Docker Smokes on Codegraph Selection (Stage 2) (#15136)
* 🎭 ci: Gate Playwright Lanes and Docker Smokes on Codegraph Selection (Stage 2)
* ci: surface the fail-open reason in the stage-2 select summaries
* ci: log the stage-2 select decision for harvesting
* ci: fail open on fetch failure or truncated file list; type-strict skip decisions (Codex)
* ci: check curl's exit status before honoring a selection (Codex r2)
|
||
|
|
e9a5b61f8c
|
⛩️ ci: Gate Backend Jest on Codegraph Selection (Stage 1) (#15132) | ||
|
|
719b04f389
|
🍃 ci: Cache MongoDB Memory-Server Binaries in Backend Test Jobs (#15131) | ||
|
|
dfd4d9dd81
|
🧩 ci: Close Workflow Path-Filter Gaps (#14728)
* 🧩 ci: Close Workflow Path-Filter Gaps Six trigger-filter gaps found by reading all workflows against the live dependency graph (AI-1755, codegraph FINDINGS §6l): - backend-review/frontend-review: root package.json/package-lock.json now trigger unit tests — a lockfile-only dependency bump previously ran zero backend or frontend unit tests while every test job installs from it - agents-integration-tests: widen to the three package src trees it builds and imports (was only src/agents/**) - cache-integration-tests: same shape — verified live that packages/api/src/flow/manager.ts (imported by mcp/oauth) matched neither integration filter - docker-smoke: plain Dockerfile had no PR-time validation despite shipping via dev-images/tag-images; new node-image-smoke job builds it, gated by paths-filter to Dockerfile/.dockerignore changes - dev-images/dev-branch-images: add config/**, skill/**, .dockerignore — the single-stage image COPYs the full build context - static-checks: eslint.config.mjs now re-triggers the lint job (gap carried over from eslint-ci.yml in the #14716 consolidation) - delete generate_embeddings.yml: fired on docs/**, which no longer exists, and its docs-root-path pointed at the same missing directory * 🧩 ci: Address Codex Review Findings - Build caches: all 26 build-* keys across 8 workflows now lead with root package.json + package-lock.json so manifest-only bumps cannot restore stale dists (data-provider embeds the root version); unifies the split key families (playwright already hashed the lockfile) - static-checks: config changes now gate on the ESLint config loading and applying to representative files, plus a report-only full-tree sweep (70 pre-existing errors at dev HEAD block a hard gate for now) - docker-smoke: the workflow file itself triggers the plain-Dockerfile build so job edits are validated - dev-images/dev-branch-images: re-include skill/**/*.md after !**.md so shipped deployment-skill Markdown rebuilds images * 🧩 ci: Gate Config Lint Sweep on Regression vs Base Config Second-round codex finding: the report-only sweep swallowed config-wide breakage in scoped blocks the representative files don't exercise. The sweep now lints the same tree under the PR's config and the base ref's config and fails only when the PR's config produces more diagnostics for some (file, rule) pair — pre-existing debt never fails the gate, and fixes are never penalized. Base-config unavailability degrades to the load gate with a notice. Outcome surfaced in the failure summary. * 🧩 ci: Harden Config Lint Gate per External Review - Coverage direction: fail when the PR config stops linting files the base config covered (set difference on linted files) — a mis-scoped ignores previously only removed diagnostics and passed both gates - Severity-aware fingerprints: (file, rule, severity) so warn->error escalations gate on a clean tree for that rule; downgrades still free - Hard-fail when the base commit is missing so a future shallow-checkout change cannot silently disable the gate; annotate fetch-depth: 0 - Tab-separated fingerprint keys (space-in-path proof), --config on both sweeps, EXIT trap for the base config copy, comment on why it must live at the repo root (flat-config pattern base paths) - Narrow skill md re-include with !skill/README.md: top-level README is documentation-only; runtime skill Markdown still rebuilds images |
||
|
|
82de3bc422
|
🚦 ci: Reduce GitHub Actions Runner Pressure (#14716)
* ci: reduce GitHub Actions runner pressure * ci: enforce static check path exclusions |
||
|
|
9e6d677751
|
⚙️ ci: Bump GitHub Actions to Node.js 24 Runtimes (#14689)
* ci: bump GitHub Actions to Node.js 24 runtimes Clear Node 20 deprecation warnings on runners by moving workflow actions to majors that declare node24 (checkout, cache, setup-node, artifacts, Docker buildx/build-push/qemu/login, github-script, setup-go, Azure login/helm, create-pull-request, axe-linter). * fix: release leader lock via ioredis on Redis Cluster @keyv/redis EVAL can surface unhandled MOVED redirects on cluster, so resign() logged failure and left LeadingServerUUID set. Use ioredis for leader election SET NX / GET / Lua (same pattern as principals and concurrency locks) so cluster redirects are retried and resignation clears the lock. |
||
|
|
ad0f72dede
|
🌀 ci: Deterministic Circular Dependency Checks (#14579)
* 🌀 ci: Deterministic Circular Dependency Checks * 🌀 ci: Enforce Type-Level Edges in Circular Dependency Scan * 🌀 ci: Materialize Import-Type Expression Edges in Cycle Scan * 🌀 ci: Collect Inline Type-Only Specifier Edges in Cycle Scan |
||
|
|
a3c92b83c8
|
🧾 ci: Skip Workflows for Markdown-Only Changes (#14378) | ||
|
|
fbc51f6e66
|
⚡ refactor: Migrate data-provider Build to tsdown (split tsc dts) (#13597)
Replace the Rollup + `rollup-plugin-typescript2` build with a split pipeline: tsdown (rolldown) bundles the JS in ~0.2s, and plain `tsc` emits the declarations to `dist/types` (~2s). Full cold build drops from ~9.2s to ~2.5s (~3.6x) with zero source changes. Unlike data-schemas, the fast oxc/isolated-declarations dts path isn't viable here: the package's 78 exported zod schemas produce 374 `isolatedDeclarations` errors (TS9013/TS9038) and a `z.ZodType<T>` annotation would break the 76 downstream `.extend`/`.shape`/`.pick` usages. Plain `tsc` keeps the rich zod types intact, and since dts was never the bottleneck (rollup-plugin-typescript2 was), the win stands. - dts stays unbundled in `dist/types/` — identical to the prior output, so the existing deep `dist/types` imports and the exports `types` paths are unchanged. - ESM output renamed `index.es.js` -> `index.mjs` (via the exports map; no consumer hardcodes the old path). cjs/types paths unchanged. - `./react-query` now emits a real cjs build + types — the exports map already promised them, but Rollup only ever built the esm file. - Kept `rollup` + the plugins used by `server-rollup.config.js` (the `rollup:api` server-bundle smoke test in backend-review.yml); removed only the deps used solely by the deleted `rollup.config.js`. - Repointed CI build-cache keys from `rollup.config.js` to `tsdown.config.mjs`. |
||
|
|
6bc75d24c8
|
⚡️ refactor: Migrate @librechat/api build to tsdown (#13595)
* ⚡️ refactor: Migrate @librechat/api build to tsdown Replace Rollup with tsdown (rolldown + oxc isolated-declarations) for the @librechat/api package build, mirroring the merged data-schemas migration. - Add tsdown.config.mjs (cjs output, oxc dts, externalize all bare deps, bundle first-party `~/` + relative imports) - Annotate exports for isolatedDeclarations (codefix-driven). Collapse the tokens.ts model->token maps to Record<string, Record<string, number>> and switch validation.ts's runtime `files` field from z.any() to z.unknown() so no explicit `any` is introduced - Repoint package.json main/types/exports to tsdown's .cjs/.d.cts output - Add src/telemetry.ts entry shim so the two index.ts entries don't collide in oxc's flat dts output (stable dist/telemetry.{cjs,d.cts}) - Delete rollup.config.js Build time ~36s -> ~0.5s. No runtime behavior change: 5712 unit tests pass, both entries load via require(), legacy /api consumes them unchanged. * 👷 ci: Hash packages/api/tsdown.config.mjs in build-api cache keys The build-api cache keys hashed `packages/api/server-rollup.config.js`, which never existed (api used `rollup.config.js`, now removed) — a copy-paste artifact from the data-provider key that matched no file. Replace it with the new `packages/api/tsdown.config.mjs` so edits to the build config (entry, format, externals) bust the api build cache, matching the data-schemas key. |
||
|
|
192703e041
|
⚡ perf: Migrate data-schemas Build to tsdown with isolatedDeclarations (#13578)
* ⚡ perf: Migrate data-schemas Build to tsdown with isolatedDeclarations Replace Rollup with tsdown (rolldown + oxc) for @librechat/data-schemas. With the source made isolatedDeclarations-clean, oxc emits .d.ts without tsc, dropping the package build from ~5.8s to ~0.8s (~7x). - Annotate exported model/method factories for isolatedDeclarations (TypeScript's fixMissingTypeAnnotationOnExports codefix plus hand-authored interfaces); type the ~44 mongoose `any`s and add an explicit PromptMethods interface (previously its declaration was silently dropped by the Rollup build). - Repoint package.json exports/main/module/types to tsdown output; drop rollup config. - Config lives in tsdown.config.mjs (native ESM) so CI without a TS-config loader can build it; bundle `dotenv` so the package stays self-contained for its env-loading side effect. - Fix a latent token `metadata` mismatch the accurate types surfaced: widen TokenCreate/UpdateData inputs to accept plain objects, flatten OAuthMetadata at the api boundary. - Update mongoMeili/aclEntry specs to the precise model types; drop redundant terser minification from data-provider's library build. All data-schemas tests pass; api builds clean against the new output. * 🔧 chore: Hash tsdown.config.mjs in data-schemas CI build-cache keys The data-schemas build switched from rollup to tsdown, but the build-data-schemas / build-api cache keys in backend-review, config-review, and playwright-mock still hashed the (now-deleted) rollup.config.js. Hash tsdown.config.mjs instead so a config-only change invalidates the cached dist/api builds. (Found by Codex review.) * 🔧 chore: Replace deprecated tsdown `external` with `deps.neverBundle` tsdown 0.22 deprecated the top-level `external` option in favor of `deps.neverBundle`. Migrate the data-schemas config and set `deps.onlyBundle: false` to silence the (intentional) dotenv bundling hint. Build output and externalization are unchanged — dotenv bundled, all peers external. |
||
|
|
bfb6b224d2
|
🔧 chore: Update ESLint config, Import Sorting script, Test Sharding, Bump @librechat/agents (#13552)
* 🔧 chore: Update ESLint config, add import sorting script, Test Sharding, Bump `@librechat/agents`
* Change 'no-nested-ternary' rule from 'warn' to 'error' in ESLint config
* Add new scripts for sorting imports in the project
* Update lint-staged configuration to include import sorting
* Modify GitHub Actions workflows to support sharding for unit tests
* chore: remove nested ternary expressions
* refactor: Extract scale multiplier logic into a separate function in CircleRender component
* refactor: Simplify auto-refill rendering logic in Balance component for better readability
* refactor: Improve width style handling in DataTable components for clarity and maintainability
* chore: remove CircleRender component
* delete: Remove CircleRender component as it is no longer needed in the project
* chore: Bump @librechat/agents to version 3.2.31 and update Node.js engine requirement
* Update @librechat/agents dependency from 3.2.2 to 3.2.31 in package-lock.json, api/package.json, and packages/api/package.json
* Change Node.js engine requirement from >=20.0.0 to >=24.0.0 in @librechat/agents
* chore: Add import sorting check to ESLint CI workflow
* Implement a new job in the GitHub Actions workflow to verify import ordering on changed files.
* The job checks for changes in specific file types and reports any import order drift, providing instructions for local fixes.
|
||
|
|
fb282a2afa
|
🐳 chore: Upgrade Docker Builds To Node 24 (#13448)
* chore: upgrade docker builds to node 24 * test: avoid array at in telemetry spec |
||
|
|
7ab925498f
|
🐢 ci: Raise test-packages-api timeout to 20 min (#13326)
The `Tests: @librechat/api` job in `backend-review.yml` had `timeout-minutes: 10`. Recent runs have started getting cancelled right at the 10-minute mark with tests still actively passing — the log streams `PASS …` lines through the final second before `The operation was canceled.` fires. Looking at recent runs in the wild: - Warm `ubuntu-latest` runners: suite finishes in ~4–5 min. - Cold/busy runners: 10+ min. No headroom = job killed mid-suite. The suite has grown over the last few months (OpenTelemetry, MCP OAuth, Bedrock, the Operational Prometheus Metrics work from #13265, plus the agents/responses harness). 10-minute ceiling that worked before is now a tail-latency cliff. Bump to 20 minutes. Still well below the runner's hard 6-hour cap, generous enough to absorb runner variance, and a stuck/genuine hang will still get killed eventually rather than billing forever. No code changes; one-line config bump. |
||
|
|
21574f02ca
|
🛡️ chore: Harden CI Supply Chain Workflows (#13090)
* chore: harden CI supply chain workflows * chore: address CI hardening review feedback * chore: tighten GitNexus dispatch hardening * chore: use app token for Locize PR automation * chore: use dedicated token for Locize PR automation |
||
|
|
f6ee2ea0ee |
📜 feat: Skills UI + Initial E2E CRUD / Sharing (#12580)
* 🎨 feat: Skills UI — Create/Edit/Share/List with Conditional File Tree First-pass UI on top of the CRUD API scaffolding (#12613). Ships the full user-facing flow for inline, single-SKILL.md skills and leaves a clean drop-in for phase-2 multi-file support. - Create a skill from /skills/new with name (kebab-case, validated), description, and SKILL.md body — wired to the real `useCreateSkillMutation` and `TCreateSkill` payload. - List skills in a sidebar (SkillsSidePanel) via `useListSkillsQuery` with live search filtering. - Edit any skill the caller has EDIT permission on — `useUpdateSkillMutation` passes `expectedVersion` for optimistic concurrency and surfaces 409 conflicts as a warning toast + cache refetch. - Non-blocking `TSkillWarning[]` (e.g. "description too short") are shown inline above the form after a successful create/patch. - Read-only mode when the current user lacks EDIT — the form still renders but inputs are marked `readOnly` and the save/reset buttons are hidden. - Share via ACL using the existing `GenericGrantAccessDialog` — the `ShareSkill` button is gated on the SHARE permission. - Delete with confirmation, driven by `useDeleteSkillMutation({ id })`. - Conditional file tree: only rendered when `useListSkillFilesQuery` returns > 0 files. The tree groups flat `relativePath` strings into a nested view (no `react-arborist` dependency) and supports per-file deletion via `useDeleteSkillFileMutation`. Upload is intentionally deferred — the backend stubs it at 501 in phase 1. - New routes: `/skills`, `/skills/new`, `/skills/:skillId`. - Sidebar accordion (`SkillsAccordion` wrapping `SkillsSidePanel`) added to `useSideNavLinks` gated on `PermissionTypes.SKILLS` USE. The initial UI branch (#12580) shipped a lot of exploration code on top of a now-superseded placeholder backend. Kept as complementary: the `Skills/` component tree, translation keys, role descriptions, `PublicSharingToggle` SKILL mapping, `resources.ts` SKILL config, `useCanSharePublic` SKILL mapping, and `data-provider/roles.ts` `useUpdateSkillPermissionsMutation`. Deferred out of this first pass: - Skill favorites (`useSkillFavorites`, `getSkillFavorites` endpoint) — the backend route doesn't exist yet; saving for a follow-up. - AgentConfig `SkillSelectDialog` integration — the UI branch had this gated behind `false &&`; rolled back with the config. - `InvocationMode` / `CategorySelector` / `parseSkillMd` / tree-node mutations — not in the Anthropic skill spec and not in the CRUD API. - `react-arborist` dependency — replaced with a hand-rolled recursive tree built from flat `TSkillFile[]`. - 38 data-schemas skill model tests: pass - 25 api skill route tests: pass - 16 user-controller cleanup tests: pass * 🔐 feat: Default-On Skills in Interface Config and Role Seeder The skills accordion was registered in the side nav gated on `PermissionTypes.SKILLS` USE, but no one was actually seeding that permission on startup, so a fresh install had the USER role with zero skill permissions and the accordion never rendered. Fixes three gaps: 1. `interfaceSchema` in data-provider's `config.ts` had no `skills` field at all. Added it alongside the existing agents/prompts shape (boolean | { use, create, share, public }) and a default of `{ use: true, create: true, share: false, public: false }`. 2. `loadDefaultInterface` in data-schemas passed every interface key through to the loaded config EXCEPT `skills`. Added the one-line passthrough so `appConfig.interfaceConfig.skills` is actually populated on boot. 3. `updateInterfacePermissions` in packages/api/src/app/permissions.ts seeds role permissions from the interface config on every restart. Added: - `SKILLS` case to `hasExplicitConfig` - `skillsDefaultUse/Create/Share/Public` extraction (mirrors prompts/agents) - `PermissionTypes.SKILLS` block in `allPermissions` that falls through config → roleDefaults → schema default, same pattern as AGENTS and PROMPTS - `SKILLS` entry in the share-backfill array so that pre-existing SKILL role docs missing SHARE/SHARE_PUBLIC get them filled on the next restart Test expectations updated: seven `expectedPermissionsFor(User|Admin)` blocks in `permissions.spec.ts` now include SKILLS, matching the role-default values (USER: use+create true, share/public false; ADMIN: all true). Result: on a fresh install, a regular USER gets skill USE/CREATE and the "Skills" accordion shows up in the chat side panel without any yaml config. Admins can lock it down per role or per tenant via `interface.skills` in librechat.yaml. Tests: - 34 packages/api permissions.spec.ts: pass - 151 packages/api app tests: pass - 38 data-schemas skill.spec.ts: pass - 928 data-provider tests: pass - 25 api skills.test.js: pass * ♻️ fix: Resolve Skills UI Review Findings Addresses the 13 findings from the PR review against the prior commit. 1. **canEdit consistency** — extracted `useSkillPermissions(skill)` as the single source of truth for owner/admin/ACL gating. `SkillsView`, `SkillForm`, `ShareSkill` all consume it; `SkillFileTree`'s per-file delete button now honors admin + EDIT-bit permissions instead of just ownership. Unit tests cover owner, admin, editor-ACL, viewer-ACL, owner-ACL, loading, and undefined-skill cases. 2. **Disabled submit buttons** — create/edit form submit buttons now set native `disabled` (not just `aria-disabled`) during `isLoading`. `onSubmit` also guards with an early return when the mutation is still in-flight so a duplicate enter-key submit can't create two skills. 3. **Wrong maxLength error message** — description/name `maxLength` rules no longer re-use `com_ui_skill_*_required`. Added dedicated `com_ui_skill_name_too_long` and `com_ui_skill_description_too_long` keys with the literal limit interpolated (`{{0}}`). 4. **Search debouncing** — `SkillsSidePanel` now threads the filter input through the existing `useDebounce` hook (250ms) so typing "skills" no longer fires six separate list queries. 5. **Frontend test coverage** — added: - `tree.test.ts` (9 tests) covering `buildTree` / `nodeKey` edge cases: empty input, single root file, multiple roots, nested folders, deeply-nested trees, lexicographic sort, empty paths, stable keys - `useSkillPermissions.test.ts` (7 tests) covering every precedence branch (owner / admin / EDIT / VIEW / owner-ACL / loading / undef) Form integration tests proved flaky against react-hook-form's async `isValid` with our jest-dom mock setup; deferred to a follow-up PR with a proper `@librechat/client` test harness. 6. **Shared `SKILL_NAME_PATTERN`** — promoted the regex plus the four length constants (`SKILL_NAME_MAX_LENGTH`, `SKILL_DESCRIPTION_MAX_LENGTH`, `SKILL_DESCRIPTION_SHORT_THRESHOLD`, `SKILL_DISPLAY_TITLE_MAX_LENGTH`, `SKILL_BODY_MAX_LENGTH`) out of `packages/data-schemas/src/methods/skill.ts` and into `packages/data-provider/src/types/skills.ts`. The data-schemas module now aliases the shared exports so the backend validator and the frontend form share one source of truth. Also fixed a latent bug: the client regex was stricter than the backend (`^[a-z0-9]+(?:-[a-z0-9]+)*$` vs. the real `^[a-z0-9][a-z0-9-]*$`), which would have rejected valid names like `foo--bar` client-side. 7. **Removed hardcoded "Claude"** — replaced `com_ui_skill_description_help` ("Claude uses this to...") with a new `com_ui_skill_create_subtitle` for the form header and `com_ui_skill_description_field_hint` ("This is what the model reads to decide...") for the inline hint. LibreChat is LLM-agnostic; the old copy misled GPT/Gemini users. 8. **Lifted tree mutation hook** — `useDeleteSkillFileMutation` is now instantiated once in `SkillFileTree` (not per `TreeRow`). A `TreeContext` provides `onDeleteFile` + `isDeleting` + `canEdit` to rows. A 60-node tree used to instantiate 60 mutation hooks; it now instantiates one. 9. **List O(n) re-render** — `SkillListItem` no longer reads `useParams()` directly. `SkillList` reads the active id once and passes `isActive` as a prop, so navigation only re-renders the two items whose `isActive` flipped (memo'd), not all N items. 10. **Deduped help text** — the field-level hint and form-level subtitle now use different translation keys with distinct copy instead of showing the same sentence twice on the same page. 11. **Removed ineffective `useCallback`** — `DeleteSkill.handleDelete`, `CreateSkillForm.onSubmit` / `.handleCancel`, `SkillForm.onSubmit`, and `SkillFileTree.handleDeleteFile` all wrapped closures around React Query `mutation` refs, whose identities change every render. Their dep arrays invalidated every render, making the memo a no-op with extra overhead. `SkillFileTree` now destructures the stable `mutate` function and inlines the arrow inside the memoized `contextValue` — one stable reference per deps change. 12. **Import order** — fixed shortest→longest package ordering and longest→shortest local ordering across all touched skill files per AGENTS.md. `react` always first where imported. 13. **Memoization principle** — documented the rule with inline comments: `memo` on components that appear in repeated contexts (`TreeRow`, `SkillListItem`) or as children of frequently-re-rendering parents (`ShareSkill` / `DeleteSkill` under `SkillForm`'s per-keystroke form-state updates). Removed `memo` from `SkillFileTree` since its parent `SkillDetailPanel` only re-renders on query-data changes. - 38 data-schemas skill.spec.ts - 34 packages/api permissions.spec.ts - 25 api skills.test.js - 16 client unit tests (9 buildTree + 7 useSkillPermissions) - All type-checks + eslint clean on touched files * 🧹 fix: Skills Duplication, Input Styling, Remove LLM-specific Copy Three UI fixes from an in-chat review pass: 1. **Sidebar duplication** — `SkillsView` was rendering its own `SkillsSidePanel` aside alongside the chat side panel's `SkillsAccordion`, so on `/skills` the user saw the skill list twice. Fixed by mirroring the `InlinePromptsView` pattern: the route content is now just the detail / create panel and the chat side panel is the sole list. Added `/skills → /skills/new` redirect and a `/skills/new` literal route so `useParams().skillId` is `undefined` for "new" (matches prompts). 2. **Name Input styling** — the big floating-label pattern used by prompts/agents for the primary name field was replaced with a conventional `<Label>` + `<Input>` above it, diverging from the rest of the app. Restored the prompts-style `text-2xl` input with the peer-focus animated label on both `CreateSkillForm` and `SkillForm`. Kept the conventional pattern for description and body since they're textareas. 3. **Remove LLM-specific copy from skill translations** — dropped `com_ui_skill_description_help` ("Claude uses this to...") and the transitional "This is what the model reads..." phrasing. Field hint is now a neutral "Be specific about when this skill should apply." and the create-page subtitle is a neutral "Author a new skill your agents can invoke." LibreChat is LLM-agnostic; baking product names into user-facing copy is wrong outside the `com_endpoint_anthropic_*` keys where the setting actually only applies to Claude models. Side-effect: the `SkillDetailView` wrapper in `SkillsView` now only renders the file-tree aside when the skill has > 0 files — same conditional-tree behavior as before, just scoped to this route instead of also trying to also render a list sidebar. - 16 client skill tests still pass - Type-check + eslint clean on touched files * 🎁 feat: Restore Skills UI from PR #12580 Brings back everything the original UI PR (#12580, commit da039917c) shipped that my earlier rebase dropped. Verbatim restores where possible; adapts the new hooks/types where the backend contract has shifted. **Scoped-out / gated-off (now restored as inert UI scaffolding):** - `hooks/useSkillFavorites.ts` + `utils/favoritesError.ts` + the `useGetSkillFavoritesQuery` / `useUpdateSkillFavoritesMutation` additions in `data-provider/Favorites.ts`. The backend route doesn't exist yet — the data-service functions resolve with empty arrays so the Star UI is a visual-only no-op until phase 2. - `dialogs/SkillSelectDialog.tsx` + the "Add Skills" section in `SidePanel/Agents/AgentConfig.tsx` (still gated behind the original `false &&`) + `skills?: string[]` on `AgentForm` / `Agent` / `AgentCreateParams` / `AgentUpdateParams` + the `skills: []` entry in `defaultAgentFormValues`. - `TUserFavorite.skillId` reserved on the shared favorites type. **Concept-is-gone / deleted-types (restored as UI-only types + stubs):** - `InvocationMode` enum and `TSkillNode`, `TSkillTreeResponse`, `TCreateSkillNodeRequest`, `TUpdateSkillNodeRequest` types in `packages/data-provider/src/types.ts`. UI-facing only; the backend flat `TSkillFile[]` contract is unchanged. - `TSkill.invocationMode?: InvocationMode` as an optional field. Forms read/write it in local state and deliberately drop it from the PATCH payload until the backend column lands. - `tree/SkillFileTree.tsx` (`react-arborist`-based), `SkillTreeNode.tsx`, `TreeToolbar.tsx`, `SkillFileEditor.tsx`, `SkillFilePreview.tsx` — full filesystem-style browser UI restored verbatim. - `data-provider/Skills/tree-queries.ts` + `tree-mutations.ts` hooks (`useGetSkillTreeQuery`, `useCreateSkillNodeMutation`, etc.). The `data-service` stubs them: `getSkillTree` returns `{ nodes: [] }`, `createSkillNode` / `updateSkillNode` / `updateSkillNodeContent` return synthetic node shapes, `deleteSkillNode` resolves void. Hooks compile and run; tree is empty until phase 2 wires a real backend. - `MutationKeys.createSkillNode` / `updateSkillNode` / `deleteSkillNode` / `updateSkillNodeContent` + `CreateSkillNodeBody` / `UpdateSkillNodeVariables` / `DeleteSkillNodeBody` / `UpdateSkillNodeContentVariables` types. - `QueryKeys.skillTree` / `skillNodeContent` / `skillFavorites` / `favorites` and the `skillTree()` endpoint helper. **Scope-simplified (restored with minimal adaptation):** - `display/SkillDetailHeader.tsx` + `display/SkillDetail.tsx`. Header now falls back to `InvocationMode.auto` when `skill.invocationMode` is undefined. - `forms/SkillContentEditor.tsx` — click-to-edit markdown preview toggle for the SKILL.md body field. Wired into both `CreateSkillForm` and `SkillForm` replacing the plain `<TextareaAutosize>`. (Needed `@ts-ignore` on `remarkPlugins` / `rehypePlugins` for the same `PluggableList` vs `Pluggable[]` shape drift `MarkdownLite.tsx` already works around.) - `forms/InvocationModePicker.tsx` + `forms/CategorySelector.tsx` — the auto/manual/both dropdown and the skill category selector. Wired into both forms inside a `FormProvider` so the Controller-based widgets can read `useFormContext`. `category` flows to the PATCH / POST payload as before; `invocationMode` is UI-only per the type note above. - `buttons/CreateSkillMenu.tsx` + `utils/parseSkillMd.ts` — dropdown with AI / Manual / Upload SKILL.md entries + the YAML frontmatter parser for the upload path. `CreateSkillForm.defaultValues` now accepts the parsed shape, so the upload → redirect → pre-populated form flow works again. - `buttons/AdminSettings.tsx` — admin permissions dialog. Uses the existing `useUpdateSkillPermissionsMutation` which was already wired. - `sidebar/FilterSkills.tsx` — restored filter + AdminSettings + CreateSkillMenu wrapper. `SkillsSidePanel.tsx` is back to the original `FilterSkills`-based layout. - `lists/SkillList.tsx` + `lists/SkillListItem.tsx` — restored verbatim. - `layouts/SkillsView.tsx` — restored the full tree + file editor + file preview layout. The chat side panel keeps its own accordion list; this view is the inline detail experience. - `hooks/Generic/useUnsavedChangesPrompt.ts` — route-leave guard hook. - `useGetSkillByIdQuery` is aliased to `useGetSkillQuery` so restored components (`SkillsView`, `SkillForm`) that import the old name resolve to the new hook. - `SkillSelectDialog` + `AgentConfig` coerce `skillsData?.skills` instead of `.data` (list response shape drift from the CRUD PR). - `CreateSkillForm` / `SkillForm` wrap their JSX in `FormProvider` so the restored `CategorySelector` and `SkillContentEditor` components — which read `useFormContext` — work inside the existing forms without another refactor. - `CreateSkillForm.defaultValues` prop accepts `Partial<Values> & { invocationMode?: unknown }` so the upload flow's `{ name, description, invocationMode }` shape passes through cleanly. - `SkillsView` route map gains `/skills/:skillId/edit` and `/skills/:skillId/file/:nodeId` so the tree-navigation URLs the original view produces actually resolve. - `client/package.json` gains `react-arborist@^3.4.3`. - ~60 translation keys the restored files reference — invocation labels, edit/create page titles, file editor chrome, tree toolbar tooltips, favorites, admin allow-settings, unknown-file-type, sr_public_skill, delete/rename _var variants — all added to `en/translation.json`. - Prompts-style floating-label name input — kept from my earlier commit so it matches the rest of the app (user reviewed and approved that styling). Hidden skill-body textarea is replaced by `SkillContentEditor` in both forms. - 38 data-schemas skill.spec.ts - 34 packages/api permissions.spec.ts - 25 api skills.test.js - 7 client useSkillPermissions.test.ts - Type-check: pre-existing error count (188) dropped to 120 because my restorations fixed some previously-broken field types. * chore: Update package-lock.json to include react-arborist and memoize-one * feat: Add support for react-arborist in Vite configuration This update introduces a new condition in the Vite configuration to handle the 'react-arborist' package, ensuring it is properly recognized during the build process. This change enhances compatibility with the recently added 'react-arborist' dependency in the project. * 🩹 fix: Hide InvocationMode, Fix SkillContentEditor Click-to-Edit 1. Hide InvocationModePicker from both CreateSkillForm and SkillForm. Component stays on disk for when the backend lands the column. 2. Fix "Click to edit" doing nothing on SkillContentEditor. The `onBlur={() => setIsEditing(false)}` on the TextareaAutosize was racing with `autoFocus` — React renders the textarea, autoFocus fires, then a layout/reconciliation blur fires immediately, bouncing back to preview mode before the user can interact. Removed onBlur; users toggle via the header button or Escape key. * 🎨 feat: Reader-First Skills UI — Match Claude.ai Layout Reworks the Skills UI from form-first to reader-first, matching Claude.ai's skill detail pattern. **Default view is now read-only.** Clicking a skill in the sidebar navigates to `/skills/:id` which renders `SkillDetail` — a clean content view with: - Skill name as the primary heading - Metadata row: "Added by" + "Last updated" (formatted date) - Description block - Rendered SKILL.md body in a bordered card with a source/rendered toggle (eye + code icons, matching Claude.ai's segmented control) No form fields, no save/cancel buttons. The user reads the skill first and takes action deliberately. **Create is now a dialog.** The `/skills/new` route is gone. `CreateSkillMenu` (the + dropdown in the sidebar) now opens `CreateSkillDialog` — a minimal modal with name, description, and instructions fields. Upload-from-file still works: parse → populate dialog → create. Matches Claude.ai's "Write skill instructions" modal. **Edit is behind an action.** The detail view shows an "Edit" button (permission-gated) that navigates to `/skills/:id/edit`, rendering the existing `SkillForm`. The edit route is preserved for direct linking. **Navigation goes to detail, not edit.** `SkillListItem` now navigates to `/skills/:id` (detail) instead of `/skills/:id/edit`. - `display/SkillMarkdownRenderer.tsx` — shared ReactMarkdown component extracted from `SkillContentEditor`. Same remark/rehype plugins, no form dependency. - `display/SkillDetail.tsx` — the reader-first view (replaces the old thin wrapper). - `dialogs/CreateSkillDialog.tsx` — OGDialog modal for skill creation. - `layouts/SkillsView.tsx` — gutted and rebuilt. Three states: no-skill (empty state), skillId (SkillDetail), skillId+edit (SkillForm). Removed full-page CreateSkillForm, removed TreeView. - `buttons/CreateSkillMenu.tsx` — opens dialog instead of navigating to `/skills/new`. Upload flow: parse → set dialog defaults → open. - `lists/SkillListItem.tsx` — navigate to detail, not edit. - `routes/index.tsx` — removed `/skills/new` and file/nodeId routes; `/skills` renders SkillsView directly (empty state). - `display/index.ts`, `dialogs/index.ts` — added new exports. - `locales/en/translation.json` — added ~10 new keys for metadata, toggle labels, dialog title, empty state. * 🩹 fix: SkillContentEditor click-to-edit z-index — button was z-0 behind rendered content * 🩹 fix: Align Edit button size with Share/Delete (size-9) * 🎨 feat: Claude.ai-Style Skill List Panel Rewrites the skills sidebar to match Claude.ai's panel layout: - Header: "Skills" title + search icon (toggles input) + add icon (opens CreateSkillDialog directly, no dropdown menu) - Collapsible "Skills" section with chevron toggle - Skill items: 24px icon badge (rounded square with ScrollText icon) + name only. No description text in the list — that lives in the detail view. Active item gets highlighted bg + bold font. - Removed AdminSettings button from sidebar header — admin config is accessible via the admin dashboard, not cluttering every user's skill list. - Removed FilterSkills wrapper (was Filter + AdminSettings + CreateSkillMenu). The search + create are now inline in the panel header. Files changed: - sidebar/SkillsSidePanel.tsx — full rewrite - sidebar/SkillsAccordion.tsx — simplified wrapper - lists/SkillList.tsx — collapsible section, no description - lists/SkillListItem.tsx — icon badge + name, memo'd * 🎨 fix: Align Skills UI Styling with Prompts Patterns Style alignment pass based on direct comparison with claude.ai and the existing prompts preview dialog. SkillsSidePanel search now replaces the title in the header row when toggled (search icon + input + X close), matching Claude.ai's pattern. Previously it pushed a separate input below the header, wasting vertical space. Close button clears the search term. Replaced `text-text-tertiary` with `text-text-secondary` across SkillDetail, SkillList, SkillForm, CreateSkillForm, CreateSkillDialog, SkillContentEditor. Tertiary was too dark / low contrast. SkillList section chevron label now reads "Personal skills" (matching Claude.ai) via the existing `com_ui_my_skills` key, instead of the generic "Skills" which duplicated the header. Aligned with `PromptDetailHeader` styling: - 48px round icon (ScrollText in bg-surface-secondary circle) - Name + public badge in the icon row - Metadata below the icon: User icon + author, Calendar icon + date (text-xs text-text-secondary with gap-3, matching prompts exactly) - Description uses the same label-above-text pattern as prompts - Content card uses `bg-transparent` border (not bg-surface-primary-alt) - Toggle buttons use size-5 icons and text-text-secondary for inactive Changed from `max-w-lg p-0` to `max-w-5xl` with the same max-height and padding pattern as the prompts PreviewPrompt dialog: `max-h-[80vh] p-1 sm:p-2 gap-3 sm:gap-4`. Close button now renders via default OGDialogContent behavior (removed showCloseButton=false). * 🩹 fix: SkillDetail fills parent height, tighter spacing (px-6 pb-6 gap-2) * 🩹 fix: Align Skills panel header padding (px-4) with list content below * 🩹 fix: Reduce Skills header top padding (pt-2) to align with sidebar icon strip * 🩹 fix: Tighten Skills header (py-2) and detail top (py-2) to align with sidebar icons and match edit view * 🩹 fix: Offset SidePanel Nav pt-2 with -mt-2 on SkillsAccordion so Skills header aligns with icon strip * 🛠️ fix: Increase Node memory limit for production build in package.json * 🩹 fix: Remove top padding from SkillDetail header row (py-2 → pb-2) * 🏗️ refactor: Move pt-2 from SidePanel/Nav wrapper to each panel Removed the global `pt-2` from `SidePanel/Nav.tsx` and pushed it into each panel's own top-level wrapper. This lets each panel own its vertical alignment independently — Skills can sit flush at the top to align with the sidebar icon strip, while other panels keep their original spacing. Panels updated with `pt-2`: - PromptsAccordion (via className on PromptSidePanel) - BookmarkPanel - FilesPanel - MemoryPanel - MCPBuilderPanel - AgentPanel (form wrapper) - AssistantPanel (form wrapper) - ParametersPanel (already had pt-2) SkillsAccordion: removed the -mt-2 hack, now naturally flush. * 🧹 fix: Align CreateSkillDialog field styling + remove 19 unused i18n keys Dialog fields: all three inputs now use consistent `rounded-xl border-border-medium px-3 py-2 text-sm` styling. Replaced the `<Input>` component with a plain `<input>` to avoid the component's built-in `rounded-lg border-border-light` overriding the dialog's border style. Labels use `font-medium` for consistency. Removed 19 unused translation keys from translation.json: com_ui_skill_body, com_ui_skill_body_placeholder, com_ui_skill_create_subtitle, com_ui_skill_file_delete_confirm, com_ui_skill_file_delete_error, com_ui_skill_file_deleted, com_ui_skill_files_empty, com_ui_skill_files_multi_hint, com_ui_skill_list, com_ui_skill_load_error, com_ui_skill_resize_file_tree, com_ui_skill_select_file, com_ui_skill_select_file_desc, com_ui_skills_load_error, com_ui_add_first_skill, com_ui_create_skill_page, com_ui_edit_skill_page, com_ui_save_skill, com_ui_no_skills_title * 🎁 feat: Upload Skill Dialog + Simplified Create Menu New `UploadSkillDialog` matching Claude.ai's upload modal: - Dashed drop zone with drag-and-drop support - Accepts .md, .zip, .skill files - Phase 1: processes .md files (parses YAML frontmatter → creates skill with body as the full file content) - Shows file requirements below the drop zone - On success: navigates to the new skill's detail view `CreateSkillMenu` now has two flat options (no sub-menu): - "Write skill instructions" → opens `CreateSkillDialog` - "Upload a skill" → opens `UploadSkillDialog` Removed the disabled "Create with AI" option and the old file input hidden-element approach. The sidebar `+` button now renders `CreateSkillMenu` directly instead of a standalone create dialog. - Removed 5 unused i18n keys (com_ui_skill_added_by, com_ui_skill_last_updated, com_ui_skills_add_first, com_ui_skills_filter_placeholder, com_ui_skills_new) - Tightened metadata gap in SkillDetail (mt-1 → mt-0.5) - Added 7 new upload-related i18n keys * 🔒 feat: Zip/Skill File Upload Support with Safety Limits Rewrites UploadSkillDialog to properly handle all three accepted file types: - `.md` — reads as text, parses YAML frontmatter, creates skill - `.zip` / `.skill` — reads as ArrayBuffer, extracts with JSZip, finds SKILL.md (at root or one level deep), parses its content, creates skill. Shows spinner during processing. Security guards against zip bombs: - MAX_ZIP_SIZE: 50MB compressed file limit - MAX_ENTRIES: 500 file limit inside the archive - Path traversal rejection: skips entries with `..` or leading `/` - SKILL.md search limited to depth ≤ 2 segments Added `jszip@^3.10.1` to client dependencies (already in the monorepo's node_modules from backend usage). The name is inferred from the zip filename if SKILL.md frontmatter doesn't have one (e.g. `skills-autofix.zip` → `skills-autofix`). * 🚀 feat: Backend Skill Import + Live File Upload Endpoints New endpoint that accepts a single multipart file (.md, .zip, .skill) and creates a skill with all its files in one request: - **.md**: parse YAML frontmatter → create skill with body - **.zip / .skill**: extract with JSZip, find SKILL.md (root or one level deep), create skill from its content, then persist every additional file via `upsertSkillFile` + local file storage strategy. Returns the created skill + an `_importSummary` with per-file results. Security: - 50MB compressed file size limit (multer) - 500 max entries in archive - 10MB per individual file - Path traversal rejection (no `..`, no absolute, validated charset) - File type filter: only .md/.zip/.skill accepted - Rate limited via existing `fileUploadIpLimiter` + `fileUploadUserLimiter` Handler lives in `packages/api/src/skills/import.ts` with injectable deps (`createSkill`, `upsertSkillFile`, `saveBuffer`) for testability. Replaced the 501 stub with a real handler: - Accepts multipart FormData with `file` + `relativePath` - Saves file via local storage strategy - Calls `upsertSkillFile` to persist the SkillFile record - Returns the upserted document - Rate limited, ACL-gated (EDIT permission required) - 10MB per file limit `UploadSkillDialog` now sends the file to `/api/skills/import` via `dataService.importSkill(formData)` — no more client-side JSZip. Removed `jszip` from client dependencies (only backend needs it). Added `importSkill()` in data-service + `importSkill()` endpoint builder in api-endpoints. Updated the file upload test from expecting 501 stub to expecting 400 "no file provided" (live validation). All 25 skill route tests pass. * 🔒 fix: Complete Import Handler — Validation, Ownership, Error Surfacing Fixes several gaps in the skill import flow: 1. **Skill validation now runs and surfaces properly.** The import handler calls the real `createSkill(CreateSkillInput)` which runs `validateSkillName`, `validateSkillDescription`, `validateSkillBody`. Validation errors (SKILL_VALIDATION_FAILED) are caught and returned as 400 with the issue messages. Duplicate-key errors return 409. Previously all errors were swallowed into a generic 500. 2. **`authorName` is now populated.** The `CreateSkillInput` requires `authorName` which was missing — resolved from `req.user.name ?? req.user.username ?? 'Unknown'`, matching the existing create handler. 3. **SKILL_OWNER permission is granted after import.** Calls `grantPermission` with `AccessRoleIds.SKILL_OWNER` so the uploader can edit/delete/share the imported skill. This was entirely missing — imported skills would have been ownerless. 4. **`tenantId` propagated.** Both the skill and each SkillFile record receive `req.user.tenantId` for multi-tenant deployments. 5. **SkillFile records are created in the DB.** Each non-SKILL.md file in the zip is saved to file storage via `saveBuffer` and recorded via `upsertSkillFile`, which validates the relativePath, infers the category from the path prefix, and atomically bumps the skill's `fileCount` and `version`. Import deps now include `grantPermission` from PermissionService, injected in `api/server/routes/skills.js`. * 🐛 fix: Import grant uses accessRoleId (not roleId) — fixes skill not appearing in list * 🎨 fix: Cache invalidation, file tree, frontmatter rendering Three fixes for the skill detail view: 1. **Cache invalidation after import.** UploadSkillDialog now calls `queryClient.invalidateQueries([QueryKeys.skills])` after a successful import so the sidebar list picks up the new skill without requiring a page refresh. 2. **File tree in detail view.** When a skill has `fileCount > 0`, the detail view now queries `useListSkillFilesQuery` and renders a file list below the body card — SKILL.md first, then folders and root files. Icons: Folder for directories, FileText for files. 3. **Frontmatter stripped and rendered as metadata.** YAML frontmatter (`---\nversion: 0.1.0\ntriggers: ...\n---`) is now parsed out of the body before markdown rendering. The `name` and `description` fields are skipped (already shown in the header). Remaining fields (version, triggers, dependencies, etc.) are displayed in a Claude.ai–style grid: label on the left, value on the right, above the rendered markdown content. Source view still shows the full raw body including frontmatter. * 🩹 fix: Always fetch skill files — fileCount may be stale in cached skill object * 🌳 feat: Inline File Tree in Sidebar Skill List Moves the file tree from the bottom of SkillDetail into the sidebar list, matching Claude.ai's pattern: - Multi-file skills show a chevron toggle on the right side of the skill list item - Clicking the chevron expands an inline file tree below the skill name: SKILL.md first, then folders (with folder icon + right chevron) and root files - File list is fetched lazily (only when expanded) via useListSkillFilesQuery - Clicking a file navigates to the skill detail view - Files section removed from SkillDetail — the sidebar is now the sole file tree location, keeping the detail panel clean SkillDetail cleaned up: removed groupFiles helper, file-related state, useListSkillFilesQuery import, FileText/Folder icon imports. * 🌲 feat: Virtualized inline file tree with react-vtree Replace hand-rolled recursive FolderRow/FileRow buttons with a proper virtualized FixedSizeTree from react-vtree for the sidebar skill list. Dynamic height tracks open folders; capped at 350px with smooth expand/collapse transitions. * chore: Remove no longer used SkillFileTree and SkillTreeNode components * chore: Update Vite config to replace 'react-arborist' with 'react-vtree' for module resolution * feat: Skill file content viewing with lazy DB caching - Add `skills` field to `fileStrategiesSchema` so operators can configure a dedicated storage backend for skill files. Falls back by type (image/document) when unset. - Fix hardcoded `FileSources.local` in skill save/import — now uses the resolved strategy via `getFileStrategy(req.config, { context })`. - Replace 501 download stub with real handler that streams from any storage backend and returns JSON `{ content, mimeType, isBinary }`. - Binary detection (null-byte + non-printable ratio on first 8 KB) flags files on first read so they're never re-fetched. - Text content ≤ 512 KB is cached in the SkillFile MongoDB document; subsequent reads skip storage entirely. - Clicking a skill row now expands inline files (not just chevron). - Clicking a file navigates to `?file=<path>` and renders content in a new SkillFileViewer (markdown, code, images, binary placeholder). * chore: Remove react-window and its type definitions from package.json and package-lock.json - Deleted `react-window` and `@types/react-window` dependencies from both `package.json` and `package-lock.json` to streamline the project and reduce unnecessary bloat. * fix: Build errors — remove endpoints import, fix Uint8Array cast - Replace `import { endpoints }` (not public) with inline URL in SkillFileViewer - Remove `as Uint8Array` cast in stream chunk handling - Extend getSkillFileByPath return type with content/isBinary to decouple from data-schemas build artifact resolution * chore: Remove 8 unused i18next keys com_ui_create_skill_ai, com_ui_create_skill_manual, com_ui_delete_folder_confirm_var, com_ui_delete_skill, com_ui_delete_skill_confirm_var, com_ui_delete_var, com_ui_rename_var, com_ui_skill_files * fix: Add configMiddleware to skills router, handle SKILL.md in viewer - Add configMiddleware to skills router so req.config is populated when getLocalFileStream (or any strategy) reads file paths. - Handle SKILL.md in download handler — serves skill.body directly from the Skill document instead of looking for a SkillFile record. - Clicking SKILL.md in sidebar tree now opens the file viewer (matching Claude.ai behavior: file view vs default detail view). * ci: Run unit tests on PRs to any branch Remove the branches filter from both test workflows so contributor PRs targeting feature branches (not just main/dev) get CI coverage. Path filters are kept so tests only run when relevant files change. * fix: Update skills route tests for download handler changes - Mock configMiddleware (sets req.config for file storage access) - Mock getStrategyFunctions and getFileStrategy (storage strategy deps) - Replace 501 stub test with SKILL.md content test + 404 test * fix: Auto-expand files, frontmatter parsing, select-none, prefetch - Auto-expand file tree when navigating directly to a skill URL - Prefetch files for the active skill (eliminates first-expand lag) - Fix frontmatter parser to handle multi-line YAML list values (triggers field was missing because it uses list syntax) - SkillFileViewer now parses frontmatter for .md files — shows structured grid + rendered body (matching SkillDetail's display) with source/rendered toggle - Add select-none to all sidebar skill and file tree buttons * refactor: Derive expanded state from isActive instead of useEffect Replace useEffect sync with deterministic derivation: expanded = hasFiles && (isActive || !collapsed) Active skill is always open. collapsed is a manual toggle that only takes effect on non-active items. * fix: Remove empty space above body card — overlay view toggle Move the rendered/source toggle from a dedicated row (40px of empty space) to an absolute-positioned overlay in the card's top-right corner, matching Claude.ai's layout. * fix: Remove header bars from content editors — overlay action buttons Collapse the full-width header bars ("Skill Content", "Text") in SkillContentEditor, PromptTextCard, and PromptEditor. Action buttons (edit/save toggle, copy, variables) are now absolute-positioned in the card's top-right corner, reclaiming ~46px of vertical space. * fix: Spinner visibility in file viewer — use text-text-secondary * fix: Address review findings — security, correctness, code quality Codex P1: Use $unset instead of undefined to clear cached content and isBinary fields on file re-upload (Mongoose strips undefined). Codex P2: Match skill-file validation errors by error.code instead of error.message substring. F1: Zip bomb defense — track cumulative decompressed bytes (500 MB cap), check declared uncompressed size before buffering each entry. F2: Remove misleading "atomically" from import handler JSDoc. F3: Static import for isBinaryBuffer instead of dynamic import(). F4: Replace console.error with logger in upload handler. F6: Add multer error handler middleware to skills router. F7: Move React import to top of SkillDetail.tsx. F9: Fix variable shadowing (trimmed → item) in parseFrontmatter. F11: Replace JSON.parse(JSON.stringify()) with toJSON() for Mongoose document serialization. F12: Remove dead dynamic import('fs') fallback (memoryStorage always provides file.buffer). F13: Hoist MIME_MAP to module scope to avoid per-call allocation. F16: Share single multer.memoryStorage() instance. * fix: Follow-up review — close zip bomb gap, fix error handler F1: Add post-decompression cumulative byte check with break (the pre-decompression check relies on undocumented JSZip internals that may be absent; this closes the gap unconditionally). F2+F3: Multer error handler now forwards non-multer errors via next(err) instead of swallowing them. Also catches file filter rejections (plain Error, not MulterError) by message prefix. F4: Move isBinaryBuffer import to local imports section per CLAUDE.md import order rules. F5: Simplify dead toJSON branch — createSkill returns a POJO. * nit: Link filter error message to handler prefix check * feat: Accordion expansion + active file highlight in sidebar - Only one skill's file tree can be expanded at a time (accordion). Expansion state lifted from SkillListItem to SkillList. - Selected file gets bg-surface-active highlight in the tree. Skill row uses subtle style (no background) when a file is active, matching Claude.ai's pattern where the file — not the skill — carries the selection state. * style: Adjust margin for file tree in SkillListItem component - Reduced left margin from 10 to 5 for improved layout consistency in the file tree display. * fix: TS error on FileTreeNode, nested ternary, chevron collapse - Make style prop optional to match react-vtree's NodeComponentProps - Flatten nested ternary for skill row active styles - Skill row click expands (but doesn't collapse) files + navigates - Chevron click explicitly toggles collapse (matching Claude.ai where clicking the chevron is how you collapse files) * fix: Upload basePath, reject SKILL.md uploads, add skills permission route - Pass basePath: 'uploads' in per-file upload handler (was defaulting to 'images' path, inconsistent with the import flow). - Reject uploads targeting SKILL.md (reserved path — download handler special-cases it to return skill.body, making an uploaded file unreachable via the API). - Add skills entry to roles router permissionConfigs so PUT /api/roles/:roleName/skills actually reaches a handler instead of returning 404. * feat: Expand content area, move controls to header, reduce padding Default detail view: - Remove rounded-xl bordered card wrapper — content flows directly into the article, capitalizing on full screen width - Move eye/code toggle inline with the divider row - Reduce px-6/pb-6 to px-4/pb-4 File viewer: - Move eye/code toggle from card overlay to the header bar - Add copy-to-clipboard button for text files in the header bar - Remove rounded-xl bordered card wrapper for markdown content - Remove bordered pre wrapper for non-markdown text - Reduce px-6/py-4 to px-4/py-3 Both views maximize content space over decorative chrome. * fix: Stable header height, restore some padding - Fix layout shift in file viewer header: use fixed h-10 so the bar height stays constant whether the eye/code toggle renders (markdown) or not (plain text). - Bump content padding from px-4/py-3 back to px-5/py-4 in both views — the previous reduction was too aggressive. * fix: Grant rollback, path validation, error format, dead code cleanup F2: grantOwnership now rolls back (compensating delete) on failure, matching the create handler. Both markdown and zip import paths check the result and return 500 on grant failure. F4: Upload handler validates relativePath with regex + traversal check before calling downstream upsertSkillFile. F5: Document JSZip _data.uncompressedSize as best-effort; the post-decompression cumulative check is the real safety net. F10: Standardize all upload handler error responses to { error } (was { message }, inconsistent with handlers.ts). F13: Single-pass fileResults accumulation in import handler. F1-5: Remove dead uploadFileStubHandler (no route references it). Codex P2: Fix delete nav from /skills/new to /skills. F12: Use cn() in UploadSkillDialog instead of template literals. * perf: Stream-first binary detection + O(1) public skill check F1: Download handler now reads only the first 8 KB for binary detection. If binary, the stream is destroyed immediately without buffering the remaining file. Text files continue reading for caching. Eliminates buffering up to 10 MB per request for binary files under concurrent load. F7: Single-skill GET and PATCH now use hasPublicPermission (O(1) ACL lookup) instead of getPublicSkillIdSet (queries ALL public skill IDs). The list handler still uses the Set approach since it serializes multiple skills. serializeSkill/serializeSkillSummary now accept boolean | Set for flexibility. * fix: Update test to match { error } response format * fix: Critical stream truncation bug, grantedBy, error format NF-1 (CRITICAL): Rewrite binary detection to single for-await loop. Breaking out of for-await-of destroys the stream via iterator.return(), so the previous two-loop approach silently truncated text files > 8KB. Now: one loop collects chunks, checks binary after 8KB accumulated, and either destroys+returns (binary) or continues reading (text). NF-2: Add grantedBy to import handler's grantPermission call and interface (was missing, inconsistent with create handler). NF-3: Standardize all import handler error responses from { message } to { error }, matching handlers.ts convention. Update client's UploadSkillDialog to read response.data.error accordingly. * fix: Prefer specific validation message over generic error field * fix: YAML quote stripping, saveBuffer null guard, dot segment rejection - Strip surrounding YAML quotes from frontmatter values so name: "my-skill" parses as my-skill (not "my-skill" with quotes that fails the name validator). - Guard resolveSkillStorage against backends with saveBuffer: null (e.g. OpenAI/vector strategies) — throws a descriptive error caught by the handler's try/catch instead of a TypeError. - Tighten upload path validation to reject . segments (e.g. docs/./a.md) matching the model-layer validator, preventing storage writes for paths the DB will reject. * fix: Orphan cleanup, stream errors, malformed zip, cache latency F1: Upload handler now deletes the stored blob if the subsequent DB upsert fails, preventing orphaned files on disk/cloud. F2: Multer error handler returns { error } (was { message }). F3: Wrap JSZip.loadAsync in try/catch — malformed zip returns 400 instead of falling through to 500. F4: Raw download stream gets an error handler — logs the error and destroys the response if headers were already sent. F8: Strip leading hyphens from inferred skill name so filenames like _my-skill.zip don't produce -my-skill (invalid name pattern). F9: Fire-and-forget all updateSkillFileContent cache writes so the response is sent immediately. Cache failures are logged but don't block or fail the read. * fix: Import orphan cleanup + Content-Disposition sanitization Finding A: Add deleteFile dep to ImportSkillDeps. The per-file loop in handleZip now cleans up stored blobs when upsertSkillFile fails, closing the second half of the F1 orphan fix (upload handler was already fixed). Finding B: Sanitize filename in Content-Disposition header for raw downloads — strip quotes, backslashes, and newlines to prevent header injection from user-uploaded filenames. * security: Prevent stored XSS via raw file downloads Non-image files served via ?raw=true now use Content-Disposition: attachment (force download) instead of inline. An uploaded .html or .svg file served inline from the LibreChat origin could execute scripts with access to the user's session — this closes that vector. Images stay inline (needed for <img> rendering in SkillFileViewer). X-Content-Type-Options: nosniff added to prevent MIME sniffing. * security: Block SVG XSS — allowlist safe raster MIME types for inline SVG (image/svg+xml) passed the startsWith('image/') check and was served inline, but SVG is a scriptable format — embedded <script> tags execute in the LibreChat origin. Replace the prefix match with a Set of safe raster-only MIME types (png, jpeg, gif, webp, avif, bmp). SVGs and any future scriptable image/* subtypes now get Content-Disposition: attachment (forced download). * fix: Cap JSON text response at 1MB, consistent md name inference F3: Text files > 1MB now return { isBinary: false } with no content field, forcing the client to use ?raw=true for download. Prevents buffering 10MB files into heap for JSON serialization. Frontend shows a download fallback when content is absent. F4: handleMarkdown now infers skill name from filename (same as handleZip) when frontmatter has no name, instead of rejecting with 400. Consistent behavior across import paths. F1 (reviewer concern): upsertSkillFile is NOT affected — it uses { new: false } for insert-vs-replace detection but does a follow-up findOne (lines 855-859) to return the post-upsert document. * fix: deleteFile arg shape, raw URL base path, hoist SAFE_INLINE_MIMES Codex P2: deleteFile expects { filepath } object, not a raw string. Both upload handler cleanup and import handler cleanup now pass { filepath } to match the strategy contract (deleteLocalFile, deleteFileFromS3 all expect a file object). Codex P2: Raw download URL in SkillFileViewer now uses apiBaseUrl prefix so subpath deployments (/chat, etc.) resolve correctly. NIT: Hoist SAFE_INLINE_MIMES Set to factory scope — was re-allocated per raw download request inside the if block. * fix: Remove inert cache write for large text files, localize aria-label N2: The { isBinary: false } cache write for text files > 1MB had no effect — subsequent requests still fell through to stream read since neither isBinary nor content provided a fast-path short-circuit. Removed the pointless DB updateOne per request. N4: Replace hardcoded "Back to skill" aria-label with localize(). * refactor: Extract shared parseFrontmatter, widen deleteFile type N3: Extract parseFrontmatter into Skills/utils/frontmatter.ts — single implementation shared by SkillDetail and SkillFileViewer. Accepts optional skipKeys set so callers control which frontmatter fields are excluded (SkillDetail skips name/description since they're shown in the header; other .md files show all fields). N5: Widen ImportSkillDeps.deleteFile file param from { filepath } to { filepath; [key: string]: unknown } to signal extensibility if strategies start accessing additional file properties. * fix: Advance i past list items for skipped keys, DRY parseSkillMd Finding A: parseFrontmatter now consumes multi-line YAML list items before checking skipKeys — prevents list lines from leaking into subsequent key parsing as spurious fields. Finding B: parseSkillMd now delegates to the shared parseFrontmatter instead of re-implementing the same frontmatter scanning loop. Reduces client-side parseFrontmatter implementations from 3 to 1. * fix: Call apiBaseUrl(), delete storage blob on file removal - apiBaseUrl is a function, not a string — call it in the template literal so raw download URLs resolve correctly. - deleteFileHandler now looks up the file record before deleting, then fire-and-forget deletes the storage blob via the strategy's deleteFile. Previously only the DB record was removed, leaving orphaned blobs in local/S3/Firebase/Azure storage. * fix: Clean up storage blobs when deleting an entire skill deleteHandler now lists all files for the skill before calling deleteSkill, then fire-and-forget deletes each blob via the storage strategy. Previously only per-file deletion cleaned up blobs — deleting a whole skill left all associated files orphaned in local/S3/Firebase/Azure storage. * refactor: useImportSkillMutation hook, fix TSkill[] unsafe cast - Create useImportSkillMutation in mutations.ts + ImportSkillOptions type. UploadSkillDialog now uses the mutation hook instead of calling dataService.importSkill directly with manual useState loading management. Eliminates unmounted-component state update risk and aligns with the React Query mutation pattern used by every other mutation in the codebase. - SkillSelectDialog: replace as unknown as TSkill[] with proper TSkillSummary typing. SkillCard props updated to TSkillSummary. The dialog only uses summary-level fields (name, description, category, author) — the cast was hiding a type mismatch. * fix: Use saved source for import cleanup, delete old blob on replace Codex P2: Import cleanup now uses file.source (the backend the file was actually saved to) instead of re-resolving from config. In mixed strategy setups, the previous approach could target the wrong backend. Codex P2: When re-uploading a file to an existing relativePath, the old blob is now deleted after successful upsert. Previously only the DB record was replaced, leaving the old storage object orphaned. * fix: Register PUT /:roleName/skills route in roles router * fix: Re-read skill after zip file processing for fresh metadata The import response was built from the skill object created before the file loop, but each upsertSkillFile bumps version and fileCount. Clients caching the stale response would get 409 conflicts on first edit and see incorrect file counts. Now re-reads the skill via getSkillById after the loop so the response reflects the current version, fileCount, and updatedAt. * fix: Size-check SKILL.md before decompression, don't gate on fileCount P1: SKILL.md was decompressed before any size accounting. A crafted archive could expand SKILL.md past 10MB before validation ran. Now checks declared size pre-decompression and actual size post, both against MAX_SINGLE_FILE_BYTES. P2: File list query was gated on cached fileCount which can be stale after mutations. Now fetches files for the active skill regardless of fileCount. hasFiles derived from fetched data with fileCount as fallback, so newly uploaded files appear without hard refresh. * fix: Move files declaration before hasFiles to avoid TDZ error * security: Stream-decompress zip entries with enforced byte cap Replace zipEntry.async('nodebuffer') (buffers entire entry before checking limits) with zipEntry.nodeStream('nodebuffer') piped through a byte counter that destroys the stream when the per-file or cumulative limit is exceeded. Previously, when JSZip's _data.uncompressedSize was absent (the common case), a high-ratio entry could allocate hundreds of MB before the post-decompression check caught it. Now decompression is aborted mid-stream at the exact byte threshold — no entry can exceed its limit regardless of compression ratio. * refactor: Reorganize access check for prompts in useSideNavLinks hook Moved the prompts access check to a new position in the code to improve readability and maintainability. This change ensures that the prompts link is added to the navigation only if the user has the appropriate access, without altering the existing functionality. --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
fda1bfc3cc
|
🔬 ci: Add TypeScript Type Checks to Backend Workflow and Fix All Type Errors (#12451)
* fix(data-schemas): resolve TypeScript strict type check errors in source files - Constrain ConfigSection to string keys via `string & keyof TCustomConfig` - Replace broken `z` import from data-provider with TCustomConfig derivation - Add `_id: Types.ObjectId` to IUser matching other Document interfaces - Add `federatedTokens` and `openidTokens` optional fields to IUser - Type mongoose model accessors as `Model<IRole>` and `Model<IUser>` - Widen `getPremiumRate` param to accept `number | null` - Widen `bulkWriteAclEntries` ops to untyped `AnyBulkWriteOperation[]` - Fix `getUserPrincipals` return type to use `PrincipalType` enum - Add non-null assertions for `connection.db` in migration files - Import DailyRotateFile constructor directly instead of relying on broken module augmentation across mismatched node_modules trees - Add winston-daily-rotate-file as devDependency for type resolution * fix(data-schemas): resolve TypeScript type errors in test files - Replace arbitrary test keys with valid TCustomConfig properties in config.spec - Use non-null assertions for permission objects in role.methods.spec - Replace `.SHARED_GLOBAL` access with `.not.toHaveProperty()` for legacy field - Add non-null assertions for balance, writeRate, readRate in spendTokens.spec - Update mock user _id to use ObjectId in user.test - Remove unused Schema import in tenantIndexes.spec * fix(api): resolve TypeScript strict type check errors across source and test files - Widen getUserPrincipals dep type in capabilities middleware - Fix federatedTokens type in createSafeUser return - Use proper mock req type for read-only properties in preAuthTenant.spec - Replace `as IUser` casts with ObjectId-typed mocks in openid/oidc specs - Use TokenExchangeMethodEnum values instead of string literals in MCP specs - Fix SessionStore type compatibility in sessionCache specs - Replace `catch (error: any)` with `(error as Error)` in redis specs - Remove invalid properties from test data in initialize and MCP specs - Add String.prototype.isWellFormed declaration for sanitizeTitle spec * fix(client): resolve TypeScript type errors in shared client components - Add default values for destructured bindings in OGDialogTemplate - Replace broken ExtendedFile import with inline type in FileIcon * ci: add TypeScript type-check job to backend review workflow Add a `typecheck` job that runs `tsc --noEmit` on all four TypeScript workspaces (data-provider, data-schemas, @librechat/api, @librechat/client) after the build step. Catches type errors that rollup builds may miss. * fix(data-schemas): add local type declaration for DailyRotateFile transport The `winston-daily-rotate-file` package ships a module augmentation for `winston/lib/winston/transports`, but it fails when winston and winston-daily-rotate-file resolve from different node_modules trees (which happens in this monorepo due to npm hoisting). Add a local `.d.ts` declaration that augments the same module path from within data-schemas' compilation unit, so `tsc --noEmit` passes while keeping the original runtime pattern (`new winston.transports.DailyRotateFile`). * fix: address code review findings from PR #12451 - Restore typed `AnyBulkWriteOperation<AclEntry>[]` on bulkWriteAclEntries, cast to untyped only at the tenantSafeBulkWrite call site (Finding 1) - Type `findUser` model accessor consistently with `findUsers` (Finding 2) - Replace inline `import('mongoose').ClientSession` with top-level import type - Use `toHaveLength` for spy assertions in playwright-expect spec file - Replace numbered Record casts with `.not.toHaveProperty()` in role.methods.spec for SHARED_GLOBAL assertions - Use per-test ObjectIds instead of shared testUserId in openid.spec - Replace inline `import()` type annotations with top-level SessionData import in sessionCache spec - Remove extraneous blank line in user.ts searchUsers * refactor: address remaining review findings (4–7) - Extract OIDCTokens interface in user.ts; deduplicate across IUser fields and oidc.ts FederatedTokens (Finding 4) - Move String.isWellFormed declaration from spec file to project-level src/types/es2024-string.d.ts (Finding 5) - Replace verbose `= undefined` defaults in OGDialogTemplate with null coalescing pattern (Finding 6) - Replace `Record<string, unknown>` TestConfig with named interface containing explicit test fields (Finding 7) |
||
|
|
c324a8d9e4
|
⚡ refactor: Parallelize CI Workflows with Isolated Caching and Fan-Out Test Jobs (#12088)
* refactor: CI Workflow for Backend with Build and Test Jobs - Updated the GitHub Actions workflow to include a new build job that compiles packages and uploads build artifacts. - Added separate test jobs for each package (`api`, `data-provider`, and `data-schemas`) to run unit tests after the build process. - Introduced caching for build artifacts to optimize build times. - Configured Jest to utilize 50% of available workers for improved test performance across all Jest configurations in the `api`, `data-schemas`, and `packages/api` directories. * refactor: Update CI Workflow for Backend with Enhanced Build and Cache Management - Modified the GitHub Actions workflow to improve the build process by separating build and cache steps for `data-provider`, `data-schemas`, and `api` packages. - Updated artifact upload and download steps to reflect the new naming conventions for better clarity. - Enhanced caching strategies to optimize build times and ensure efficient artifact management. * chore: Node Modules Caching in CI Workflow - Updated the GitHub Actions workflow to implement caching for the `node_modules` directory, improving build efficiency by restoring cached dependencies. - Adjusted the installation step to conditionally run based on cache availability, optimizing the overall CI process. * refactor: Enhance CI Workflow for Frontend with Build and Test Jobs - Updated the GitHub Actions workflow to introduce a structured build process for frontend packages, including separate jobs for building and testing on both Ubuntu and Windows environments. - Implemented caching strategies for `node_modules` and build artifacts to optimize build times and improve efficiency. - Added artifact upload and download steps for `data-provider` and `client-package` builds, ensuring that builds are reused across jobs. - Adjusted Node.js version specification for consistency and reliability across different jobs. * refactor: Update CI Workflows for Backend and Frontend with Node.js 20.19 and Enhanced Caching - Updated Node.js version to 20.19 across all jobs in both backend and frontend workflows for consistency. - Enhanced caching strategies for build artifacts and `node_modules`, increasing retention days from 1 to 2 for better efficiency. - Adjusted cache keys to include additional files for improved cache hit rates during builds. - Added conditional installation of dependencies to optimize the CI process. * chore: Configure Jest to Use 50% of Available Workers Across Client and Data Provider - Added `maxWorkers: '50%'` setting to Jest configuration files for the client and data provider packages to optimize test performance by utilizing half of the available CPU cores during test execution. * chore: Enhance Node Modules Caching in CI Workflows - Updated caching paths in both backend and frontend GitHub Actions workflows to include additional `node_modules` directories for improved dependency management. - This change optimizes the caching strategy, ensuring that all relevant modules are cached, which can lead to faster build times and more efficient CI processes. * chore: Update Node Modules Cache Keys in CI Workflows - Modified cache keys in both backend and frontend GitHub Actions workflows to include the Node.js version (20.19) for improved cache management. - This change ensures that the caching mechanism is more specific, potentially enhancing cache hit rates and build efficiency. * chore: Refactor Node Modules Cache Keys in CI Workflows - Updated cache keys in backend and frontend GitHub Actions workflows to be more specific, distinguishing between frontend and backend caches. - Removed references to `client/node_modules` in backend workflows to streamline caching paths and improve cache management. |
||
|
|
a0f9782e60
|
🪣 fix: Prevent Memory Retention from AsyncLocalStorage Context Propagation (#11942)
* fix: store hide_sequential_outputs before processStream clears config processStream now clears config.configurable after completion to break memory retention chains. Save hide_sequential_outputs to a local variable before calling runAgents so the post-stream filter still works. * feat: memory diagnostics * chore: expose garbage collection in backend inspect command Updated the backend inspect command in package.json to include the --expose-gc flag, enabling garbage collection diagnostics for improved memory management during development. * chore: update @librechat/agents dependency to version 3.1.52 Bumped the version of @librechat/agents in package.json and package-lock.json to ensure compatibility and access to the latest features and fixes. * fix: clear heavy config state after processStream to prevent memory leaks Break the reference chain from LangGraph's internal __pregel_scratchpad through @langchain/core RunTree.extra[lc:child_config] into the AsyncLocalStorage context captured by timers and I/O handles. After stream completion, null out symbol-keyed scratchpad properties (currentTaskInput), config.configurable, and callbacks. Also call Graph.clearHeavyState() to release config, signal, content maps, handler registry, and tool sessions. * chore: fix imports for memory utils * chore: add circular dependency check in API build step Enhanced the backend review workflow to include a check for circular dependencies during the API build process. If a circular dependency is detected, an error message is displayed, and the process exits with a failure status. * chore: update API build step to include circular dependency detection Modified the backend review workflow to rename the API package installation step to reflect its new functionality, which now includes detection of circular dependencies during the build process. * chore: add memory diagnostics option to .env.example Included a commented-out configuration option for enabling memory diagnostics in the .env.example file, which logs heap and RSS snapshots every 60 seconds when activated. * chore: remove redundant agentContexts cleanup in disposeClient function Streamlined the disposeClient function by eliminating duplicate cleanup logic for agentContexts, ensuring efficient memory management during client disposal. * refactor: move runOutsideTracing utility to utils and update its usage Refactored the runOutsideTracing function by relocating it to the utils module for better organization. Updated the tool execution handler to utilize the new import, ensuring consistent tracing behavior during tool execution. * refactor: enhance connection management and diagnostics Added a method to ConnectionsRepository for retrieving the active connection count. Updated UserConnectionManager to utilize this new method for app connection count reporting. Refined the OAuthReconnectionTracker's getStats method to improve clarity in diagnostics. Introduced a new tracing utility in the utils module to streamline tracing context management. Additionally, added a safeguard in memory diagnostics to prevent unnecessary snapshot collection for very short intervals. * refactor: enhance tracing utility and add memory diagnostics tests Refactored the runOutsideTracing function to improve warning logic when the AsyncLocalStorage context is missing. Added tests for memory diagnostics and tracing utilities to ensure proper functionality and error handling. Introduced a new test suite for memory diagnostics, covering snapshot collection and garbage collection behavior. |
||
|
|
b9792160e2
|
💾 feat: Add Memory Configuration Options for CI unit tests (#10567)
* 💾 feat: Add Memory Configuration Options for CI unit tests - configured GitHub Actions workflows with configurable Node.js heap allocation, defaults to 6144 MiB - added heap usage logging for memory monitoring and debugging - increased Docker frontend build memory allocation to ensure consistent memory limits - optimized Jest timeout for tokenSplit test * 💾 feat: Add Memory Configuration Options for CI unit tests - responding to PR feedback from Copilot |
||
|
|
ac68e629e6
|
📡 refactor: MCP Runtime Config Sync with Redis Distributed Locking (#10352)
* 🔄 Refactoring: MCP Runtime Configuration Reload
- PrivateServerConfigs own cache classes (inMemory and Redis).
- Connections staleness detection by comparing (connection.createdAt and config.LastUpdatedAt)
- ConnectionsRepo access Registry instead of in memory config dict and renew stale connections
- MCPManager: adjusted init of ConnectionsRepo (app level)
- UserConnectionManager: renew stale connections
- skipped test, to test "should only clear keys in its own namespace"
- MCPPrivateServerLoader: new component to manage logic of loading / editing private servers on runtime
- PrivateServersLoadStatusCache to track private server cache status
- New unit and integration tests.
Misc:
- add es lint rule to enforce line between class methods
* Fix cluster mode batch update and delete workarround. Fixed unit tests for cluster mode.
* Fix Keyv redis clear cache namespace awareness issue + Integration tests fixes
* chore: address copilot comments
* Fixing rebase issue: removed the mcp config fallback in single getServerConfig method:
- to not to interfere with the logic of the right Tier (APP/USER/Private)
- If userId is null, the getServerConfig should not return configs that are a SharedUser tier and not APP tier
* chore: add dev-staging branch to workflow triggers for backend, cache integration, and ESLint checks
---------
Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com>
|
||
|
|
14660d75ae
|
🆕 feat: Enhanced Title Generation Config Options (#8580)
* 🏗️ refactor: Extract reasoning key logic into separate function
* refactor: Ensure `overrideProvider` is always defined in `getProviderConfig` result, and only used in `initializeAgent` if different from `agent.provider`
* feat: new title configuration options across services
- titlePrompt
- titleEndpoint
- titlePromptTemplate
- new "completion" titleMethod (new default)
* chore: update @librechat/agents and conform openai version to prevent SDK errors
* chore: add form-data package as a dependency and override to v4.0.4 to address CVE-2025-7783
* feat: add support for 'all' endpoint configuration in AppService and corresponding tests
* refactor: replace HttpsProxyAgent with ProxyAgent from undici for improved proxy handling in assistant initialization
* chore: update frontend review workflow to limit package paths to data-provider
* chore: update backend review workflow to include all package paths
|
||
|
|
cdf42b3a03
|
✨ feat: Add Dynamic User Field Placeholder Support in MCP Variables (#7825)
* chore: linting in mcp.spec.ts * chore: linting in mcp.ts * feat(mcp): support dynamic user field placeholders in MCP environment variables - Added user object handling in MCP options, allowing for dynamic user field processing in environment variables, headers, and URLs. - Updated `processMCPEnv` to utilize user fields for more flexible configurations. * chore: update backend review workflow to include unit tests for @librechat/data-schemas |
||
|
|
29ef91b4dd
|
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context
chore: mcp typing, use `t`
WIP: first pass, Memories UI
- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.
chore: move mcp-related files to own directory
chore: rename librechat-mcp to librechat-api
WIP: first pass, memory processing and data schemas
chore: linting in fileSearch.js query description
chore: rename librechat-api to @librechat/api across the project
WIP: first pass, functional memory agent
feat: add MemoryEditDialog and MemoryViewer components for managing user memories
- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.
feat: add memory permissions management
- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.
feat: move AdminSettings component to a new position in MemoryViewer for better visibility
refactor: clean up commented code in MemoryViewer component
feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration
- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.
refactor: optimize memory filtering in MemoryViewer using match-sorter
- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.
feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling
feat: implement access control for MemoryEditDialog and MemoryViewer components
refactor: remove commented out code and create runMemory method
refactor: rename role based files
feat: implement access control for memory usage in AgentClient
refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code
refactor: make `agents` dir in api package
refactor: migrate Azure utilities to TypeScript and consolidate imports
refactor: move sanitizeFilename function to a new file and update imports, add related tests
refactor: update LLM configuration types and consolidate Azure options in the API package
chore: linting
chore: import order
refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file
chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json
refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports
refactor: move createRun function to a new run.ts file and update related imports
fix: ensure safeAttachments is correctly typed as an array of TFile
chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file
refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options
feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling
fix: update types due to new TEndpointOption typing
fix: ensure safe access to group parameters in initializeOpenAIOptions function
fix: remove redundant API key validation comment in initializeOpenAIOptions function
refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation
refactor: decouple req.body fields and tool loading from initializeAgentOptions
chore: linting
refactor: adjust column widths in MemoryViewer for improved layout
refactor: simplify agent initialization by creating loadAgent function and removing unused code
feat: add memory configuration loading and validation functions
WIP: first pass, memory processing with config
feat: implement memory callback and artifact handling
feat: implement memory artifacts display and processing updates
feat: add memory configuration options and schema validation for validKeys
fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements
refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling
WIP: initial tokenLimit config and move Tokenizer to @librechat/api
refactor: update mongoMeili plugin methods to use callback for better error handling
feat: enhance memory management with token tracking and usage metrics
- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.
feat: implement memory artifact handling in attachment handler
- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.
refactor: restructure web search key extraction logic
- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors
feat: add personalization settings and memory preferences management
- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.
style: personalization switch class
feat: add PersonalizationIcon and align Side Panel UI
feat: implement memory creation functionality
- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.
feat: enhance message processing with configurable window size
- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.
chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json
chore: remove OpenAPIPlugin and its associated tests
chore: remove MIGRATION_README.md as migration tasks are completed
ci: fix backend tests
chore: remove unused translation keys from localization file
chore: remove problematic test file and unused var in AgentClient
chore: remove unused import and import directly for JSDoc
* feat: add api package build stage in Dockerfile for improved modularity
* docs: reorder build steps in contributing guide for clarity
|
||
|
|
b51cd21b3c
|
📦 refactor: Move DB Models to @librechat/data-schemas (#6210)
* 🚀 feat: Introduce data schemas and refactor models to use @librechat/data-schemas * 🚀 feat: Add installation step for Data Schemas Package in backend review workflow * chore: Add `data-schemas` package to update/rebuild packages scripts * chore: Update Dockerfile to include data-schemas package build process * fix: add missing @rollup/plugin-typescript package * chore: Add GitHub Actions workflow for publishing data-schemas package --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
7f6b32ff04
|
🖼️ refactor: Enhance Env Extraction & Agent Image Handling (#6131)
* refactor: use new image output format for agents using DALL-E tools * refactor: Enhance image fetching with proxy support and adjust logging placement in DALL-E 3 integration * refactor: Enhance StableDiffusionAPI to support agent-specific return values and display message for generated images * refactor: Add unit test execution for librechat-mcp in backend review workflow * refactor: Update environment variable extraction logic, export from serpate module to avoid circular refs, and remove deprecated tests * refactor: Add unit tests for environment variable extraction and enhance StdioOptionsSchema to process env variables |
||
|
|
86134415e9
|
🧹 chore: Migrate to Flat ESLint Config & Update Prettier Settings (#5737)
* chore: migrated eslint v8 to v9 * chore: migrated eslint v8 to v9 * ESLint only checks the files that have changed in the pull request. * fix: ESLint only checks the files that have changed in the pull request. * refactor: eslint only on changed files * refactor: eslint only on changed files or added files * refactor: eslint only on changed files or added files * refactor: eslint only on changed files or added files but only include files that are not deleted (ACMRTUXB: A, C, M, R, T, U, X, B). * whoops missed something |
||
|
|
e391347b9e
|
🔧 feat: Initial MCP Support (Tools) (#5015)
* 📝 chore: Add comment to clarify purpose of check_updates.sh script
* feat: mcp package
* feat: add librechat-mcp package and update dependencies
* feat: refactor MCPConnectionSingleton to handle transport initialization and connection management
* feat: change private methods to public in MCPConnectionSingleton for improved accessibility
* feat: filesystem demo
* chore: everything demo and move everything under mcp workspace
* chore: move ts-node to mcp workspace
* feat: mcp examples
* feat: working sse MCP example
* refactor: rename MCPConnectionSingleton to MCPConnection for clarity
* refactor: replace MCPConnectionSingleton with MCPConnection for consistency
* refactor: manager/connections
* refactor: update MCPConnection to use type definitions from mcp types
* refactor: update MCPManager to use winston logger and enhance server initialization
* refactor: share logger between connections and manager
* refactor: add schema definitions and update MCPManager to accept logger parameter
* feat: map available MCP tools
* feat: load manifest tools
* feat: add MCP tools delimiter constant and update plugin key generation
* feat: call MCP tools
* feat: update librechat-data-provider version to 0.7.63 and enhance StdioOptionsSchema with additional properties
* refactor: simplify typing
* chore: update types/packages
* feat: MCP Tool Content parsing
* chore: update dependencies and improve package configurations
* feat: add 'mcp' directory to package and update configurations
* refactor: return CONTENT_AND_ARTIFACT format for MCP callTool
* chore: bump @librechat/agents
* WIP: MCP artifacts
* chore: bump @librechat/agents to v1.8.7
* fix: ensure filename has extension when saving base64 image
* fix: move base64 buffer conversion before filename extension check
* chore: update backend review workflow to install MCP package
* fix: use correct `mime` method
* fix: enhance file metadata with message and tool call IDs in image saving process
* fix: refactor ToolCall component to handle MCP tool calls and improve domain extraction
* fix: update ToolItem component for default isInstalled value and improve localization in ToolSelectDialog
* fix: update ToolItem component to use consistent text color for tool description
* style: add theming to ToolSelectDialog
* fix: improve domain extraction logic in ToolCall component
* refactor: conversation item theming, fix rename UI bug, optimize props, add missing types
* feat: enhance MCP options schema with base options (iconPath to start) and make transport type optional, infer based on other option fields
* fix: improve reconnection logic with parallel init and exponential backoff and enhance transport debug logging
* refactor: improve logging format
* refactor: improve logging of available tools by displaying tool names
* refactor: improve reconnection/connection logic
* feat: add MCP package build process to Dockerfile
* feat: add fallback icon for tools without an image in ToolItem component
* feat: Assistants Support for MCP Tools
* fix(build): configure rollup to use output.dir for dynamic imports
* chore: update @librechat/agents to version 1.8.8 and add @langchain/anthropic dependency
* fix: update CONFIG_VERSION to 1.2.0
|
||
|
|
0ae98ff011
|
🧪 ci: Add .env.test for backend-review.yml (#2501)
* 🧪 ci: Add `.env.test` for `backend-review.yml`
* chore: touch example file
|
||
|
|
9bab595204
|
🔬 chore: Add Circular Dependency Check to backend-review (#2149)
* 🔬 chore: Add Circular Dependency check to `backend-review`
* chore: touch random file for workflow trigger
* chore: workflow step order
* chore: update workflow to create empty auth.json file
* fix: attempt empty auth.json creation
* chore: add test_bundle ESLint ignore pattern
|
||
|
|
ecd63eb9f1
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header * feat: assistname in textarea placeholder, and use `Assistant` as default name * feat: display non-image files in user messages * fix: only render files if files.length is > 0 * refactor(config -> file-config): move file related configuration values to separate module, add excel types * chore: spreadsheet file rendering * fix(Landing): dark mode style for Assistant Name * refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\) * refactor(useContentHandler): add empty Text part if last part was completed tool or image * chore: add accordion trigger border styling for dark mode * feat: Assistant Builder model selection * chore: use Spinner when Assistant is mutating * fix(get/assistants): return correct response object `AssistantListResponse` * refactor(Spinner): pass size as prop * refactor: make assistant crud mutations optimistic, add types for options * chore: remove assistants route and view * chore: move assistant builder components to separate directory * feat(ContextButton): delete Assistant via context button/dialog, add localization * refactor: conditionally show use and context menu buttons, add localization for create assistant * feat: save side panel states to localStorage * style(SidePanel): improve avatar menu and assistant select styling for dark mode * refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely * fix: resize handle and navToggle behavior * fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image * WIP: file UI components from PR #576 * refactor(OpenAIMinimalIcon): pass className * feat: formatDate helper fn * feat: DataTableColumnHeader * feat: add row selection, formatted row values, number of rows selected * WIP: add files to Side panel temporarily * feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest` * fix(TFile): correct `source` type with `FileSources` * fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile * chore: add generic setter type * refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors * feat: `useFileDeletion` to reuse file deletion logic * refactor(useFileDeletion): make `setFiles` an optional param and use object as param * feat: useDeleteFilesFromTable * feat: use real `files` data and add deletion action to data table * fix(Table): make headers sticky * feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type * style(DropdownMenu): replace `slate` with `gray` * style(DataTable): apply dark mode themes and other misc styling * style(Columns): add color to OpenAI Storage option * refactor(FileContainer): make file preview reusable * refactor(Images): make image preview reusable * refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style * feat(Columns): add file/image previews, set a minimum size to show for file size in bytes * WIP: File Panel with real files and formatted * feat: open files dialog from panel * style: file data table mobile and general column styling fixes * refactor(api/files): return files sorted by the most recently updated * refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns * refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type * feat: attach files through file panel * refactor(useFileHandling): move files to the start of cache list when uploaded * refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server * fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file * style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button * feat: action utilities and tests * refactor(actions): add `ValidationResult` type and change wording for no server URL found * refactor(actions): check for empty server URL * fix(data-provider): revert tsconfig to fix type issue resolution * feat(client): first pass of actions input for assistants * refactor(FunctionSignature): change method to output object instead of string * refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant` * feat: post actions input first pass - create new Action document - add actions to Assistant DB document - create /action/:assistant_id POST route - pass more props down from PanelSwitcher, derive assistant_id from switcher - move privacy policy to ActionInput - reset data on input change/validation - add `useUpdateAction` - conform FunctionSignature type to FunctionTool - add action, assistant doc, update hook related types * refactor: optimize assistant/actions relationship - past domain in metadata as hostname and not a URL - include domain in tool name - add `getActions` for actions retrieval by user - add `getAssistants` for assistant docs retrieval by user - add `assistant_id` to Action schema - move actions to own module as a subroute to `api/assistants` - add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks - fix Action type def * feat: show assistant actions in assistant builder * feat: switch to actions on action click, editing action styling * fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder * refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed * chore: rename `processActions` -> `processRequiredActions` * chore: rename Assistant API Action to RequiredAction * refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage * fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants * WIP: working first pass of toolCalls generated from openapi specs * WIP: first pass ToolCall styling * feat: programmatic iv encryption/decryption helpers * fix: correct ActionAuth types/enums, and define type for AuthForm * feat: encryption/decryption helpers for Action AuthMetadata * refactor(getActions): remove sensitive fields from query response * refactor(POST/actions): encrypt and remove sensitive fields from mutation response * fix(ActionService): change ESM import to CJS * feat: frontend auth handling for actions + optimistic update on action update/creation * refactor(actions): use the correct variables and types for setAuth method * refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields * refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not * refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter * fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call * feat: delete assistant actions * refactor: conditionally show Available actions * refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox` * chore: remove shadow-stroke from messages * WIP: first pass of Assistants Knowledge attachments * refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist * fix: prevent deleting files on assistant file upload * chore: remove console.log * refactor(useUploadFileMutation): update files and assistants cache on upload * chore: disable oauth option as not supported yet * feat: cancel assistant runs * refactor: initialize OpenAI client with helper function, resolve all related circular dependencies * fix(DALL-E): initialization * fix(process): openai client initialization * fix: select an existing Assistant when the active one is deleted * chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant * fix: add assistant_id on newConvo * fix(initializeClient): import fix * chore: swap setAssistant for setOption in useEffect * fix(DALL-E): add processFileURL to loadTools call * chore: add customConfig to debug logs * feat: delete threads on convo delete * chore: replace Assistants icon * chore: remove console.dir() in `abortRun` * feat(AssistantService): accumulate text values from run in openai.responseText * feat: titling for assistants endpoint * chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files * refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value * chore: disable `import from url` action for now * chore: remove textMimeTypes from default fileConfig for now * fix: catch tool errors and send as outputs with error messages * fix: React warning about button as descendant of button * style: retrieval and cancelled icon * WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name * fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing * ci: fix dall-e tests * ci: fix client tests * fix: button types in actions panel * fix: plugin auth form persisting across tool selections * fix(ci): update AppService spec with `loadAndFormatTools` * fix(clearConvos): add id check earlier on * refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes * feat(assistants): addTitle cache setting * fix(useSSE): resolve rebase conflicts * fix: delete mutation * style(SidePanel): make grip visible on active and hover, invisible otherwise * ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig * fix: handle edge case where auth object is undefined, and log errors * refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests * chore: remove comment * fix(ActionsInput): re-render bug when initializing states with action fields * fix(patch/assistant): filter undefined tools * chore: add logging for errors in assistants routes * fix(updateAssistant): map actions to functions to avoid overwriting * fix(actions): properly handle GET paths * fix(convos): unhandled delete thread exception * refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService * refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages * refactor(assistants/chat): invoke `createOnTextProgress` after thread creation * chore: add typing * style: sidepanel styling * style: action tool call domain styling * feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example * feat: assistants api key in EndpointService * refactor: set assistant model to conversation on assistant switch * refactor: set assistant model to conversation on assistant select from panel * fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging * feat: retrieval styling, handling, and logging * chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL * feat: FileContext for file metadata * feat: context file mgmt and filtering * style(Select): hover/rounded changes * refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint * fix(AssistantAvatar): make empty previewURL if no avatar present * refactor: side panel mobile styling * style: merge tool and action section, optimize mobile styling for action/tool buttons * fix: localStorage issues * fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time * style: light mode fixes * fix: prevent sidepanel nav styling from shifting layout up * refactor: change default layout (collapsed by default) * style: mobile optimization of DataTable * style: datatable * feat: client-side hide right-side panel * chore(useNewConvo): add partial typing for preset * fix(useSelectAssistant): pass correct model name by using template as preset * WIP: assistant presets * refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors * refactor: organize imports and use native TavilySearchResults * fix(TavilySearchResults): stringify result * fix(ToolCall): show tool call outputs when not an action * chore: rename Prompt Prefix to custom instructions (in user facing text only) * refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture * feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only) * feat: Assistant preset settings (client-side) * fix(Switcher): only set assistant_id and model if current endpoint is Assistants * feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting * feat(Assistants): add instructions field to settings * feat(chat/assistants): pass conversation settings to run body * wip: begin localization and only allow actions if the assistant is created * refactor(AssistantsPanel): knowledge localization, allow tools on creation * feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined * chore: trim console logs and make more meaningful * chore: toast messages * fix(ci): date test * feat: create file when uploading Assistant Avatar * feat: file upload rate limiting from custom config with dynamic file route initialization * refactor: use file upload limiters on post routes only * refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests * refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook * feat: supportedMimeTypesSchema, test for array of regex * feat: configurable file config limits * chore: clarify assistants file knowledge prereq. * chore(useTextarea): default to localized 'Assistant' if assistant name is empty * feat: configurable file limits and toggle file upload per endpoint * fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment * fix(AssistantSelect): set last selected assistant only when timeout successfully runs * refactor(queries): disable assistant queries if assistants endpoint is not enabled * chore(Switcher): add localization * chore: pluralize `assistant` for `EModelEndpoint key and value * feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals * fix(compactEndpointSchemas): use EModelEndpoint for schema access * feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout` * fix: naming issue * wip: revert landing * 🎉 happy birthday LibreChat (#1768) * happy birthday LibreChat * Refactor endpoint condition in Landing component * Update birthday message in Eng.tsx * fix(/config): avoid nesting ternaries * refactor(/config): check birthday --------- Co-authored-by: Danny Avila <messagedaniel@protonmail.com> * fix: landing * fix: landing * fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead * fix(ci): convo test revert to main * fix(assistants/chat): fix issue where assistant_id was being saved as model for convo * chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params * refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data * refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs * fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages * refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging * chore(typedefs): add httpAgent type to OpenAIClient * refactor: use custom implementation of retrieving run with axios to allow for timing out run query * fix(waitForRun): handle timed out run retrieval query * refactor: update preset conditions: - presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch - no longer use `navigateToConvo` on preset select * fix: temporary calculator hack as expects string input when invoked * fix: cancel abortRun only when cancelling error is a result of the run already being cancelled * chore: remove use of `fileMaxSizeMB` and total counterpart (redundant) * docs: custom config documentation update * docs: assistants api setup and dotenv, new custom config fields * refactor(Switcher): make Assistant switcher sticky in SidePanel * chore(useSSE): remove console log of data and message index * refactor(AssistantPanel): button styling and add secondary select button to bottom of panel * refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client * feat(assistants): token tracking for assistant runs * chore(spendTokens): improve logging * feat: support/exclude specific assistant Ids * chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging * chore: mount docker logs to root of project * chore: condense axios errors * chore: bump vite * chore: vite hot reload fix using latest version * chore(getOpenAIModels): sort instruct models to the end of models list * fix(assistants): user provided key * fix(assistants): user provided key, invalidate more queries on revoke --------- Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com> |
||
|
|
49571ac635
|
chore: Get the latest of all github actions (#1335) | ||
|
|
70590251d1
|
chore: add back BrowserOp, make changes to CI env (#1088)
* chore: add back BrowserOp * chore: make CI env and not DEV env generate refresh tokens every time * chore: make 'CI' env var captilization uniform across the app * chore: change NODE_ENV for playwright to |
||
|
|
7b2cedf5ff
|
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903)
* refactor: require Auth middleware in route index files * feat: concurrent message limiter * feat: complete concurrent message limiter with caching * refactor: SSE response methods separated from handleText * fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body * chore: minor name changes * refactor: add isUUID condition to saveMessage * fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization * chore: bump keyv and remove console.log from Message * fix(concurrentLimiter): ensure messages are only saved in later message children * refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future * feat: add denyRequest function for error responses * feat(utils): add isStringTruthy function Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true' * feat: add optional message rate limiters by IP and userId * feat: add optional message rate limiters by IP and userId to edit route * refactor: rename isStringTruthy to isTrue for brevity * refactor(getError): use map to make code cleaner * refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation * feat: check if errorMessage is object, stringify if so * chore: send object to denyRequest which will stringify it * feat: log excessive requests * fix(getError): correctly pluralize messages * refactor(limiters): make type consistent between logs and errorMessage * refactor(cache): move files out of lib/db into separate cache dir >> feat: add getLogStores function so Keyv instance is not redundantly created on every violation feat: separate violation logging to own function with logViolation * fix: cache/index.js export, properly record userViolations * refactor(messageLimiters): use new logging method, add logging to registrations * refactor(logViolation): make userLogs an array of logs per user * feat: add logging to login limiter * refactor: pass req as first param to logViolation and record offending IP * refactor: rename isTrue helper fn to isEnabled * feat: add simple non_browser check and log violation * fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch * chore: adjust nodemon ignore paths to properly ignore logs * feat: add math helper function for safe use of eval * refactor(api/convos): use middleware at top of file to avoid redundancy * feat: add delete all static method for Sessions * fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case) * refactor(getLogStores): adjust return type * feat: add ban violation and check ban logic refactor(logViolation): pass both req and res objects * feat: add removePorts helper function * refactor: rename getError to getMessageError and add getLoginError for displaying different login errors * fix(AuthContext): fix type issue and remove unused code * refactor(bans): ban by ip and user id, send response based on origin * chore: add frontend ban messages * refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy * feat: add ban check to AI messaging routes * feat: add ban check to login/registration * fix(ci/api): mock KeyvMongo to avoid tests hanging * docs: update .env.example > refactor(banViolation): calculate interval rate crossover, early return if duration is invalid ci(banViolation): add tests to ensure users are only banned when expected * docs: improve wording for mod system * feat: add configurable env variables for violation scores * chore: add jsdoc for uaParser.js * chore: improve ban text log * chore: update bun test scripts * refactor(math.js): add fallback values * fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead refactor(getLogStores): get a single log store by type * fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0 * fix(banViolation.js): getLogStores import * feat: handle 500 code error at login * fix(middleware): handle case where user.id is _id and not just id * ci: add ban secrets for backend unit tests * refactor: logout user upon ban * chore: log session delete message only if deletedCount > 0 * refactor: change default ban duration (2h) and make logic more clear in JSDOC * fix: login and registration limiters will now return rate limiting error * fix: userId not parsable as non ObjectId string * feat: add useTimeout hook to properly clear timeouts when invoking functions within them refactor(AuthContext): cleanup code by using new hook and defining types in ~/common * fix: login error message for rate limits * docs: add info for automated mod system and rate limiters, update other docs accordingly * chore: bump data-provider version |
||
|
|
afd43afb60
|
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808)
* feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function
fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId
feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function
chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum
* wip: add generation buttons
* refactor(cleanupPreset.ts): simplify cleanupPreset function
refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function
feat(utils): add getDefaultConversation function
This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters.
The `getDefaultConversation` function takes in an object with the following properties:
- `conversation`: The conversation object to be used as a base.
- `endpointsConfig`: The configuration object containing information about the available endpoints.
- `preset`: An optional preset object that can be used to override the default behavior.
The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint.
Once the target endpoint is determined,
* fix(utils): remove console.error statement in buildDefaultConversation function
fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema
* fix: endpoint not changing on change of preset from other endpoint, wip: refactor
* refactor: preset items to TSX
* refactor: convert resetConvo to TS
* refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability
refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability
* feat(svg): add ContinueIcon component
feat(svg): add RegenerateIcon component
feat(svg): add ContinueIcon and RegenerateIcon components to index.ts
* feat(Button.tsx): add onClick and className props to Button component
feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages
feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function
feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function
* fix(TextChat.jsx): reorder imports and variables for better readability
fix(TextChat.jsx): fix typo in condition for isNotAppendable variable
fix(TextChat.jsx): remove unused handleStopGenerating function
fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements
fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions
fix(useMessageHandler.ts): remove unused variables in return statement
* fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function
feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage
* fix(OpenAIClient.js): add support for streaming result in sendCompletion method
feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method
feat(Message.js): add finish_reason field to Message model
feat(messageSchema.js): add finish_reason field to messageSchema
feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption
feat(openAI.js): add addMetadata function to store metadata in ask function
feat(openAI.js): add metadata to response if available
feat(schemas.ts): add finish_reason field to tMessageSchema
* feat(types.ts): add TOnClick and TGenButtonProps types for button components
feat(Continue.tsx): create Continue component for generating button
feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component
feat(Regenerate.tsx): create Regenerate component for regenerating button
feat(Stop.tsx): create Stop component for stop generating button
* feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations
fix(Root.jsx): fix import paths for Nav and MessageHandler components
* feat(useMessageHandler.ts): add support for generation parameter in ask function
feat(useMessageHandler.ts): add support for isEdited parameter in ask function
feat(useMessageHandler.ts): add support for continueGeneration function
fix(createPayload.ts): replace endpoint URL when isEdited parameter is true
* chore(client): set skipLibCheck to true in tsconfig.json
* fix(useMessageHandler.ts): remove unused clientId variable
fix(schemas.ts): make clientId field in tMessageSchema nullable and optional
* wip: edit route for continue generation
* refactor(api): move handlers to root of routes dir
* fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null
fix(useMessageHandler.ts): update initialResponse text to use responseText variable
fix(useMessageHandler.ts): update setMessages logic for isRegenerate case
fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler
* fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString()
fix(schemas.ts): change type annotation of TMessage from infer to input
* refactor(useMessageHandler.ts): rename AskProps type to TAskProps
refactor(useMessageHandler.ts): remove generation property from ask function arguments
refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||)
refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission
* fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values
* fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method
feat(BaseClient.js): add support for isEdited flag in sendMessage method
feat(BaseClient.js): add generation to responseMessage text in sendMessage method
* fix(openAI.js): remove unused imports and commented out code
feat(openAI.js): add support for generation parameter in request body
fix(openAI.js): remove console.log statement
fix(openAI.js): remove unused variables and parameters
fix(openAI.js): update response text in case of error
fix(openAI.js): handle error and abort message in case of error
fix(handlers.js): add generation parameter to createOnProgress function
fix(useMessageHandler.ts): update responseText variable to use generation parameter
* refactor(api/middleware): move inside server dir
* refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils
* fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models
feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests
fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI
refactor(openAI.js): use getAbortData function to get data for abortAsk function
* refactor: move endpoint specific logic to an endpoints dir
* refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js
* feat(openAI): add new endpoint for adding a title to a conversation
- Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory.
- The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions:
- If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps:
- Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters.
- Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details.
- Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function.
- This change adds
* fix(abortMiddleware.js): remove console.log statement
refactor(gptPlugins.js): update imports and function parameters
feat(gptPlugins.js): add support for abortController and getAbortData
refactor(openAI.js): update imports and function parameters
feat(openAI.js): add support for abortController and getAbortData
fix(openAI.js): refactor code to use modularized functions and middleware
fix(buildOptions.js): refactor code to use destructuring and update variable names
* refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers
feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers
* feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption
* fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object
fix(abortMessage.js): remove console.log statement for aborted message
fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space
* fix(BaseClient.js): import addSpaceIfNeeded function from server/utils
fix(BaseClient.js): add space before generation in text property
fix(index.js): remove getCitations and citeText exports
feat(buildEndpointOption.js): add buildEndpointOption middleware
fix(index.js): import buildEndpointOption middleware
fix(anthropic.js): remove buildOptions function and use endpointOption from req.body
fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body
fix(openAI.js): remove buildOptions function and use endpointOption from req.body
feat(utils): add citations.js and handleText.js modules
fix(utils): fix import statements in index.js module
* refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider
* feat(gptPlugins): complete 'continue generating'
* wip: anthropic continue regen
* feat(middleware): add validateRegistration middleware
A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed.
* feat(Anthropic): complete continue regen
* chore: add librechat-data-provider to api/package.json
* fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed
* chore(ci): remove unneeded SEARCH env var
* style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com
* style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size
* chore(client): 'Editting' typo
* feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component
feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component
feat(useGenerations.ts): create useGenerations hook to handle generation logic
fix(schemas.ts): add searchResult field to tMessageSchema
* refactor(HoverButtons): convert to TSX and utilize new useGenerations hook
* fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging
* refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas
* chore: remove console.logs from test files
* ci: add backend tests for AnthropicClient, focusing on new buildMessages logic
* refactor(FakeClient): use actual BaseClient sendMessage method for testing
* test(BaseClient.test.js): add test for loading chat history
test(BaseClient.test.js): add test for sendMessage logic with isEdited flag
* fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object
wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true
* fix(Button.tsx): add data-testid attribute to button component
fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component
fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response
feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing
* refactor(onProgress): speed up time to save initial message for editable routes
* chore: disable AI message editing (for now), was accidentally allowed
* refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now)
* fix: add test id to generation buttons so they never resolve to 2+ items
* chore(package.json): add 'packages/' to the list of ignored directories
chore(data-provider/package.json): bump version to 0.1.5
|
||
|
|
c6f5d5d65c
|
ci: add e2e workflow, optimize client code for testing (#771)
* refactor(e2e): fix tests with latest changes, convert to TS, use test Ids * chore(EndpointMenu.jsx): add data-testid attribute to new-conversation-menu button * refactor(EndpointItem): add data-testid attr., convert to TS * refactor(e2e): remove unnecessary awaits and convert to TS * chore(playwright.config.local.ts): add absolute path to server index.js file chore(playwright.config.local.ts): add dotenv configuration chore(playwright.config.local.ts): change webServer command to use absolute path chore(playwright.config.local.ts): add NODE_ENV and process.env to webServer env chore(playwright.config.local.ts): remove unused import chore(login.spec.js): delete login.spec.js file * chore(.gitignore): add 'my.secrets' to the list of ignored files fix(Registration.tsx): add 'data-testid' attribute to the error message div fix(Registration.spec.tsx): comment out test case that calls 'registerUser.mutate' * chore(ConvoIcon.tsx): add data-testid attribute to svg element chore(messages.spec.ts): refactor conversation navigation logic * chore(playwright.config.ts): add support for absolute path to server index.js file feat(playwright.config.ts): add support for dotenv configuration feat(playwright.config.ts): set NODE_ENV to 'production' in webServer environment variables * chore(workflows): comment out push event and specify paths for pull_request event in backend-review.yml chore(workflows): comment out push event and specify paths for pull_request event in frontend-review.yml * chore(install.js): add check to skip install script in CI environment * chore: complete playwright workflow * chore(Landing.tsx): add data-testid attribute to landing title element chore(authenticate.ts): update selector to wait for landing title element by test id instead of text content * chore(playwright.yml): add step to upload screenshot artifact on failure fix(authenticate.ts): capture screenshot before waiting for landing title and increase timeout due to GH Actions load time * chore(playwright.yml): rename artifact name from 'screenshot' to 'login-screenshot' feat(LoginForm.tsx): add data-testid attribute to login button fix(authenticate.ts): change screenshot name to 'login-screenshot.png' and conditionally take screenshot only in CI environment * chore(playwright.yml): add CI environment variable and set it to true * chore(playwright.yml): update Playwright installation command chore(playwright.config.ts): update storageState path to use process.cwd() * fix(playwright.yml): update node version to 18 in setup-node action fix(playwright.yml): update actions/cache to v3 in Cache Node.js modules step fix(playwright.yml): update actions/cache to v3 in Cache Playwright installations step fix(authenticate.ts): change login button click to press 'Enter' on password input * chore(playwright.yml): update E2E_USER_EMAIL and E2E_USER_PASSWORD values for testing purposes chore(authenticate.ts): add console.dir to log user object for debugging * chore(playwright.yml): add step to upload storageState artifact The storageState artifact is now uploaded as part of the workflow. This artifact contains the state of the storage used during the end-to-end tests. It will be retained for 2 days. * chore(playwright.yml): comment out upload screenshot step chore(playwright.config.ts): change NODE_ENV to development chore(authenticate.ts): comment out screenshot related code * chore(playwright.config.ts): add SESSION_EXPIRY environment variable with value 86400000 * chore(playwright.yml): update environment variables in Playwright workflow fix(General.tsx): add data-testid attributes to clear conversations buttons test(messages.spec.ts): add setup and teardown steps for clearing conversations before and after tests * fix(messages.spec.ts): fix clearing conversations before and after message tests feat(messages.spec.ts): add beforeEach and afterEach hooks to create and close new page for each test * chore: remove storageStage upload artifact |
||
|
|
600a0d15b1
|
fix(backend-review.yml): update Node.js version from 19.x to 20.x (#767)
fix(frontend-review.yml): update Node.js version from 19.x to 20.x |