Commit graph

19 commits

Author SHA1 Message Date
Marco Beretta
e36598f019
⏩ ci: Record Playwright Shard Timings Automatically and Run Fifteen Shards (#18115)
* ci: Record Playwright Shard Timings From Green Dev Runs
* ci: Run the Playwright Mock Suite on Fifteen Memory Shards
* ci: Run the Playwright Workflow Script Tests Once per Run
* test: Make the Due Schedule E2E Wait for One Engine Tick Only
* test: Cover the Shard Timings Hand-off With Acceptance Scenarios
2026-11-14 01:25:48 +01:00
Danny Avila
fb4150e077
🧪 test: Route the Benchmark and BYOM Unroutes Through the Shared Helper (#18085) 2026-11-12 09:47:24 -05:00
Danny Avila
adcb19fff8
🔌 test: Keep Harness Connections Open for Playwright's Idle Sockets (#18070)
Playwright sends route.fetch(), the request fixture and page.request through one
keep-alive agent per worker with no idle timeout, and Node's agent ignores a
server's Keep-Alive hint without one. LibreChat and the fixture servers closed
idle sockets after 6 s, so a request reusing a socket at that moment was reset
(read ECONNRESET) or dropped (socket hang up): the remaining retried passes in
wrapping.spec.ts and thread-fold.spec.ts.

Run LibreChat with HTTP_KEEP_ALIVE_TIMEOUT_MS=0 and pass every fixture server
and the replica proxy through keepIdleConnections(), and check in
fixtures.spec.ts that each harness server answers without an idle deadline.
2026-11-12 07:19:46 -05:00
Danny Avila
876d212e1a
🧪 test: Keep Handlers Answering When a Mock Spec Removes a Route (#18069)
Some checks failed
Codegraph E2E Vote Gate / Gate (push) Waiting to run
Codegraph E2E Votes / vote (shard 1/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 10/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 2/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 3/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 4/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 5/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 6/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 7/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 8/10) (push) Waiting to run
Codegraph E2E Votes / vote (shard 9/10) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Frontend Unit Tests / frontend (push) Blocked by required conditions
Playwright E2E Tests / Codegraph select (push) Waiting to run
Playwright E2E Tests / e2e () (push) Blocked by required conditions
Playwright E2E Tests / MCP list_changed (replica count 1) (push) Blocked by required conditions
Playwright E2E Tests / MCP list_changed (replica count 2) (push) Blocked by required conditions
Playwright E2E Tests / MCP Apps (browser and server) (push) Waiting to run
Playwright E2E Tests / e2e (push) Blocked by required conditions
Channel Gateway / gateway (push) Has been cancelled
Main Promotion Policy Tests / Main promotion denial and fast-forward tests (push) Has been cancelled
2026-11-11 19:17:26 -05:00
Danny Avila
bbd4117928
🧪 test: Release Route Handlers Before Mock Spec Contexts Close (#18064)
A route handler that calls `route.fetch()` can still be running when its test returns. Closing the browser context disposes the fetched response, the handler throws `Response has been disposed`, and Playwright reports that against a test whose assertions had all passed. With `failOnFlakyTests` on, that fails the shard.

Specs that answer requests through `route.fetch()` now take `test` from `e2e/specs/mock/fixtures.ts`. Its `context` fixture detaches the routes on the context and its pages with `unrouteAll({ behavior: 'ignoreErrors' })` before Playwright closes it; contexts a spec opens itself close through `closeContext`.

`ignoreErrors` rather than `wait`: a wait never ends for a handler parked on a promise the test leaves unsettled, and with two handlers in flight Playwright stops intercepting when the first finishes and continues the other request itself, so the second handler's `route.fulfill()` throws `Route is already handled!`.
2026-11-11 16:23:44 -05:00
Danny Avila
c6b7a5ac12
🎲 ci: Fail Playwright Runs When a Test Passes Only on a Retry (#18051)
Playwright reports a test that fails and then passes on a retry as "flaky" and exits 0, so a
regression that fails two attempts in three merges green. Set `failOnFlakyTests` in CI on the
mock profile (inherited by the Redis transport, registered-email and other derived profiles) and
on the base profile. Retry counts are unchanged: the later attempts still record the trace and
video and show whether a failure repeats.

A run failed this way still prints "N flaky" beside a passed count, so add a reporter that names
each retried pass in the job log and the job summary and says that it is what failed the run.
2026-11-11 10:31:06 -05:00
Marco Beretta
558e0144fe
⚖️ ci: Refresh Playwright Shard Timings From Recent Runs (#17765) 2026-11-04 11:21:59 +01:00
Gil Raphaelli
b2aeb9c6a1
feat: add masked session replay lifecycle (#17194) 2026-10-21 14:51:15 -04:00
Danny Avila
27e8930d52
⏱️ ci: Balance Playwright Shards by Recorded Duration (#17019)
Plan the complete mock suite's shards from recorded per-file durations instead of Playwright's equal-count split, widen the memory lanes from six to ten, and give the nightly full suite ten shards per store so it fits its 30-minute job timeout again.
2026-10-16 07:00:43 -04:00
Marco Beretta
c6e5f98552
⚙️ fix: Default Running Chat Rename to On (#16769)
interface.runningChatRename now defaults to true, so a running chat or a placeholder-title chat can be renamed without a librechat.yaml override. Set it to false during a rolling upgrade from a version without title ownership.
2026-10-05 06:47:22 -04:00
lia-by-librechat[bot]
19092e55bd
🖲️ fix: Enable Running Chat Menus and Preserve Renamed Titles (#16697)
Reconciles current dev while preserving manual title ownership, existing menus, and configuration regressions.

Co-authored-by: Lia <lia@librechat.ai>
2026-10-03 23:17:15 -04:00
Marco Beretta
4ffe3825ec 📧 feat: add email address change (#14597)
(cherry picked from commit 9ff251dd96)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/14597
2026-10-01 03:46:16 +00:00
lia-by-librechat[bot]
07e91ccdec
📷 test: Capture Revision-Pinned PR Screenshot Evidence (#16191)
Co-authored-by: Lia <lia@librechat.ai>
2026-09-22 06:32:31 -04:00
Danny Avila
50d46a38a5
📍 test: Add Deployed Smoke Profile (#16135)
* test: add deployed instance Playwright smoke profile

* test: normalize deployed smoke account state

* test: harden deployed smoke lifecycle

* test: secure deployed smoke cleanup
2026-09-20 12:34:02 -04:00
Danny Avila
ee21066590
🏎️ ci: Focus Redis E2E Coverage (#14842) 2026-08-14 12:54:30 -04:00
Ravi Kumar L
7cf4c3f73f
🧪 test(e2e): add Bombadil property exploration (#14462)
* test(e2e): add Bombadil property exploration

* fix(e2e): address Bombadil review feedback
2026-08-10 02:11:06 +02:00
Danny Avila
52b2ebf948
🧪 test: Run mock E2E against Redis in shards (#14551)
* 🧪 test: Run mock E2E against Redis in shards

* 🧪 test: Isolate local Redis E2E data
2026-07-31 12:10:43 -04:00
Danny Avila
a1bfa3b298
🎭 test: Run Mock E2E Suite Through createRun With In-Process Fake Model (#13508)
* 🎭 test: Run Mock E2E Suite Through createRun With In-Process Fake Model

Replace the standalone HTTP mock LLM server with an in-process fake model
injected into the real createRun -> Run.create pipeline via
run.Graph.overrideTestModel, so the mock suite exercises the agents
integration end-to-end without a live provider or a separate server.

- Bump @librechat/agents to 3.2.2 for the FakeChatModel/createFakeStreamingLLM exports
- Add an env-gated applyTestRunHook seam in packages/api createRun (no /api changes)
- Add e2e/setup/fake-model.js to drive default replies + the skill-authoring tool-call flow
- Drop the mock-llm webServer from playwright.config.mock.ts and set LIBRECHAT_TEST_RUN_HOOK

* 🧹 test: Retire Standalone Mock LLM Server From E2E Recorder

Migrate the `--profile=mock` recorder onto the same in-process fake model
as the Playwright mock suite, then delete the now-unused HTTP mock server
so the fake-LLM logic lives in a single place.

- Point record.js mock profile at the fake model via LIBRECHAT_TEST_RUN_HOOK
- Remove the mock-llm-server spawn/wait and MOCK_LLM_PORT plumbing from record.js
- Delete e2e/setup/mock-llm-server.js (e2e/setup/fake-model.js is now the only source)
- Update e2e/README.md to describe the in-process fake LLM

* 🏷️ ci: Rename Playwright Mock E2E Check to Playwright E2E Tests
2026-06-04 08:33:28 -04:00
Teresa Blanco
b45e4aeae5
🎭 feat: Add Credential-Free Playwright Smoke Suite with a Local Mock LLM (#13472)
* 🧪 feat: add e2e playwright tests

* 🧪 feat: Add Playwright Recording Harness

* test: fix mock playwright config

* test: harden mock e2e environment

* test: preserve mock dotenv secrets

* test: harden mock isolation setup

* ci: cache mock e2e builds

* test: harden e2e cache and recorder checks

* test: preserve data-provider exports in oauth route test

* test: isolate mock auth logout state

* test: allow isolated logout smoke setup

* test: prepare logout smoke auth via api

* test: isolate oauth route module mock

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-06-02 16:36:39 -04:00