Commit graph

6 commits

Author SHA1 Message Date
Danny Avila
445668a371
🧭 ci: Assert Lighthouse Blocking Time at a Reference CPU on the Quietest Run (#18406)
Total blocking time is CPU time, and the hosted runner's CPU is not a constant: Lighthouse's
own benchmark of the runner moves by more than 2x between consecutive runs of one build on a
two-core runner, and blocking time moves with it. A median of three raw measurements against
a fixed budget therefore fails on the runner rather than on the page.

Rescale each run's main-thread tasks by that run's benchmark index to a reference CPU, assert
the lowest run instead of the median, and measure an over-budget batch once more before
failing. Budgets move to their own file so a fork with a heavier client changes numbers only.
2026-11-18 07:47:01 -05:00
Danny Avila
cbd4cbb5fb
🧲 fix: Settle Fast Pinned Drops, Defer the Hidden Mobile Header Menu, and Close Untraced E2E Races (#18087)
* 🧪 test: Read the Artifact Scrim's Blur From the Persistent Scrim

"a small blur surface still blurs by four pixels" read the backdrop filter
of `#artifact-viewer [role="status"]`.first() once, with a single evaluate.

The artifact panel's lazy tabs also render a `role="status"` loading
placeholder until their chunk arrives, and the panel re-renders while it
opens. A one-shot read that resolves the wrong status, or a node React has
just replaced, gets the computed style of a detached element: an empty
string instead of `blur(4px)`.

Select the always-mounted refresh scrim, the only status that carries
`aria-hidden`, and assert its filter with `toHaveCSS`, which re-resolves and
retries until the value settles.

Untraced, on two CPUs: 11 of 20 runs failed with `Received: ""` before,
20 of 20 pass after.

* ⚡ perf: Mount the Mobile Header Menu Only Once a Phone Layout Has Shown It

The chat header rendered both of its action surfaces on every load: the
desktop buttons and the mobile overflow menu, hidden with `className="hidden"`
on wide screens. The hidden menu still mounted its whole tree on desktop,
with its bookmark, export, share, temporary-chat and pull-request hooks and
its dialog providers.

Mount the overflow menu only while the drawer viewport is active, and keep
it mounted once it has been shown, so resizing back to desktop does not drop
a dialog opened from it.

Measured with a React DevTools hook counting component renders over a cold
desktop load of a saved conversation (7 warm navigations each, production
build): 1,313 component mounts before and 1,242 after, the overflow trigger
no longer in the DOM, and the median component re-renders over the load
5,835 before and 5,682 after.

`dropped-classes.spec.ts` read the overflow trigger's size on a desktop
viewport, where it was only attached; it now reads it at phone width, where
the trigger is visible.

* 🧲 fix: Settle a Pinned Row's Position on Drop, Not Only on Hover

A pinned row reorders in its `hover` callback, which judges the pointer
against the hovered row's midpoint using the offset react-dnd last recorded.
The HTML5 backend delivers hover once per animation frame, so a release that
lands in the same frame as the move across the midpoint is judged against the
offset from before it: the swap is refused and the drop keeps the old order.
A quick, deliberate drag is discarded.

The drop handler now runs the same placement against the final pointer offset
before returning its result, so `end` commits the order the pointer actually
ended on. Hovering is unchanged.

The new regression holds the hover frame and fires dragover/drop at a point
past the midpoint and at one short of it. On the previous handler the first
case keeps the old order; with this change both cases match the pointer.

In a real browser, with a drag that releases without waiting for a hover
frame: "a pinned chat cannot be dragged through a pinned model" kept the old
order in 4 of 15 untraced runs on the previous handler and 0 of 15 with this
one.

* 🧲 test: Release a Pinned Drag Without Waiting for a Hover Frame

The pinned drag helper waited for an animation frame after each settling
move, so react-dnd had delivered a hover at the final position before the
release. That kept the reorder scenario green while a release landing in the
same frame as its last move was discarded by the app.

The drop handler now settles the final pointer position itself, so that wait
no longer stands between the scenario and a failure; it only hides the
fast-release path. Release straight after the settling moves again, so the
scenario covers it.

Untraced, on two CPUs, with this helper: "a pinned chat cannot be dragged
through a pinned model" failed 4 of 15 runs against the previous drop handler
and passed 15 of 15 with the settled drop; the whole pinned-drag-rules file
passed 30 of 30.

* 🧪 test: Publish the Reaper Fixture's PID Only After It Is Written

Two reaper tests wait for a PID file to exist and then read it. The shell
creates the file when it opens the redirection and writes the PID a moment
later, so a read in between gets an empty file. `Number('')` is 0, and
`isGone(0)` reports true because there is no `/proc/0`: "escalates to
SIGKILL when the tree ignores SIGTERM" and "sweep reaps a group that
outlived a clean root exit" then fail their "still running" assertion.

Write the PID to a temporary file and rename it into place, so the file
appears only with its content. With a 0.3 s pause forced between the
redirection and the write, both tests fail on the old fixture and pass on
this one.

Also kill each spawned process group after every test, so a failing test
does not leave its trap-protected `sleep 30` behind, and read the process
state with `ps` where `/proc` does not exist.

* 🔦 test: Keep a Measured Trace and a Startup Profile When Lighthouse Fails

When the serial-latency audit fails a budget, the uploaded `.lighthouse/`
holds the JSON and HTML reports only. They name the slow metric but not the
main-thread work behind it, so a total-blocking-time failure cannot be traced
to the code that blocked without reproducing it locally.

- Run Lighthouse with `--save-assets`, keeping each run's measured
  `lhr-*-0.trace.json` for Chrome's Performance panel. Its raw DevTools
  network log is deleted; the trace is parsed and every string value has the
  session cookie values redacted, so numeric timings and JSON structure
  survive even for short cookie values. A trace that does not parse (an
  interrupted run) is deleted rather than uploaded raw.
- After a failed audit, navigate once more with the CDP profiler running and
  save `.lighthouse/startup.cpuprofile`, including three seconds after the
  transcript appears so late catalog callbacks are sampled. This runs after
  the measured audits, so it cannot change their numbers, and a profiling
  error never replaces the original failure.

Budgets, runs and the workflow are unchanged; the workflow already uploads
`.lighthouse/`.

* 🔦 test: Keep Lighthouse Traces Only When the Audit Fails

Each saved trace is about 45 MB, so keeping all three on every run turned the
always-uploaded `lighthouse` artifact from about 2 MB into about 137 MB for
runs that pass and need no diagnosis. Delete the traces once the test's last
assertion has passed; a failing run still uploads them with its reports.

* 🧪 test: Read the Streaming Shimmer From the Live Footer

"an in-flight label sweeps, and high contrast holds both stops opaque" read
the stream-elapsed label's styles once through `locator.evaluate`. The label
lives in the streaming footer, which re-renders while the reply streams; when
the node the locator resolved was replaced before the evaluate ran, the read
came from a detached element or failed.

Assert the animation name and background clip with `toHaveCSS`, which
re-resolves and retries, and read the shimmer's dip alpha by querying the
current node inside the evaluate, polling until it settles.

Untraced, on two CPUs: 4 of 20 runs failed with `Received: ""` before,
20 of 20 pass after.

* 🧪 test: Click the Drawer's New Chat Link, Not Whichever One Is Exposed

"returns focus after right-clicking an unopened running menu on a small
screen" opens the mobile drawer and clicks
`getByRole('link', { name: 'New chat', exact: true })`. On a saved chat the
header row also offers a New chat link, and it hides only when the sidebar's
expanded state commits. On mobile that commit is deferred: the drawer starts
sliding first and the state flips a few frames later inside a transition,
which a streaming run keeps pushing back. The drawer's Close button is
already visible during the slide, so the test reaches the click while both
links are exposed and fails with a strict mode violation.

Scope the locator to `#mobile-drawer`, the link the test means.

Untraced, on two CPUs: 20 of 20 runs failed on the previous locator and 20
of 20 pass with this one.

---------

Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
2026-11-12 09:47:46 -05:00
lia-by-librechat[bot]
21f9bb57c6
🧻 fix: Keep Lighthouse Session Cookies Out of Failure Logs (#17410)
* 🧻 fix: Keep Lighthouse Session Cookies Out of Failure Logs

* ci: Use Lockfile-Pinned Chromium for Lighthouse

* ci: Align Lighthouse Browser Launch With Playwright

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-10-24 11:45:22 +02:00
Marco Beretta
246e7e5de7
💠 fix: Paint Native Tool Verification Badges (#15862)
* 🪪 style: Give Native Tools a Painted Verified Mark

The native badge on a tool card was lucide's BadgeCheck filled with status-success-strong: green, the same hue the card already uses for selected, and stroked in currentColor, so the white check color also outlined the badge and left a white halo on any surface that is not the card's own white — hover, selection, dark mode.\n\nVerifiedIcon paints the badge instead of stroking it and carries the check as the only stroked path, and status-verified gives provenance its own blue across all four themes.

* 🩹 fix: Hold the Verified Mark at 3:1 on Every Card State

The dark fill was tuned against surface-secondary, but ToolCard repaints to surface-tertiary (#2f2f2f) on hover, where #0b74d4 fell to 2.85:1 — below the non-text floor, and invisible to the spec, which asserted the panel only. The fill moves to #1a7fd8 and the contract now covers every background the card takes. Both of the mark's relationships are graphical objects under WCAG 1.4.11, so the check is held at 3:1 rather than AA: no hue clears 3:1 on the hover surface and 4.5:1 under a white check at once.\n\nA theme that predates the token names status-success-strong and not status-verified, and would otherwise wear LibreChat's stock blue beside its own text-on-status and card surfaces. resolveTheme and the legacy mapColors adapter now carry the mark onto the fill it had before the token existed, following the six derived-token fallbacks already there.

* ✅ test: Exercise the Verified Mark in the Tool Library

Four scenarios in the mock harness, each named by the behavior it observes: the mark a native card wears, its silhouette under the dark hover repaint that the retune fixes, a pre-token deployment palette carrying it on the theme's own success fill, and a third-party MCP server card carrying no mark at all. The contrast assertions read the rendered colors rather than the tokens, so a regression in the component or in theme application fails them.

* 🧪 fix: Teach the Marketplace Dialog Spec About VerifiedIcon

The suite hand-lists the @librechat/client surface it mocks, so ToolCard's new icon resolved to undefined and every render in the file threw.

* 🧪 fix: Reach the Agent Builder on a Narrow Viewport

The scenario run covers mobile emulation, where the side rail is collapsed and the panel opens from the header's Control Panel toggle; the helper only knew the rail. The MCP scenario also minted its access token before the app had loaded, so the refresh request had no origin to resolve against.

* 🧪 fix: Run the Verified-Mark Scenarios on a Touch Viewport

Mobile emulation has no side rail: the panel switcher lives inside a sidebar that starts off-canvas, so the helper opens the sidebar and picks the Agent Builder entry from the switcher's menu. A touch viewport also has no hover state, so the hover scenario measures whatever background the card actually takes there and still holds the mark to 3:1 against it.

* 🩹 fix: Keep Pre-Token Themes on the Fill the Mark Inherited

The first fallback only fired when a theme restated status-success-strong, so a partial palette that customizes surfaces or text-on-status and inherits the green still landed on the stock blue. resolveTheme now derives the omission for any custom theme from the resolved success fill, and the legacy adapter takes the mode's bundled palette from ThemeProvider so it can do the same for a theme that names neither.\n\nThe scenario helper picked its path from a one-second probe for the desktop rail, which turns a slow first paint into the mobile branch; it now decides on viewport width and keeps the suite's full assertion timeout in each branch. The mock project is Desktop Chrome, so a touch scenario asks for a touch context explicitly rather than leaving that layout unexercised in CI.

* 🧪 fix: Match the App's Inclusive Layout Breakpoint

useSidebarState switches on (max-width: 768px), so a 768px-wide context renders the sidebar switcher while the helper's strict comparison went looking for the desktop rail.

* 🩹 fix: Scope the Verified Fallback to Themes That Painted the Mark

Two rounds pulled the same predicate in opposite directions: keying it on a theme restating status-success-strong missed palettes that only repaint the check or the card, and keying it on any custom color at all gave a new partial theme a green mark identical to the selected cue. Both are answered by asking what the theme actually painted — the fill the mark wore, the text-on-status check it carries, or the surfaces ToolCard takes — which is the same shape as the series-scale fallback above it. A theme that repaints none of those keeps the bundled blue.

* 🧪 fix: Wait for the Shell Before Reading the Sidebar State

isVisible() answers immediately, so a slow first paint read as an already-open drawer and the helper then waited on a switcher sealed inside it. It now waits for whichever of the opener and the switcher the shell paints first, which is the same assertion timeout the desktop branch gets.

* 📝 fix: Keep the Token Catalog Entry Catalog-Sized

The entry restated the icon's construction, the contrast rule and the pre-token fallback, all of which live on the token's own doc comment and in semanticTokens.spec.ts; the catalog around it is one line per role.

* 🔦 fix: Let a Crashed Lighthouse Run Be Retried

The audit shells out to Lighthouse three times and fails the lane if any single process exits nonzero, so a NO_NAVSTART — Lighthouse losing the trace of a page load, whose own message is "run Lighthouse again" — sinks a pull request that has nothing wrong with it. That is what happened on this branch: runs 1 and 2 wrote reports, run 3 crashed, and the identical head passed on rerun.\n\nEach run now gets a second attempt before the failure propagates. The repeat is per run, so the median is still taken over three completed reports, and a page that genuinely cannot be traced fails both attempts and still fails the lane.

* 🧪 fix: Read the Drawer State From Its Inert Marker

The readiness wait accepted the panel switcher, which is mounted whether or not the drawer is open — translated off-canvas and inert — so a shell that painted the sidebar before the chat header could satisfy it with the drawer closed and leave the helper clicking an off-viewport control. The drawer's own inert attribute is the state, so the helper waits for the drawer to attach and reads it there, with no timing probe on either side.

* 🧪 fix: Open the Drawer Before Trusting the Restored Form

A narrow context that had already selected the builder restores the form inside the closed drawer, and a translated element passes a visibility check, so the early probe accepted a form sitting under an inert drawer and handed it to the caller. The layout decision now comes first and the drawer is opened before anything inside it is read.
2026-09-12 16:03:03 -04:00
Danny Avila
c883e9ef32
🔦 chore: Drop @lhci/cli and Run Lighthouse 13 Directly (#15687)
`@lhci/cli@0.15.1` pins `lighthouse` to an exact `12.6.1`, so the Lighthouse
lane could never move off a release whose `puppeteer-core` still depends on
`extract-zip` — an advisory with no patched version. That pin, not Lighthouse
itself, was holding 8 of the repository's 16 `npm audit` findings open.

LHCI was only ever a subprocess wrapper here: `audit.ts` shelled out to
`@lhci/cli/src/cli.js` for `collect` and `assert`, and LHCI's own node runner
shells out to `lighthouse/cli/index.js` in turn. Call that CLI directly, run
the three navigations in a loop, and assert the median budgets in TypeScript
next to the assertions `load.spec.ts` already makes.

- `npm audit`: 16 findings (7 high, 1 moderate, 8 low) -> 6 low. The remainder
  is the pre-existing `elliptic` chain under `vite-plugin-node-polyfills`,
  which has no fixed version and is unrelated to this lane.
- Lighthouse 13 removed `largest-contentful-paint-element` and moved the LCP
  node into `lcp-breakdown-insight`. Reading it through a `lcpElement` helper
  keeps the "the transcript must be the LCP element" guard working and makes
  the next rename fail loudly in one place.
- Reports move to `.lighthouse/` as `lhr-N.report.{json,html}`; cookie
  redaction, the API timing table and the desktop/provided-throttling settings
  are unchanged. Budgets are now printed as a table immediately before they are
  asserted, so the workflow's 80-line failure comment always contains them.
- `CHROME_PATH` and `LIGHTHOUSE_CHROME_FLAGS` are documented: chrome-launcher
  prefers the Windows Chrome under WSL, whose debugging port Linux cannot
  reach.
2026-09-06 23:20:22 -04:00
Ravi Kumar L
de888cc362
🎈 ci: Gate Page Loads Under Database Latency (#15646)
Co-authored-by: Danny Avila <danny@librechat.ai>
2026-09-06 12:01:44 -04:00