LibreChat/e2e/benchmarks-fade/replay.perf.spec.ts
Danny Avila c3882c09ad
🧵 perf: Stop Re-rendering and Re-styling the Thread on Stream Events and Run Boundaries (#18148)
* 🎞️ perf: Run the In-Flight Label Sweep on the Compositor

The sweep on a running tool call, streaming reasoning and the elapsed timer
animated `background-position` under `background-clip: text`, which no
browser can hand to the compositor. Every frame of it ran style, paint and
commit on the main thread, and each paint walked the transcript around the
label, so the cost of a frame grew with the conversation and the sweep
stuttered whenever the main thread was busy.

`Shimmer` in `@librechat/client` draws the same sweep from two transforms
that cancel: a masked band slides across the label while a second copy of the
text inside it slides back, so the glyphs hold still and only the gap in the
mask moves. The copy is generated content, so the label is still one text
node. Every in-flight label renders through it.

`e2e/benchmarks-fade` measures a frame and each kind of stream event inside
an exported conversation, from a Chromium trace of the built client.

* 🧪 test: Let the Frame Benchmark Open a Chat in a Chosen Theme

* 🧵 perf: Keep the Composer, Nav Rail and Row Toolbars Out of Every Stream Event

A streamed delta rewrites one row, but each one also re-rendered the composer
form (and re-measured its textarea), the message nav rail (re-translating a
label per message), the sidebar's chat provider and the pending-steers row,
because each subscribed to the whole streaming message or the whole messages
array to read something a delta never changes.

- The chat contract and steering read a projection of the branch tail (where it
  sits, whether the run is paused) and fetch the message itself when a turn is
  built, through `useBranchTailGetter`. `ChatTurnsParams` carries
  `getLatestMessage` in place of `latestMessage`.
- The compact action and pending steers select what they use of the tail.
- The URL-prompt hook observes the history only while a submission waits on it.
- The nav rail's ribs are memoized apart from the rail.
- A row's toolbar keeps one element per glyph and stable handlers, so a send
  and its completion no longer re-render every button of every row.

`e2e/benchmarks-fade` gains a render-reason probe (`FADE_PERF_WHY`), a whole-run
profile, and windows for the send and its confirming frame.

* 🫥 perf: Conceal Rerun During a Run Instead of Unmounting It on Every Row

A run starting or ending withholds, then restores, the rerun control on every
row of the thread at once. Unmounting and remounting a button on each row made
the browser re-style the whole transcript both times: the stylesheet holds
`:has(:disabled)` rules and a message row is a `:has()` anchor, so a button
entering, leaving or flipping `disabled` inside a row invalidates that row's
entire subtree. In a 250-message thread that was 80 to 100 ms inside the send
keypress and again inside the final frame.

The control now stays mounted for as long as the row has a rerun to offer and
is concealed by visibility while a run is in flight: no focus, no pointer, no
place in the accessibility tree, and no layout where the toolbar starts at the
row's edge. Below `lg`, where the toolbar is centered, it leaves the flow as
before. It is deliberately not also `disabled`. The reply being written still
mounts no controls at all.

* 🎯 perf: Write :has() Rules So a Change in a Row Does Not Re-style the Row

Two selector shapes in the stylesheet left the browser unable to tell which
elements a `:has()` rule could reach, so it re-styled the whole subtree of
every `:has()` anchor above any change:

- Tailwind's `group-has-[…]`, which compiles to
  `.x:is(:where(.group):has(…) *)`. Message rows and their toolbars are
  `group`s, so a button mounting or unmounting in a row re-styled the row.
- `theme-disabled-within:[&_*]:…`, which compiles to `…:has(:disabled) *`.
  With it in the sheet, a `disabled` flip did the same.

Either one alone was enough: flipping `disabled` on one button per row of a
250-message thread took 100 to 110 ms, and mounting one took 88 ms. A run
starting or ending did one of those on every row.

The row-hover reveal, the header label crossfade and the toolbar's
hover-active hold now use `grouped-*` custom variants that emit
`:where(.group):has(…) .x`: the same elements at the same specificity, with a
subject the browser can read. A wrapper's parts take the disabled fill from
classes the component adds while it is disabled, through `theme-disabled-fill`,
instead of from `:has(:disabled) *`. The same two changes now cost 25 to 42 ms
and 27 ms.

`has-selectors.spec.ts` reads the stylesheet the app ships and fails on either
shape; `restyle.perf.spec.ts` measures the cost.

* 📱 fix: Keep a Concealed Rerun Hidden on Narrow Screens

The narrow-screen stylesheet forces every toolbar button visible from outside
any cascade layer, so it outranked the utilities that conceal a rerun while a
run is in flight: below 1024px the button stayed on screen and in the tab
order, inert. A concealed button now carries `data-concealed`, and that rule
leaves it alone.

Covered at a narrow and a wide viewport by what the reader gets: the settled
reply's rerun is hidden and cannot take focus during a run, and is back after.

* 🧮 perf: Skip the Pause Scan in Pending Steers When None Are Pending

The pending-steers row follows the conversation's messages, so it renders for
every streamed delta, and it resolved the run's pause state before checking
whether it had anything to show. That check walks every part of every message.
It now runs only with a steer to display.
2026-11-14 12:15:48 -05:00

404 lines
16 KiB
TypeScript

import fs from 'node:fs';
import { randomUUID } from 'node:crypto';
import { expect, test } from '@playwright/test';
import type { Page } from '@playwright/test';
import type { ExportedMessage } from './chats';
import type { WindowCost } from './trace';
import type { Part } from './replay';
import {
CONVOS,
ENDPOINT,
awaitRunEnd,
chatName,
draft,
openChat,
removeChats,
seedChat,
} from './chats';
import { buildFrames, startReplayServer } from './replay';
import { installRenderProbe } from './why';
import { formatFrameCost, traceFrames, traceWindows } from './trace';
/**
* What each kind of stream event costs the main thread.
*
* One assistant message from the exported chat is streamed back into the
* client at the tail of that same chat, a frame at a time with a pause between
* frames, so the work each frame causes lands in its own window: a tool call
* starting, its arguments, its result, a label filling, a reasoning delta.
* Windows with no frame give the floor the run's animations set on their own.
*/
/** Pause after each frame; long enough for the commit, its effects and its frames. */
const GAP_MS = Number(process.env.FADE_PERF_GAP_MS) || 150;
/** Frames per trace. A trace is parsed whole, so the run is recorded in pieces. */
const SEGMENT = 48;
/** An idle window after every this many frames. */
const IDLE_EVERY = 8;
/** How long the run is held mid-stream to read the cost of its animations alone. */
const QUIET_MS = 3000;
/** Position of the assistant message to replay; the most varied one by default. */
const REPLAY_INDEX = process.env.FADE_PERF_REPLAY_INDEX;
const MAX_PARTS = Number(process.env.FADE_PERF_REPLAY_PARTS) || Number.POSITIVE_INFINITY;
const CPU_RATE = Number(process.env.FADE_PERF_CPU) || 1;
const DUMP_DIR = process.env.FADE_PERF_DUMP;
/** `1` samples the page's JavaScript across the replay instead of tracing it; needs `FADE_PERF_DUMP`. */
const WITH_PROFILE = process.env.FADE_PERF_PROFILE === '1';
/** A stylesheet laid over the shipped one, to try a remedy. */
const EXTRA_CSS = process.env.FADE_PERF_CSS;
/** `1` prints what is animating after each idle window, and on which properties. */
const LIST_ANIMATIONS = process.env.FADE_PERF_ANIMATIONS === '1';
/** `1` counts component renders across the replay and says what caused each. */
const WITH_WHY = process.env.FADE_PERF_WHY === '1';
/** How many components the render count lists, most rendered first. */
const WHY_LIMIT = Number(process.env.FADE_PERF_WHY_LIMIT) || 80;
const MARK = 'fade:';
function variety(message: ExportedMessage): number {
const kinds = new Set<string>();
for (const part of message.content ?? []) {
const toolCall = part.tool_call as { name?: string } | undefined;
kinds.add(part.type === 'tool_call' ? `tool:${toolCall?.name}` : String(part.type));
}
return kinds.size;
}
function pickSource(messages: ExportedMessage[]): ExportedMessage {
if (REPLAY_INDEX != null) {
return messages[Number(REPLAY_INDEX)];
}
return messages
.filter((message) => !message.isCreatedByUser && Array.isArray(message.content))
.reduce((best, message) => (variety(message) > variety(best) ? message : best));
}
/** What is animating right now, and which properties: anything other than
* `opacity` or `transform` is driven from the main thread, frame by frame. */
function runningAnimations(page: Page): Promise<string> {
return page.evaluate(() =>
document
.getAnimations()
.filter((animation) => animation.playState === 'running')
.map((animation) => {
const effect = animation.effect as KeyframeEffect | null;
const target = effect?.target;
const properties = new Set<string>();
for (const keyframe of effect?.getKeyframes() ?? []) {
for (const key of Object.keys(keyframe)) {
if (!['offset', 'easing', 'composite', 'computedOffset'].includes(key)) {
properties.add(key);
}
}
}
const name = 'animationName' in animation ? animation.animationName : 'transition';
const where = target
? `${target.tagName.toLowerCase()}.${String(target.getAttribute('class')).slice(0, 60)}`
: '?';
return `${name}[${Array.from(properties).join(',')}]@${where}`;
})
.join(' | '),
);
}
const median = (values: number[]): number => {
const sorted = [...values].sort((a, b) => a - b);
return sorted.length === 0 ? 0 : sorted[Math.floor(sorted.length / 2)];
};
type Row = { kind: string; count: number } & Record<
'task' | 'worst' | 'script' | 'layout' | 'style' | 'paint' | 'gc',
number
>;
function summarize(windows: WindowCost[], kindOf: (name: string) => string): Row[] {
const groups = new Map<string, WindowCost[]>();
for (const window of windows) {
const kind = kindOf(window.name);
groups.set(kind, [...(groups.get(kind) ?? []), window]);
}
return Array.from(groups.entries())
.map(([kind, group]) => ({
kind,
count: group.length,
task: median(group.map((window) => window.taskMs)),
worst: Math.max(...group.map((window) => window.worstTaskMs)),
script: median(group.map((window) => window.scriptMs)),
layout: median(group.map((window) => window.layoutMs)),
style: median(group.map((window) => window.styleMs)),
paint: median(group.map((window) => window.paintMs)),
gc: median(group.map((window) => window.gcMs)),
}))
.sort((a, b) => b.task - a.task);
}
function print(title: string, rows: Row[]): void {
console.log(`\n${title}`);
console.log(
`${'event'.padEnd(46)}${'n'.padStart(4)}${'task'.padStart(8)}${'worst'.padStart(8)}` +
`${'script'.padStart(8)}${'layout'.padStart(8)}${'style'.padStart(7)}${'paint'.padStart(7)}`,
);
for (const row of rows) {
console.log(
`${row.kind.padEnd(46)}${String(row.count).padStart(4)}${row.task.toFixed(1).padStart(8)}` +
`${row.worst.toFixed(1).padStart(8)}${row.script.toFixed(1).padStart(8)}` +
`${row.layout.toFixed(1).padStart(8)}${row.style.toFixed(1).padStart(7)}` +
`${row.paint.toFixed(1).padStart(7)}`,
);
}
}
test.use({ trace: 'off', video: 'off' });
test.describe('stream event cost', () => {
test.skip(CONVOS.length === 0, 'FADE_PERF_CHATS names no exported conversation');
test.afterAll(removeChats);
for (const convo of CONVOS) {
test(`events replayed into ${chatName(convo)}`, async ({ page, browser }) => {
const messages = await seedChat(convo);
const source = pickSource(messages);
const content = (source.content ?? []).slice(0, MAX_PARTS) as Part[];
const runId = randomUUID();
const frames = buildFrames(content, runId);
const why = WITH_WHY ? await installRenderProbe(page) : null;
const size = await openChat(page, convo);
console.log(
`\n##### ${chatName(convo)}: ${size.rows} rows, ${size.nodes} elements; ` +
`replaying ${content.length} parts as ${frames.length} frames, ${GAP_MS}ms apart, cpu x${CPU_RATE}`,
);
if (EXTRA_CSS) {
await page.addStyleTag({ content: EXTRA_CSS });
}
const server = await startReplayServer();
let sent: Record<string, unknown> = {};
await page.route(/\/api\/agents\/chat\/stream\//, (route) =>
route.continue({ url: server.url }),
);
await page.route(/\/api\/agents\/chat\/[^/]+$/, async (route) => {
const request = route.request();
if (request.method() !== 'POST' || request.url().endsWith('/abort')) {
await route.fallback();
return;
}
sent = request.postDataJSON() as Record<string, unknown>;
await route.fulfill({
contentType: 'application/json',
headers: { 'X-LibreChat-Generation-Protocol': '2' },
body: JSON.stringify({
streamId: convo.id,
conversationId: convo.id,
generationCreatedAt: Date.now(),
status: 'started',
generationProtocolVersion: 2,
}),
});
});
if (CPU_RATE > 1) {
const session = await page.context().newCDPSession(page);
await session.send('Emulation.setCPUThrottlingRate', { rate: CPU_RATE });
}
await draft(page, 'Replay the recorded run.');
/** One profile from the keypress to the end of the run, so the send, every
* event and the final frame can each be read out of it by time. */
const profiler = WITH_PROFILE ? await page.context().newCDPSession(page) : null;
if (profiler) {
await profiler.send('Profiler.enable');
await profiler.send('Profiler.setSamplingInterval', { interval: 200 });
await profiler.send('Profiler.start');
}
const windows: WindowCost[] = [];
const mark = (name: string) =>
page.evaluate((value) => {
performance.mark(value);
}, `${MARK}${name}`);
/** A profile or a render count wants the events alone, without the trace's marks. */
const untraced = WITH_PROFILE || WITH_WHY;
/** The send: the keypress that publishes the turn, then the frame that confirms it. */
let requestMessage: Record<string, unknown> = {};
const send = async () => {
await mark('send|0');
await page.keyboard.press('Enter');
await Promise.race([
server.connected,
page.waitForTimeout(30_000).then(() => {
throw new Error('The client never opened the replayed stream');
}),
]);
await page.waitForTimeout(500);
requestMessage = {
messageId: sent.messageId,
parentMessageId: sent.parentMessageId,
conversationId: convo.id,
text: sent.text,
sender: 'User',
isCreatedByUser: true,
};
await mark('created|0');
server.send({ created: true, message: requestMessage, streamId: convo.id });
await page.waitForTimeout(1500);
await mark('end|0');
};
if (untraced) {
await why?.start();
await send();
const rendered = await why?.finish(25);
if (rendered) {
console.log(`\ncomponent renders for the send (${rendered.commits} commits):`);
for (const line of rendered.lines) {
console.log(line);
}
}
} else {
const traced = await traceWindows(browser, page, MARK, send);
windows.push(...traced.filter((window) => !window.name.startsWith('end|')));
}
if (untraced) {
await why?.start();
for (const frame of frames) {
server.send(frame.payload);
await page.waitForTimeout(GAP_MS);
}
const rendered = await why?.finish(WHY_LIMIT);
if (rendered) {
console.log(
`\ncomponent renders across ${frames.length} frames (${rendered.commits} commits):`,
);
for (const line of rendered.lines) {
console.log(line);
}
}
}
for (let offset = 0; !untraced && offset < frames.length; offset += SEGMENT) {
const segment = frames.slice(offset, offset + SEGMENT);
if (offset > 0 && offset <= SEGMENT) {
/** The run held still: no frame arrives, so whatever the main thread
* does here it does for the run's animations alone. */
await page.waitForTimeout(1500);
const quiet = await traceFrames(browser, page, () => page.waitForTimeout(QUIET_MS));
console.log(formatFrameCost(`run held still for ${QUIET_MS}ms`, quiet));
if (DUMP_DIR) {
/** Every sweep frozen at one instant, so two builds can be compared by eye. */
await page.evaluate(() => {
for (const animation of document.getAnimations()) {
animation.pause();
animation.currentTime = 1250;
}
});
await page.waitForTimeout(300);
await page
.locator(`[id="${runId}"]`)
.screenshot({ path: `${DUMP_DIR}/${chatName(convo)}.held.png` });
await page.evaluate(() => {
for (const animation of document.getAnimations()) {
animation.play();
}
});
}
}
const traced = await traceWindows(browser, page, MARK, async () => {
for (const [position, frame] of segment.entries()) {
await mark(`${frame.kind}|${frame.index}`);
server.send(frame.payload);
await page.waitForTimeout(GAP_MS);
if ((position + 1) % IDLE_EVERY === 0) {
await mark(`idle|${frame.index}`);
await page.waitForTimeout(GAP_MS);
if (LIST_ANIMATIONS) {
console.log(`animating after ${frame.kind}: ${await runningAnimations(page)}`);
}
}
}
await mark('end|0');
});
windows.push(...traced.filter((window) => !window.name.startsWith('end|')));
}
const finish = async () => {
await mark('final|0');
server.send({
final: true,
conversation: {
conversationId: convo.id,
title: convo.title,
endpoint: ENDPOINT.label,
endpointType: 'custom',
model: ENDPOINT.model,
},
title: convo.title,
requestMessage,
responseMessage: {
messageId: runId,
parentMessageId: sent.messageId,
conversationId: convo.id,
isCreatedByUser: false,
sender: 'Assistant',
content,
text: '',
endpoint: ENDPOINT.label,
model: ENDPOINT.model,
},
generationProtocolVersion: 2,
});
server.end();
await awaitRunEnd(page, true);
await page.waitForTimeout(500);
};
if (profiler) {
await finish();
const { profile } = await profiler.send('Profiler.stop');
if (DUMP_DIR) {
fs.writeFileSync(
`${DUMP_DIR}/${chatName(convo)}.replay.cpuprofile`,
JSON.stringify(profile),
);
}
} else if (why) {
await why.start();
await finish();
const rendered = await why.finish(25);
console.log(`\ncomponent renders for the final frame (${rendered.commits} commits):`);
for (const line of rendered.lines) {
console.log(line);
}
} else {
windows.push(...(await traceWindows(browser, page, MARK, finish)));
}
await server.close();
const kindOf = (name: string) => name.split('|')[0];
print('per event kind (median ms per window)', summarize(windows, kindOf));
print(
'per event family',
summarize(windows, (name) => kindOf(name).split(':')[0]),
);
/** The same events early and late in the run: a cost that grows with the
* parts already streamed shows up as the gap between these. */
const third = Math.ceil(content.length / 3);
print(
'per event family, by position in the run',
summarize(windows, (name) => {
const [kind, index] = name.split('|');
const position = Number(index);
const band = ['first', 'middle', 'last'][Math.min(2, Math.floor(position / third))];
return `${kind.split(':')[0]} (${band} third)`;
}).sort((a, b) => a.kind.localeCompare(b.kind)),
);
if (DUMP_DIR) {
fs.writeFileSync(
`${DUMP_DIR}/${chatName(convo)}.replay.windows.json`,
JSON.stringify(windows),
);
}
const rendered = await page.evaluate(
(id) => document.getElementById(id)?.querySelectorAll('*').length ?? 0,
runId,
);
console.log(`\nreplayed row holds ${rendered} elements after the run`);
expect(rendered).toBeGreaterThan(0);
});
}
});