mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 14:08:21 +00:00
* ⚡ perf: Swap the Transcript With the URL on Conversation Switch Switching conversations left the PREVIOUS transcript painted under the new URL. Two things on the critical path caused it, both fixed here. `RouterProvider` commits location updates inside `React.startTransition` by default in react-router v7, and a transition keeps the outgoing tree on screen until the incoming one has fully rendered — so every millisecond the next thread took to render was time spent looking at the previous one, and React yields during that render, stretching it well past its CPU cost. Nothing here reads route data through router loaders, so the transition bought no pending UI; conversation state also still lives in Recoil, whose transition-safe reads are gated behind `_TRANSITION_SUPPORT_UNSTABLE` hooks this app does not use. `useTransitions={false}` puts the route change back in the click's own task. `navigateToConvo` also awaited `GET /api/convos/:id` before calling `navigate()`, so the route did not change until a full server round trip completed. The clicked row already carries its conversation, so the route and conversation state now change together and the refetch reconciles afterwards. The row is a list projection, so any previously fetched full record underlays it — prompt prefix, sampling params and files survive the switch, and a send during the reconcile window still carries the real settings. Measured on the built client with a 250ms conversation-fetch latency, switching between two 30-turn conversations: before cold click→url 527ms click→paint 931ms 14 stale frames (297ms) warm click→url 474ms click→paint 838ms 12 stale frames (277ms) after cold click→url ~190ms click→paint ~450ms 0 stale frames warm click→url ~280ms click→paint ~280ms 0 stale frames The warm switch now paints the new transcript in the same commit as the URL. The warm-cache message loading this depends on is untouched. Adds `e2e/benchmarks-navigation`, a react-scan benchmark that guards the result: an in-page sampler records the route and the mounted conversation once per animation frame, so a frame pairing the next URL with the previous transcript is caught directly. The react-scan harness the reasoning benchmark had inlined moves to `e2e/perf/scan.ts` and is now shared. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qZJDNkyH5rgCz6KcLjseq * 🎯 fix: Resolve Sidebar Rows by Their Accessible Button in the Nav Benchmark The a11y pass on the sidebar moved the conversation row's `role="button"` and `aria-label` off the `convo-item` container and onto a real `<button>` that `ConvoLink` renders inside it. The benchmark's click helper required a single node carrying both the testid and the label, so after merging dev it found nothing and threw. Match on whichever node inside a row carries the label and let the click bubble to the container's handler, which still owns the navigation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qZJDNkyH5rgCz6KcLjseq * 🛡️ fix: Close Three Navigation Races Found in Review Codex review of the optimistic-navigation path found three real defects. Superseded reconciliations were written unconditionally. Selecting B then C before both records settled let B's response land last and restore B into conversation state while the route and transcript showed C — and sends read from that state, so a user could submit into a conversation they were no longer looking at. Navigations now claim a shared token before any await and late responses are discarded. The token is module state rather than a ref because every sidebar row mounts its own hook instance, so a ref cannot see that a click on a different row superseded this one. The first visit to a conversation installed the sidebar row as active state. That row is a projection without prompt prefix, sampling params, tools or files, so the composer became usable with settings that silently fell back to defaults. Only a conversation whose full record is already cached now takes the instant path; the first visit keeps the previous behavior and moves the route once the record is in hand. Every later switch to it is instant, which is the case this PR set out to fix. A failed record fetch removed the target's message cache even though, after optimistic navigation, that query is already mounted — a transient error could cancel an in-flight history fetch, or discard one that had succeeded, with no route change left to remount it. That removal is now limited to a conversation confirmed gone, and the first-visit path still clears before the route moves, where a fresh mount follows. The benchmark's round-trip assertion was also unfalsifiable: nothing delayed the record request, so an implementation that awaits it still answered inside the threshold. It now holds that request open and asserts the warm switch completes while it is unresolved, which no wall-clock bound can fake. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qZJDNkyH5rgCz6KcLjseq * 🧭 fix: Tie Pending Navigation Work to the Route, Not a Token Codex found that the navigation token only tracked calls made through this hook. Every other way out of a conversation — `useNewConvo`, a link, a redirect, the back button — moves the route without touching it, so a record still in flight for the conversation being left passed the guard. On the cached path that overwrote the new route's conversation state; on the first-visit path it was worse, calling `navigate()` and pulling the user back into a chat they had already left. The token was the wrong question. What makes pending work still wanted is not "was this the last conversation clicked" but "is the user still where they were when it started" — and only the browser's own location sees every way that can change. Each async step now captures the route before its request and re-reads it before writing, which subsumes the superseded-click case the token was added for and removes the module state entirely. Reading `window.location` directly rather than `useLocation` keeps this free of subscriptions: every sidebar row mounts this hook, so subscribing would re-render all of them on every navigation — the cost this hook exists to avoid. Comparing pathname against pathname also makes the basename cancel. The tests move from `MemoryRouter` to a real history, since the mechanism is now the browser location itself, and cover both bypass paths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qZJDNkyH5rgCz6KcLjseq * 🔢 fix: Keep the Last Click Authoritative Across First-Visit Navigations Codex found that the route guard cannot separate two first-visit clicks from each other. That path deliberately leaves the route where it is until the record arrives, so clicking two uncached conversations in quick succession has both requests capture the same pathname — whichever the network answered first then navigated, and the later click was discarded. Response order decided where the user landed instead of click order. Restores a generation counter alongside the route check. Claiming the last PR's removal of the token as a subsumption was wrong: the two guards answer different questions and neither covers the other. The generation says "a newer intent replaced this one"; the route says "the user left by some means this hook never saw". Both are needed, and both are cheap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qZJDNkyH5rgCz6KcLjseq * 🧹 refactor: Stop Writing Server Snapshots Into User-Owned Conversation State Four review findings on this branch were all the same defect: navigation started a background fetch and wrote its result into the conversation atom. That atom is user-editable — model, endpoint, prompt prefix, sampling params — and the target chat is interactive from the moment the route changes, so a late write races the user and every other writer. Each round added another predicate to the write ("is this still the last click?", "is the user still on this route?"), and each predicate left one more writer uncovered; the last one is a setting picked on the same route by the same navigation, which no ordering or route guard can see. Remove the write instead of guarding it. The warm path refreshes the React Query cache and stops there, so the optimistic merge that lands with the route is the navigation's last word. The refreshed record is consumed by the next switch to that conversation, which is where a cached record is read anyway. This also dissolves the queued-focus finding: `applyConversation` (and its `requestChatFocus`) is now reachable only from paths that navigate, so a focus intent can no longer outlive the navigation that requested it. Scope the synchronous route commit to conversation switches. `useTransitions` on the provider disabled transitions for every route, including the lazily loaded prompts, skills, insights and project screens, where yielding to input during a large first render is worth more than an atomic swap. The opt-out now travels per navigation as `chatNavigation` (`flushSync`), applied in `useNavigateToConvo` and `useNewConvo`. Tests: the four behavioural guards fail against an implementation that restores the background write, including a new case where the user picks a model while the refresh is in flight. * 🎯 fix: Decide Route Commit Once, and Keep Refreshed Settings Refreshed Reverts the per-navigation transition opt-out from the previous commit. Review asked to scope `useTransitions={false}` to conversation switches, and I scoped it by passing an option at the call sites I knew about — then immediately missed one: `finalHandler` promotes `/c/new` to the server-assigned ID and navigates without it, so the atom identifies the real conversation while the route and message query still say `new`. That is not a missed call site, it is the wrong shape. Fourteen call sites across components, chat hooks and SSE handlers navigate into `/c/*`; an opt-out carried by each one is a list that rots as call sites are added, and five of the fourteen were covered. The property is route-shaped, so the decision goes back to the one place that sees every navigation. Answering the original critique on its merits: nothing in the app reads route data through router loaders or renders pending UI from `useNavigation`, so the transition produces no interstitial on any route — it only defers the commit, which on the chat route is the bug this PR exists to fix. Two conversation fixes alongside it: Sidebar rows no longer reinstate settings the background refresh replaced. The row projection carries `endpoint`, `model` and `spec`, and the warm path spreads the row over the cached record — so a row from before an edit made on another device would undo that edit on every switch until the list refetched. The refresh now merges the record into the list cache, which is what made "picked up on the next switch" true rather than merely intended. Starting a new chat now supersedes a pending first visit. "New chat" from `/c/new` lands on `/c/new`, so the pathname is unchanged and the record for a conversation the user just abandoned would land and pull them into it. The navigation counter is exported as `supersedeNavigation` and called from `useNewConvo`. Deliberately not called from the stream recoveries in `useEventHandlers`/`useChatFunctions`: those are the app reacting, not the user changing their mind, and they should not cancel a conversation the user opened. Intent is a closed set; navigation is not. Both new tests fail against the implementation they guard. * 🧷 fix: Keep the Record Refresh Off List State and Off Background Composers Three fixes to the previous two commits, all the same underlying mistake in different places: something that started earlier landing on top of something the user did later. The list-cache write added last commit merged the whole fetched record into every sidebar and pinned row. That response is a snapshot from before the target was interactive, and the list is where renaming, pinning and sharing land — so a rename completing while the request was in flight was silently undone. This is the same stale-snapshot-over-live-state mistake the refresh had just stopped making against the conversation atom, reintroduced one layer down. It now writes `endpoint`, `model` and `spec` only, which is what the staleness it exists to fix is about, and which no list mutation touches. The route comparison ignored the query string. `/c/new?projectId=A` is a different conversation scope than `/c/new`, and the landing chip re-scopes a draft by writing the atom and rewriting search params in place — never through a conversation hook, so neither the pathname nor the recorded intent moved. A pending first-visit record would then land on the draft the user had just re-scoped. The comparison now includes `search`. Superseding moved from `switchToConversation` into `newConversation`, guarded by `keepComposerState`. That flag marks a call that re-renders a composer an earlier call already opened — agent metadata arriving late, for instance. The user asked for nothing there, so it must not cancel a conversation they clicked while it was in flight. `switchToConversation` has no callers outside this hook, so the move loses no coverage. The first two are covered by tests that fail against the implementation they guard. The third is verified by inspection: exercising it needs the whole `useNewConvo` provider tree, which is disproportionate for a one-line guard. --------- Co-authored-by: Claude <noreply@anthropic.com>
254 lines
11 KiB
TypeScript
254 lines
11 KiB
TypeScript
import { expect, test } from '@playwright/test';
|
||
import {
|
||
MOCK_ENDPOINTS,
|
||
NEW_CHAT_PATH,
|
||
messagesView,
|
||
selectMockEndpoint,
|
||
sendMessage,
|
||
} from '../specs/mock/helpers';
|
||
import {
|
||
buildTextSection,
|
||
buildThinkSection,
|
||
countModelChunks,
|
||
END_MARKER,
|
||
SENTENCE,
|
||
} from './payload';
|
||
import {
|
||
attachSnapshot,
|
||
installReactScan,
|
||
resetPerf,
|
||
snapshotPerf,
|
||
topComponents,
|
||
totals,
|
||
} from '../perf/scan';
|
||
|
||
test.describe('reasoning stream perf (react-scan)', () => {
|
||
test('one long unsplit reasoning + markdown reply stays render-bounded', async ({
|
||
page,
|
||
}, testInfo) => {
|
||
test.setTimeout(6 * 60 * 1000);
|
||
|
||
const thinkSection = buildThinkSection();
|
||
const textSection = buildTextSection();
|
||
const thinkChunks = countModelChunks(thinkSection);
|
||
const textChunks = countModelChunks(textSection);
|
||
const sectionCount = (textSection.match(/## Section /g) ?? []).length;
|
||
|
||
/** The payload always opens with reasoning, so the first ThinkingContent
|
||
* render is the first assistant-content paint — anchor the measured
|
||
* interval there. */
|
||
await installReactScan(page, 'ThinkingContent');
|
||
/** Stream with the reasoning box EXPANDED — the heavier layout path a
|
||
* user gets with "Show Thinking" enabled — so the measured interval
|
||
* covers live paragraph layout inside the box, not just the collapsed
|
||
* header. */
|
||
await page.addInitScript(() => {
|
||
localStorage.setItem('showThinking', 'true');
|
||
});
|
||
|
||
/** First load through the vite dev server transforms the module graph. */
|
||
await page.goto(NEW_CHAT_PATH, { timeout: 180_000 });
|
||
await selectMockEndpoint(page, MOCK_ENDPOINTS[0]);
|
||
|
||
/** Reset BEFORE the send: with a 1ms chunk delay the earliest deltas can
|
||
* render between the response headers resolving and any later
|
||
* evaluation, and a post-send reset would erase them. The clock anchors
|
||
* to the first ThinkingContent render — the payload always opens with
|
||
* reasoning, so that is the first assistant-content paint — keeping
|
||
* composer renders and idle request setup out of the denominators (the
|
||
* few pre-stream composer renders stay in the tally, which only makes
|
||
* the bounds stricter). */
|
||
await resetPerf(page);
|
||
await sendMessage(page, 'Stream the long reasoning benchmark reply.');
|
||
|
||
await expect(messagesView(page).getByText(END_MARKER)).toBeVisible({
|
||
timeout: 4 * 60 * 1000,
|
||
});
|
||
/** The marker only proves the final text delta painted — generation
|
||
* finalization (usage chunk, terminal events, save-time re-render) is
|
||
* part of the measured stream, so wait for it to finish first. */
|
||
await expect(page.getByRole('button', { name: 'Stop generating' })).toBeHidden({
|
||
timeout: 30_000,
|
||
});
|
||
const streaming = await snapshotPerf(page);
|
||
const streamMs = Math.ceil(streaming.elapsedMs);
|
||
|
||
/**
|
||
* The whole reasoning section must land in ONE think part — a single
|
||
* Thoughts toggle. More than one means something re-split the reasoning.
|
||
*/
|
||
const thoughtToggles = messagesView(page).getByRole('button', {
|
||
name: /^(Thoughts|Thinking)$/,
|
||
});
|
||
await expect(thoughtToggles).toHaveCount(1);
|
||
|
||
/**
|
||
* One nonempty toggle is not enough — the ENTIRE reasoning section must
|
||
* survive the pipeline, internal paragraph breaks included (the box
|
||
* renders whitespace-pre-wrap, so they are user-visible content). The
|
||
* only transforms the UI applies are inline-tag stripping and edge
|
||
* trimming, so the comparison is exact after trimming the source edges.
|
||
* The box is already expanded via the seeded showThinking preference.
|
||
*/
|
||
const thinkGroup = messagesView(page).getByRole('group', {
|
||
name: /^(Thoughts|Thinking)$/,
|
||
});
|
||
const renderedThink = (await thinkGroup.locator('p').first().textContent()) ?? '';
|
||
expect(renderedThink).toBe(thinkSection.trim());
|
||
|
||
/**
|
||
* The markdown body must also arrive whole — END_MARKER only proves the
|
||
* suffix rendered. Structure alone is not enough either: verify the prose
|
||
* itself — every section's heading, doubled-sentence paragraph, and both
|
||
* list items, plus the exact table count with cell values and the code
|
||
* block's lines — so the measured render work covers the full payload.
|
||
*/
|
||
expect(sectionCount).toBeGreaterThan(0);
|
||
const doubledSentence = `${SENTENCE}${SENTENCE}`.trim();
|
||
await expect(messagesView(page).getByText(doubledSentence, { exact: true })).toHaveCount(
|
||
sectionCount,
|
||
);
|
||
for (let section = 1; section <= sectionCount; section += 1) {
|
||
await expect(
|
||
messagesView(page).getByRole('heading', { name: `Section ${section}`, exact: true }),
|
||
).toBeVisible();
|
||
await expect(
|
||
messagesView(page).getByText(`Point one for section ${section}`, { exact: true }),
|
||
).toBeVisible();
|
||
await expect(
|
||
messagesView(page).getByText(`Point two for section ${section}`, { exact: true }),
|
||
).toBeVisible();
|
||
}
|
||
await expect(messagesView(page).getByRole('listitem')).toHaveCount(sectionCount * 2);
|
||
const tableCount = Math.floor(sectionCount / 4);
|
||
await expect(messagesView(page).getByRole('table')).toHaveCount(tableCount);
|
||
for (const cellValue of ['120000', '135500', '151200']) {
|
||
await expect(
|
||
messagesView(page).getByRole('cell', { name: cellValue, exact: true }),
|
||
).toHaveCount(tableCount);
|
||
}
|
||
const codeBlockCount = Math.floor(sectionCount / 3);
|
||
expect(codeBlockCount).toBeGreaterThan(0);
|
||
for (const codeLine of ['export function estimate', 'return Math.round(total * rate);']) {
|
||
await expect(messagesView(page).locator('code', { hasText: codeLine })).toHaveCount(
|
||
codeBlockCount,
|
||
);
|
||
}
|
||
|
||
await resetPerf(page);
|
||
const input = page.getByRole('textbox', { name: 'Message input' });
|
||
await input.click();
|
||
await input.pressSequentially('typing latency probe after long transcript', { delay: 25 });
|
||
const typing = await snapshotPerf(page);
|
||
|
||
const streamTotals = totals(streaming);
|
||
const typingTotals = totals(typing);
|
||
const longTaskTotal = streaming.longTasks.reduce((sum, duration) => sum + duration, 0);
|
||
const worstLongTask = streaming.longTasks.reduce((max, duration) => Math.max(max, duration), 0);
|
||
|
||
console.log(`\n=== Streaming phase (${streamMs}ms wall) ===`);
|
||
console.log(`model chunks: think=${thinkChunks} text=${textChunks}`);
|
||
console.log(
|
||
`total renders=${streamTotals.renders} render-time=${streamTotals.time.toFixed(0)}ms ` +
|
||
`longtask-total=${longTaskTotal.toFixed(0)}ms worst-longtask=${worstLongTask.toFixed(0)}ms`,
|
||
);
|
||
for (const line of topComponents(streaming, 15)) {
|
||
console.log(` ${line}`);
|
||
}
|
||
console.log('key components:');
|
||
for (const component of ['ThinkingContent', 'MarkdownBlock', 'MarkdownBlocks', 'TextPart']) {
|
||
const slot = streaming.renders[component];
|
||
console.log(
|
||
` ${component.padEnd(20)} renders=${slot?.count ?? 0} time=${(slot?.time ?? 0).toFixed(1)}ms`,
|
||
);
|
||
}
|
||
console.log('=== Typing phase (40 keys) ===');
|
||
console.log(
|
||
`total renders=${typingTotals.renders} render-time=${typingTotals.time.toFixed(0)}ms`,
|
||
);
|
||
for (const line of topComponents(typing, 10)) {
|
||
console.log(` ${line}`);
|
||
}
|
||
|
||
await attachSnapshot(testInfo, 'streaming-renders.json', streaming, {
|
||
streamMs,
|
||
thinkChunks,
|
||
textChunks,
|
||
});
|
||
await attachSnapshot(testInfo, 'typing-renders.json', typing, {});
|
||
|
||
/**
|
||
* rAF coalescing must keep per-token work bounded: cache flushes happen at
|
||
* most once per animation frame, so render counts scale with elapsed
|
||
* frames, never with chunk count. The bound is derived from wall time
|
||
* (60fps + 50% headroom) — without coalescing, renders track chunks
|
||
* (~5k in ~13s) and blow far past it (measured baseline: 122). The floor
|
||
* guards against the instrumentation (or the component name) silently
|
||
* disappearing, which would zero the count and void the upper bound.
|
||
*/
|
||
const framesUpperBound = Math.ceil((streamMs / 1000) * 90);
|
||
const thinkingContentRenders = streaming.renders['ThinkingContent']?.count ?? 0;
|
||
expect(thinkingContentRenders).toBeGreaterThan(10);
|
||
expect(thinkingContentRenders).toBeLessThan(framesUpperBound);
|
||
/** Rate-independent companion bound: even on a slow stream (where the
|
||
* frame bound balloons), one-render-per-chunk behavior must still fail. */
|
||
expect(thinkingContentRenders).toBeLessThan(thinkChunks / 4);
|
||
|
||
/**
|
||
* Markdown must not re-render every block on every token — total
|
||
* MarkdownBlock renders stay in the order of frames + blocks (measured
|
||
* baseline: 153), far below blocks × tokens (~100k). Same floor rationale
|
||
* as above: zero means the guard lost its subject, not that it passed.
|
||
*/
|
||
const markdownBlockRenders = streaming.renders['MarkdownBlock']?.count ?? 0;
|
||
expect(markdownBlockRenders).toBeGreaterThan(10);
|
||
expect(markdownBlockRenders).toBeLessThan(framesUpperBound);
|
||
expect(markdownBlockRenders).toBeLessThan(textChunks / 4);
|
||
|
||
/**
|
||
* The main thread must stay responsive while the huge block streams:
|
||
* no single stall past 250ms, no more than 10% of the stream's wall time
|
||
* in long tasks (measured baseline: one 51-96ms task), and — because
|
||
* sustained sub-50ms work never surfaces as a long task — cumulative
|
||
* render time capped as well (measured baseline: ~4-7% of wall time).
|
||
*/
|
||
expect(worstLongTask).toBeLessThan(250);
|
||
expect(longTaskTotal).toBeLessThan(streamMs * 0.1);
|
||
expect(streamTotals.time).toBeLessThan(streamMs * 0.25);
|
||
|
||
/**
|
||
* Typing after the long transcript must not re-render the transcript:
|
||
* message-content components stay quiet while the composer updates.
|
||
*/
|
||
const transcriptComponents = [
|
||
'MarkdownBlock',
|
||
'MarkdownBlocks',
|
||
'Markdown',
|
||
'ThinkingContent',
|
||
'TextPart',
|
||
'Part',
|
||
'MessageContent',
|
||
];
|
||
for (const component of transcriptComponents) {
|
||
const renders = typing.renders[component]?.count ?? 0;
|
||
expect(renders, `${component} re-rendered while typing`).toBeLessThanOrEqual(2);
|
||
}
|
||
|
||
/**
|
||
* Quiet transcript components alone don't prove keystrokes feel fast —
|
||
* slow input handlers or layout can lag without re-rendering any named
|
||
* component. Bound the typing phase's own long tasks and cumulative
|
||
* render time (measured baseline: no long tasks, ~3% render time).
|
||
*/
|
||
const typingWorstLongTask = typing.longTasks.reduce(
|
||
(max, duration) => Math.max(max, duration),
|
||
0,
|
||
);
|
||
const typingLongTaskTotal = typing.longTasks.reduce((sum, duration) => sum + duration, 0);
|
||
expect(typingWorstLongTask).toBeLessThan(150);
|
||
/** Absolute cumulative budget — repeated sub-threshold stalls both evade
|
||
* a worst-case check and inflate elapsedMs, so no ratio is used here. */
|
||
expect(typingLongTaskTotal).toBeLessThan(300);
|
||
expect(typingTotals.time).toBeLessThan(typing.elapsedMs * 0.25);
|
||
});
|
||
});
|