mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-04 14:57:42 +00:00
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* feat: surface interrupt-steer escalation on waiting messages The interrupt & steer feature shipped reachable only through the composer chord, the send-button hovercard, and the composer button; a message already waiting (queued for after the run, or steered and parked at the next tool boundary) had no path to it. Both waiting surfaces now carry one: - Queued rows get an icon-only ZapOff escalation button beside the existing Steer primary. It routes through sendQueuedNow, which now takes a preempt option on its live-run path. The tooltip teaches the composer chord, derived through resolveComposerKeyDown so a rebound or yielded chord is never advertised. - In-flight steer bubbles get an "Interrupt now" overflow entry with the same race rules as Edit: reclaim first, and only a `reclaimed` outcome resubmits (via retrySteer with preempt, swapping the chip for an interrupting one). `applied` and run-ended-mid-reclaim outcomes stop at the existing informational toasts, so the words can never land twice. Not offered on a steer already preempting. - Every during-run overflow menu gains an "Always interrupt instead" toggle for steerInterruptsByDefault, next to the existing steer/queue default toggle. MenuEntry supports disabled for the new entries. Only one interrupt can be unresolved at a time: while one preempt is pending (or the run is paused on approval, where the server 409s), every escalation control disables instead of racing the same seal. Ten new tests across both surfaces; 381 green in the affected suites. * fix: lock escalation across its reclaim window, keep the paused control visible, label as steer Codex round 1, all three findings. P2, escalation race. The single-interrupt invariant had a window between clicking "Interrupt now" and the reclaim resolving, where no preempt chip existed for the chip-derived gate to see: two bubbles escalated back-to-back could both resubmit. A shared escalating flag (Jotai, per-conversation) now covers the window and disables every escalation control on both surfaces, and a fresh recheck before resubmitting catches an interrupt armed elsewhere meanwhile (composer chord, queued row); those words re-home to the queue with an informational toast instead of breaking the invariant. P2, unreachable paused state. canSteer is defined as hasRealConvoId && !pausedOnApproval, so gating the button on canSteer removed it exactly when it was meant to render disabled; the test only passed on an impossible stub combination. The render gate is now duringRunActive && (canSteer || pausedOnApproval), and the test uses the real invariant. P2, label semantics. "Interrupt & send now" borrowed the name of the hard-abort action; this one preserves the partial answer and steers. Renamed to "Interrupt & steer now" (com_ui_interrupt_steer_now). Both behavior fixes counterfactually verified; 384 tests green across the affected suites. * fix: disable bubble escalation while the run cannot accept a steer Codex round 2, one P2. Answer mode (ask_user_question) sets duringRunActive false while pausedOnApproval stays false, since that flag only detects approval-bearing tool calls. The bubble's escalation entry stayed enabled there, so clicking it cancelled a healthy waiting steer and the preempt resubmission bounced off RUN_PAUSED, degrading the words to the queue. The entry now also disables on !duringRunActive, matching the queued-row control's gate. Counterfactually verified: reverting the gate fails the new answer-mode test. * fix: recheck live run state after the reclaim, not just at the click Codex round 3, one P2, and it is the round-1 recheck principle applied one level deeper: the entry-time disable cannot see a run that pauses (tool approval, answer mode) while the reclaim round-trip is in flight, and the .then closure held the render's stale steering controls, so the resubmit would fire into a RUN_PAUSED rejection after the reclaim had already surrendered the steer's boundary slot. The escalation continuation now reads the LIVE controls through a latest-ref: if the run can no longer accept a steer, the words re-home to the queue with an informational toast instead of resubmitting, and the resubmit itself also goes through the live controls. Counterfactually verified: reading the stale closure instead of the ref fails the new mid-reclaim pause test. * refactor: make escalation one atomic server-side arm, in place Codex round 4: four P2s, every one an interleaving of the same window — escalation as reclaim-then-repost is a compound, non-atomic operation whose continuation must revalidate the world (FIFO position lost, ref assigned too late, no run fence, competing bubble actions). Rounds 1-3 patched that window with a lock and rechecks; round 4 shows the window itself is the defect, so this removes it instead of guarding it again. Escalation is now POST /chat/steer/arm: the server flips preempt on the EXISTING queued item in one atomic store op (new IJobStore.armSteer; a decode-patch-encode LSET Lua on Redis, an in-place mutation in memory), fenced to the validated generation and refused once the queue closes. The handler mirrors the steer POST's preempt contract exactly: durable flag gated on the owner's recorded capability, volatile requestPreempt fire-and-forget because the durable flag is the truth resume/handover re-arm from. By construction this resolves all four findings: FIFO survives (the item never moves; the whole queue still drains in instruction order at the seal), there is no continuation to hold stale controls, the store op is fenced to the original run, and a competing Edit/Queue/Cancel either beats the arm (armed:false, chip untouched) or operates on the armed item, whose cancel already disarms. The client escalation entry becomes one mutation: armed:true relabels the chip in place (same steerId, same position), PREEMPT_UNSUPPORTED and lost races toast honestly, and the round 1-3 machinery — the escalating lock atom, the latest-ref, the post-reclaim rechecks and their two toast strings — is deleted rather than extended. Verified: 7 new handler tests on the real in-memory manager (including FIFO preservation and the stale-generation fence), 2 Redis integration tests against real Redis (in-place arm keeps order and every field; missing/stale/closed all refuse), client suites 396 green. * fix: decide capability inside the atomic arm, neutralize the lost-race toast Codex round 5, both findings, both edges of the new arm design rather than its mechanism. P2, capability TOCTOU. A HITL resume on a rolling deploy rewrites preemptCapable for the SAME generation, so the handler's read could go stale between validation and the flag flip, arming a steer the live owner cannot seal. armSteer now returns armed | missing | incapable, with the owner's live capability part of the same atomic predicate as the generation fence (HGET preemptCapable inside the Lua; the flat job field, not a metadata blob — the in-memory store reads the same field). The handler's pre-check is deleted rather than kept alongside; the store predicate is the single source. New handler test rewrites the capability after queueing and expects PREEMPT_UNSUPPORTED with the item left unflagged; the Redis guards test now asserts the incapable refusal against real Redis. P2, ambiguous toast. armed:false covers injected, cancelled, re-homed, and run-over alike, so telling the user the message "already reached the agent" claimed one specific outcome. The lost-race branch now uses a neutral message (com_ui_steer_arm_lost_race) and defers to the events for what actually happened. * fix: flip the escalation lock synchronously before the arm request Codex round 6, one P2. Round 4 deleted the escalating flag along with the reclaim continuation it guarded, but that left the one-interrupt gate blind during the arm request's own round trip: the chip-derived check cannot see an arm until its response relabels the chip, so on a slow connection two bubbles could both arm before either response landed. Double-arm is harmless server-side now (the run seals once and drains the whole queue in order), but every escalation control advertises "one interrupt at a time" by disabling, and the controls must tell the truth. The per-conversation escalating flag returns as a pure UX gate: set synchronously at click, before the mutation, cleared on settlement, and folded into interruptPending on both surfaces. Unlike its round 1-3 ancestor there is no continuation behind it to guard and no recheck to pair with it. Counterfactually verified: without the synchronous set, the two-bubble race test arms twice. 207 tests green across the Chat Input suites. * test(e2e): cover escalation of waiting messages through the real seal Three mock-harness tests on E2E_SLOW_REPLY, a 160-chunk stream with no tool boundary, so an in-thread steer part can ONLY come from a genuine mid-stream seal — which makes each test a behavioral proof rather than a UI check: - Queued row escalation: the ZapOff button turns a waiting queued message into a preempt-armed steer (202 echoes preempt: true) that seals and injects, where the sibling steering.spec test proves the unescalated path waits for run end instead. - Bubble in-place arm: an ordinary steer (202 with no preempt echo) waits as a bubble, POST /chat/steer/arm answers armed: true, the bubble relabels in place (same single bubble, same text, escalation no longer offered on reopen), and the armed steer seals mid-stream. - Always-interrupt toggle: flipped from a waiting row's overflow menu, plain Enter now produces a preempt: true steer that seals in the SAME run, and the menu offers the way back. An afterEach clears the localStorage preference so a mid-test failure cannot leak preempt-by-default into the rest of the serial suite. All three verified locally through the full harness (real backend, mock LLM, seeded DB): 3 passed in 27s. * feat: dedicated escalation arrow + shortcut, menu split into actions and preferences The escalation was still half-hidden: the bubble only offered it inside the overflow menu, and the tooltip taught the composer chord, which does a different thing (interrupts with typed text, not this chip). Three changes make it a first-class command: - A shared EscalateNowButton (circular arrow, ghost-bordered like the composer's interrupt control) is always visible on BOTH surfaces: beside each queued row's Steer primary and on every waiting steer bubble next to its menu. It disappears once a steer is interrupting. - A dedicated registry shortcut, escalateSteer (Cmd/Ctrl+Shift+.), editing-allowed and rebindable like every other action. Deliberately NOT an Enter chord: the composer owns every Enter chord, and the yield design rests on no default binding using Enter besides submit. Its handler clicks the newest enabled arrow control (bubbles beat queued rows), so the shortcut can never diverge from the button, and the arrow's tooltip teaches THIS command via the registry display. - The overflow menus separate one-off actions from sticky behavior changes: Edit, Cancel, Queue, then a smaller "Preferences" section holding the queueing and always-interrupt toggles, each with the standard InfoHoverCard reusing the Settings panel's descriptions. "Interrupt & steer now" leaves the menu entirely. 386 client tests green, including a menu-structure test locking the order and the absence of the escalation entry; bubble escalation tests drive the visible arrow. The e2e spec's bubble test now clicks the arrow, and a fourth test drives the dedicated shortcut end to end through a real mid-stream seal. * style: bind the escalation arrow to its message (variant A anatomy) Two same-weight circles in a row read as one control group, leaving the arrow's ownership ambiguous, and a floating arrow stops meaning anything once several messages stack. The shared control now carries variant A's anatomy: a thin divider binds a small SOLID arrow (filled, inverted) to the message region on its left, and the menu ellipsis stays a bare glyph, so the two affordances can no longer blur together — and the divider+arrow pairing repeats cleanly per chip at N messages. * chore: drop the unused within import CI lint caught * fix: advertise the escalation shortcut only while the control is live Codex on the e2e head, one P2: the tooltip appended the chord hint even while the button was disabled, advertising a shortcut that does nothing during an approval pause. The flagged control (InterruptNowButton) was since replaced by the shared EscalateNowButton, which inherited the pattern; the successor now omits the chord whenever the control is disabled, matching the rule the during-run hovercard already follows. * fix: harden steer escalation lifecycle and recovery * test(e2e): disambiguate accessible steer preferences * test: align abort persistence coverage with prerequisites * chore(i18n): remove obsolete steer race message * chore: normalize imports across steering changes * test: exercise stream integration on Redis Cluster * test: scope HITL checkpoints to generation * test: fix cluster cleanup and locale policy * fix: keep escalation visible during ask pauses * fix: fence recovery downgrade and stale predecessors * fix: require generation owner abort acknowledgement * fix: validate delayed preempt arms * test: align final escalation fixtures * fix: preserve in-memory predecessor abort handoff * fix: restore controls for recovered queued messages * test: cover recovered queue controls * fix: close final steering review gaps
1248 lines
43 KiB
TypeScript
1248 lines
43 KiB
TypeScript
import { expect, test } from '@playwright/test';
|
|
import type { Page, Response, Route } from '@playwright/test';
|
|
import {
|
|
isAgentGenerationStart,
|
|
MOCK_ENDPOINTS,
|
|
NEW_CHAT_PATH,
|
|
fetchJson,
|
|
getAccessToken,
|
|
selectMockEndpoint,
|
|
sendMessage,
|
|
} from './helpers';
|
|
|
|
const NO_PARENT = '00000000-0000-0000-0000-000000000000';
|
|
|
|
type TextContentPart = {
|
|
type?: string;
|
|
text?: string | { value?: string };
|
|
error?: string;
|
|
};
|
|
|
|
type E2EMessage = {
|
|
messageId: string;
|
|
parentMessageId?: string | null;
|
|
conversationId?: string | null;
|
|
text?: string;
|
|
content?: TextContentPart[];
|
|
isCreatedByUser?: boolean;
|
|
error?: boolean;
|
|
unfinished?: boolean;
|
|
};
|
|
|
|
type ForkResponse = {
|
|
conversation: {
|
|
conversationId?: string;
|
|
};
|
|
messages: E2EMessage[];
|
|
};
|
|
|
|
type JsonResponse = {
|
|
ok: boolean;
|
|
status: number;
|
|
text: string;
|
|
json: unknown;
|
|
};
|
|
|
|
const uniqueLabel = (name: string) => `${name}-${Date.now()}-${Math.floor(Math.random() * 1e6)}`;
|
|
|
|
const replyPrompt = (label: string) => `E2E_REPLY:${label}`;
|
|
const replyText = (label: string) => `E2E reply ${label}`;
|
|
const countedPrompt = (label: string) => `E2E_COUNTED_REPLY:${label}`;
|
|
const countedReplyText = (label: string, count: number) => `E2E counted reply ${label} #${count}`;
|
|
const slowPrompt = (label: string) => `E2E_SLOW_REPLY:${label}`;
|
|
const slowReplyPrefix = (label: string) => `E2E slow reply ${label}`;
|
|
const slowCountedPrompt = (label: string) => `E2E_SLOW_COUNTED_REPLY:${label}`;
|
|
const slowCountedReplyText = (label: string, count: number) =>
|
|
`E2E slow counted reply ${label} #${count}`;
|
|
|
|
const messagesView = (page: Page) => page.getByTestId('messages-view');
|
|
const messageRender = (page: Page, text: string) =>
|
|
page.locator('.message-render').filter({ hasText: text }).last();
|
|
const conversationPath = (conversationId: string) => `/c/${encodeURIComponent(conversationId)}`;
|
|
|
|
function contentText(part: TextContentPart): string {
|
|
if (typeof part.text === 'string') {
|
|
return part.text;
|
|
}
|
|
if (part.text?.value) {
|
|
return part.text.value;
|
|
}
|
|
return part.error ?? '';
|
|
}
|
|
|
|
function messageText(message: E2EMessage): string {
|
|
if (message.text) {
|
|
return message.text;
|
|
}
|
|
return message.content?.map(contentText).filter(Boolean).join('\n') ?? '';
|
|
}
|
|
|
|
function sseMessage(payload: unknown): string {
|
|
return `event: message\ndata: ${JSON.stringify(payload)}\n\n`;
|
|
}
|
|
|
|
function findMessage(messages: E2EMessage[], text: string, isCreatedByUser?: boolean): E2EMessage {
|
|
const message = messages.find((candidate) => {
|
|
const roleMatches =
|
|
isCreatedByUser === undefined || candidate.isCreatedByUser === isCreatedByUser;
|
|
return roleMatches && messageText(candidate).includes(text);
|
|
});
|
|
if (!message) {
|
|
throw new Error(
|
|
`Expected message containing "${text}". Saw:\n${messages.map(messageText).join('\n---\n')}`,
|
|
);
|
|
}
|
|
return message;
|
|
}
|
|
|
|
function expectParent(
|
|
messages: E2EMessage[],
|
|
childText: string,
|
|
parentText: string,
|
|
childIsUser?: boolean,
|
|
) {
|
|
const child = findMessage(messages, childText, childIsUser);
|
|
const parent = findMessage(messages, parentText);
|
|
expect(child.parentMessageId, `${childText} should be a child of ${parentText}`).toBe(
|
|
parent.messageId,
|
|
);
|
|
}
|
|
|
|
function expectNoFoldedMessages(messages: E2EMessage[]) {
|
|
const ids = new Set(messages.map((message) => message.messageId));
|
|
const folded = messages.filter((message) => {
|
|
const parentId = message.parentMessageId;
|
|
return parentId != null && parentId !== '' && parentId !== NO_PARENT && !ids.has(parentId);
|
|
});
|
|
expect(
|
|
folded.map((message) => ({
|
|
text: messageText(message),
|
|
messageId: message.messageId,
|
|
parentMessageId: message.parentMessageId,
|
|
})),
|
|
'messages must not render as parent-less folded children',
|
|
).toEqual([]);
|
|
|
|
const roots = messages.filter((message) => {
|
|
const parentId = message.parentMessageId;
|
|
return parentId == null || parentId === '' || parentId === NO_PARENT;
|
|
});
|
|
expect(
|
|
roots.map((message) => ({
|
|
text: messageText(message),
|
|
isCreatedByUser: message.isCreatedByUser,
|
|
})),
|
|
'only user messages should be roots',
|
|
).toEqual(roots.map(() => expect.objectContaining({ isCreatedByUser: true })));
|
|
}
|
|
|
|
async function expectVisibleMessages(page: Page, texts: string[]) {
|
|
for (const text of texts) {
|
|
await expect(messagesView(page).getByText(text)).toBeVisible({ timeout: 30000 });
|
|
}
|
|
}
|
|
|
|
async function reloadAndExpectMessages(page: Page, texts: string[]) {
|
|
await page.reload({ timeout: 10000 });
|
|
await expectVisibleMessages(page, texts);
|
|
}
|
|
|
|
async function revisitConversationAndExpectMessages(
|
|
page: Page,
|
|
conversationId: string,
|
|
texts: string[],
|
|
) {
|
|
await page.goto(NEW_CHAT_PATH, { timeout: 10000 });
|
|
await page.goto(conversationPath(conversationId), { timeout: 10000 });
|
|
await expectVisibleMessages(page, texts);
|
|
}
|
|
|
|
async function mockActiveOAuthResumeStream({
|
|
page,
|
|
authUrl,
|
|
conversationId,
|
|
parentMessageId,
|
|
pendingPrompt,
|
|
pendingUserMessageId,
|
|
postAuthRunId,
|
|
postAuthText,
|
|
}: {
|
|
page: Page;
|
|
authUrl: string;
|
|
conversationId: string;
|
|
parentMessageId: string;
|
|
pendingPrompt: string;
|
|
pendingUserMessageId: string;
|
|
postAuthRunId?: string;
|
|
postAuthText?: string;
|
|
}) {
|
|
const pendingResponseMessageId = `${pendingUserMessageId}_`;
|
|
const toolCallId = `${pendingUserMessageId}:Google-Workspace`;
|
|
const stepId = 'step_oauth_login_Google-Workspace';
|
|
const messageStepId = 'step_post_auth_message';
|
|
const resumeState = {
|
|
runSteps: [],
|
|
aggregatedContent: [],
|
|
responseMessageId: pendingResponseMessageId,
|
|
conversationId,
|
|
userMessage: {
|
|
messageId: pendingUserMessageId,
|
|
parentMessageId,
|
|
conversationId,
|
|
text: pendingPrompt,
|
|
},
|
|
replayEvents: [
|
|
{
|
|
event: 'on_run_step',
|
|
data: {
|
|
runId: 'USE_PRELIM_RESPONSE_MESSAGE_ID',
|
|
id: stepId,
|
|
type: 'tool_calls',
|
|
index: 0,
|
|
stepDetails: {
|
|
type: 'tool_calls',
|
|
tool_calls: [
|
|
{
|
|
id: toolCallId,
|
|
name: 'oauth_mcp_Google-Workspace',
|
|
type: 'tool_call_chunk',
|
|
},
|
|
],
|
|
},
|
|
},
|
|
},
|
|
{
|
|
event: 'on_run_step_delta',
|
|
data: {
|
|
id: stepId,
|
|
delta: {
|
|
type: 'tool_calls',
|
|
tool_calls: [
|
|
{
|
|
id: toolCallId,
|
|
name: 'oauth_mcp_Google-Workspace',
|
|
type: 'tool_call_chunk',
|
|
args: '',
|
|
},
|
|
],
|
|
auth: authUrl,
|
|
expires_at: Date.now() + 120000,
|
|
},
|
|
},
|
|
},
|
|
],
|
|
};
|
|
const streamPayloads: unknown[] = [
|
|
{
|
|
sync: true,
|
|
resumeState,
|
|
pendingEvents: [],
|
|
},
|
|
];
|
|
|
|
if (postAuthText) {
|
|
streamPayloads.push(
|
|
{
|
|
event: 'on_run_step',
|
|
data: {
|
|
runId: postAuthRunId ?? 'USE_PRELIM_RESPONSE_MESSAGE_ID',
|
|
id: messageStepId,
|
|
type: 'message_creation',
|
|
index: 0,
|
|
stepDetails: {
|
|
type: 'message_creation',
|
|
message_creation: {
|
|
message_id: `${pendingResponseMessageId}-post-auth`,
|
|
},
|
|
},
|
|
usage: null,
|
|
},
|
|
},
|
|
{
|
|
event: 'on_message_delta',
|
|
data: {
|
|
id: messageStepId,
|
|
delta: {
|
|
content: [
|
|
{
|
|
type: 'text',
|
|
text: postAuthText,
|
|
},
|
|
],
|
|
},
|
|
},
|
|
},
|
|
);
|
|
}
|
|
|
|
await page.route(`**/api/agents/chat/status/${conversationId}**`, (route) =>
|
|
route.fulfill({
|
|
status: 200,
|
|
contentType: 'application/json',
|
|
body: JSON.stringify({
|
|
active: true,
|
|
streamId: conversationId,
|
|
status: 'running',
|
|
aggregatedContent: [],
|
|
createdAt: Date.now(),
|
|
resumeState,
|
|
}),
|
|
}),
|
|
);
|
|
|
|
await page.route(`**/api/agents/chat/stream/${conversationId}**`, (route) =>
|
|
route.fulfill({
|
|
status: 200,
|
|
contentType: 'text/event-stream',
|
|
body: streamPayloads.map(sseMessage).join(''),
|
|
}),
|
|
);
|
|
}
|
|
|
|
async function mockPreCreatedOAuthStream({
|
|
page,
|
|
authUrl,
|
|
conversationId,
|
|
parentMessageId,
|
|
prompt,
|
|
serverUserMessageId,
|
|
responseMessageId,
|
|
postAuthText,
|
|
}: {
|
|
page: Page;
|
|
authUrl: string;
|
|
conversationId: string;
|
|
parentMessageId: string;
|
|
prompt: string;
|
|
serverUserMessageId: string;
|
|
responseMessageId: string;
|
|
postAuthText: string;
|
|
}) {
|
|
const toolCallId = `${serverUserMessageId}:Google-Workspace`;
|
|
const oauthStepId = 'step_oauth_login_Google-Workspace';
|
|
const reasoningStepId = 'step_post_auth_reasoning';
|
|
const messageStepId = 'step_post_auth_message';
|
|
const payloads = [
|
|
{
|
|
event: 'on_run_step',
|
|
data: {
|
|
runId: 'USE_PRELIM_RESPONSE_MESSAGE_ID',
|
|
id: oauthStepId,
|
|
type: 'tool_calls',
|
|
index: 0,
|
|
stepDetails: {
|
|
type: 'tool_calls',
|
|
tool_calls: [
|
|
{
|
|
id: toolCallId,
|
|
name: 'oauth_mcp_Google-Workspace',
|
|
type: 'tool_call_chunk',
|
|
},
|
|
],
|
|
},
|
|
},
|
|
},
|
|
{
|
|
event: 'on_run_step_delta',
|
|
data: {
|
|
id: oauthStepId,
|
|
delta: {
|
|
type: 'tool_calls',
|
|
tool_calls: [
|
|
{
|
|
id: toolCallId,
|
|
name: 'oauth_mcp_Google-Workspace',
|
|
type: 'tool_call_chunk',
|
|
args: '',
|
|
},
|
|
],
|
|
auth: authUrl,
|
|
expires_at: Date.now() + 120000,
|
|
},
|
|
},
|
|
},
|
|
{
|
|
event: 'on_run_step_completed',
|
|
data: {
|
|
result: {
|
|
id: oauthStepId,
|
|
index: 0,
|
|
tool_call: {
|
|
id: toolCallId,
|
|
name: 'oauth_mcp_Google-Workspace',
|
|
args: '',
|
|
output: 'OAuth authentication completed',
|
|
type: 'tool_call',
|
|
},
|
|
},
|
|
},
|
|
},
|
|
{
|
|
created: true,
|
|
message: {
|
|
messageId: serverUserMessageId,
|
|
parentMessageId,
|
|
conversationId,
|
|
sender: 'User',
|
|
text: prompt,
|
|
isCreatedByUser: true,
|
|
},
|
|
streamId: conversationId,
|
|
},
|
|
{
|
|
event: 'on_run_step',
|
|
data: {
|
|
runId: responseMessageId,
|
|
id: reasoningStepId,
|
|
type: 'message_creation',
|
|
index: 0,
|
|
stepDetails: {
|
|
type: 'message_creation',
|
|
message_creation: {
|
|
message_id: `${responseMessageId}-reasoning`,
|
|
},
|
|
},
|
|
usage: null,
|
|
},
|
|
},
|
|
{
|
|
event: 'on_reasoning_delta',
|
|
data: {
|
|
id: reasoningStepId,
|
|
delta: {
|
|
content: [
|
|
{
|
|
type: 'think',
|
|
think: 'The user completed OAuth.',
|
|
},
|
|
],
|
|
},
|
|
},
|
|
},
|
|
{
|
|
event: 'on_run_step',
|
|
data: {
|
|
runId: responseMessageId,
|
|
id: messageStepId,
|
|
type: 'message_creation',
|
|
index: 1,
|
|
stepDetails: {
|
|
type: 'message_creation',
|
|
message_creation: {
|
|
message_id: `${responseMessageId}-message`,
|
|
},
|
|
},
|
|
usage: null,
|
|
},
|
|
},
|
|
{
|
|
event: 'on_message_delta',
|
|
data: {
|
|
id: messageStepId,
|
|
delta: {
|
|
content: [
|
|
{
|
|
type: 'text',
|
|
text: postAuthText,
|
|
index: 1,
|
|
},
|
|
],
|
|
},
|
|
},
|
|
},
|
|
];
|
|
|
|
await page.route(`**/api/agents/chat/stream/${conversationId}**`, (route) =>
|
|
route.fulfill({
|
|
status: 200,
|
|
contentType: 'text/event-stream',
|
|
body: payloads.map(sseMessage).join(''),
|
|
}),
|
|
);
|
|
}
|
|
|
|
async function openMockChat(page: Page) {
|
|
await page.goto(NEW_CHAT_PATH, { timeout: 10000 });
|
|
await selectMockEndpoint(page, MOCK_ENDPOINTS[0]);
|
|
}
|
|
|
|
function isAgentGenerationResponse(response: Response, expectedStatus: number) {
|
|
const { pathname } = new URL(response.url());
|
|
const isAgentsChat = pathname === '/api/agents/chat' || pathname.startsWith('/api/agents/chat/');
|
|
return (
|
|
response.request().method() === 'POST' &&
|
|
isAgentsChat &&
|
|
!pathname.endsWith('/abort') &&
|
|
response.status() === expectedStatus
|
|
);
|
|
}
|
|
|
|
async function waitForGenerationStart(page: Page, action: () => Promise<void>): Promise<Response> {
|
|
const [response] = await Promise.all([
|
|
page.waitForResponse(isAgentGenerationStart, { timeout: 30000 }),
|
|
action(),
|
|
]);
|
|
expect(response.ok()).toBeTruthy();
|
|
return response;
|
|
}
|
|
|
|
async function sendAndExpectReply(page: Page, prompt: string, expectedReply: string) {
|
|
const response = await sendMessage(page, prompt);
|
|
expect(response.ok()).toBeTruthy();
|
|
await expect(messagesView(page).getByText(expectedReply)).toBeVisible({ timeout: 30000 });
|
|
}
|
|
|
|
async function submitMessageExpectingGenerationFailure(
|
|
page: Page,
|
|
prompt: string,
|
|
expectedStatus: number,
|
|
) {
|
|
const input = page.getByRole('textbox', { name: 'Message input' });
|
|
await expect(input).toBeEnabled({ timeout: 30000 });
|
|
await input.click();
|
|
await input.fill(prompt);
|
|
const [response] = await Promise.all([
|
|
page.waitForResponse((res) => isAgentGenerationResponse(res, expectedStatus), {
|
|
timeout: 30000,
|
|
}),
|
|
input.press('Enter'),
|
|
]);
|
|
return response;
|
|
}
|
|
|
|
async function conversationIdFromPage(page: Page): Promise<string> {
|
|
await expect(page).toHaveURL(/\/c\/(?!new)[0-9a-fA-F-]{36}$/);
|
|
const id = new URL(page.url()).pathname.split('/').pop();
|
|
if (!id) {
|
|
throw new Error(`Could not parse conversation id from ${page.url()}`);
|
|
}
|
|
return id;
|
|
}
|
|
|
|
async function fetchMessages(
|
|
page: Page,
|
|
conversationId: string,
|
|
accessToken?: string,
|
|
): Promise<E2EMessage[]> {
|
|
const token = accessToken ?? (await getAccessToken(page));
|
|
return fetchJson<E2EMessage[]>(
|
|
page,
|
|
`/api/messages/${encodeURIComponent(conversationId)}`,
|
|
token,
|
|
);
|
|
}
|
|
|
|
async function postJsonWithStatus(page: Page, path: string, token: string, body: unknown) {
|
|
return page.evaluate(
|
|
async ({ accessToken, requestBody, urlPath }): Promise<JsonResponse> => {
|
|
const response = await fetch(urlPath, {
|
|
method: 'POST',
|
|
credentials: 'include',
|
|
headers: {
|
|
Authorization: `Bearer ${accessToken}`,
|
|
'Content-Type': 'application/json',
|
|
},
|
|
body: JSON.stringify(requestBody),
|
|
});
|
|
const text = await response.text();
|
|
let json: unknown = null;
|
|
try {
|
|
json = text ? JSON.parse(text) : null;
|
|
} catch {
|
|
json = null;
|
|
}
|
|
return { ok: response.ok, status: response.status, text, json };
|
|
},
|
|
{ accessToken: token, requestBody: body, urlPath: path },
|
|
);
|
|
}
|
|
|
|
async function waitForMessages(
|
|
page: Page,
|
|
conversationId: string,
|
|
predicate: (messages: E2EMessage[]) => boolean,
|
|
description: string,
|
|
): Promise<E2EMessage[]> {
|
|
let latest: E2EMessage[] = [];
|
|
const token = await getAccessToken(page);
|
|
for (let attempt = 0; attempt < 80; attempt++) {
|
|
latest = await fetchMessages(page, conversationId, token);
|
|
if (predicate(latest)) {
|
|
return latest;
|
|
}
|
|
await new Promise((resolve) => setTimeout(resolve, 250));
|
|
}
|
|
|
|
throw new Error(
|
|
`Timed out waiting for ${description}. Latest messages:\n${latest
|
|
.map(
|
|
(message) => `${message.messageId} <- ${message.parentMessageId}: ${messageText(message)}`,
|
|
)
|
|
.join('\n')}`,
|
|
);
|
|
}
|
|
|
|
async function clickMessageTitleButton(page: Page, messageTextValue: string, title: string) {
|
|
const render = messageRender(page, messageTextValue);
|
|
await render.scrollIntoViewIfNeeded();
|
|
await render.hover();
|
|
await render.locator(`button[title="${title}"]`).last().click();
|
|
}
|
|
|
|
async function clickSibling(page: Page, messageTextValue: string, direction: 'Previous' | 'Next') {
|
|
const render = messageRender(page, messageTextValue);
|
|
await render.scrollIntoViewIfNeeded();
|
|
await render.hover();
|
|
await render.getByRole('button', { name: `${direction} sibling message` }).click();
|
|
}
|
|
|
|
async function expectCanCycleSiblingTexts(page: Page, previousText: string, nextText: string) {
|
|
const previous = messagesView(page).getByText(previousText);
|
|
const next = messagesView(page).getByText(nextText);
|
|
if (await previous.isVisible()) {
|
|
await clickSibling(page, previousText, 'Next');
|
|
await expect(next).toBeVisible();
|
|
await clickSibling(page, nextText, 'Previous');
|
|
await expect(previous).toBeVisible();
|
|
return;
|
|
}
|
|
|
|
if (await next.isVisible()) {
|
|
await clickSibling(page, nextText, 'Previous');
|
|
await expect(previous).toBeVisible();
|
|
await clickSibling(page, previousText, 'Next');
|
|
await expect(next).toBeVisible();
|
|
return;
|
|
}
|
|
|
|
throw new Error(`Expected either sibling "${previousText}" or "${nextText}" to be visible`);
|
|
}
|
|
|
|
async function clickForkVisibleMessages(
|
|
page: Page,
|
|
messageTextValue: string,
|
|
): Promise<ForkResponse> {
|
|
const render = messageRender(page, messageTextValue);
|
|
await render.scrollIntoViewIfNeeded();
|
|
await render.hover();
|
|
await render.getByRole('button', { name: 'Open Fork Menu' }).click();
|
|
|
|
const [response] = await Promise.all([
|
|
page.waitForResponse(
|
|
(res) =>
|
|
res.request().method() === 'POST' &&
|
|
res.url().includes('/api/convos/fork') &&
|
|
res.status() === 200,
|
|
{ timeout: 30000 },
|
|
),
|
|
page.getByRole('button', { name: 'Visible messages only', exact: true }).click(),
|
|
]);
|
|
|
|
return (await response.json()) as ForkResponse;
|
|
}
|
|
|
|
test.describe('message tree stream operations', () => {
|
|
test.setTimeout(180000);
|
|
|
|
test('streams follow-ups and keeps an aborted response as the next parent', async ({ page }) => {
|
|
const label = uniqueLabel('abort');
|
|
const firstPrompt = replyPrompt(`${label}-first`);
|
|
const firstReply = replyText(`${label}-first`);
|
|
const secondPrompt = replyPrompt(`${label}-second`);
|
|
const secondReply = replyText(`${label}-second`);
|
|
const abortPrompt = slowPrompt(`${label}-stop`);
|
|
const abortReply = slowReplyPrefix(`${label}-stop`);
|
|
const afterAbortPrompt = replyPrompt(`${label}-after-stop`);
|
|
const afterAbortReply = replyText(`${label}-after-stop`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, firstPrompt, firstReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
await sendAndExpectReply(page, secondPrompt, secondReply);
|
|
|
|
const slowStart = await sendMessage(page, abortPrompt);
|
|
expect(slowStart.ok()).toBeTruthy();
|
|
await expect(messagesView(page).getByText(abortReply)).toBeVisible({ timeout: 30000 });
|
|
|
|
const [abortResponse] = await Promise.all([
|
|
page.waitForResponse(
|
|
(response) =>
|
|
response.request().method() === 'POST' &&
|
|
response.url().includes('/api/agents/chat/abort'),
|
|
{ timeout: 30000 },
|
|
),
|
|
page.getByRole('button', { name: 'Stop generating' }).click(),
|
|
]);
|
|
expect(abortResponse.ok()).toBeTruthy();
|
|
await expect(page.getByRole('button', { name: 'Stop generating' })).toBeHidden({
|
|
timeout: 30000,
|
|
});
|
|
|
|
let messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(abortReply)),
|
|
'aborted response to persist',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, secondPrompt, firstReply, true);
|
|
expectParent(messages, abortReply, abortPrompt, false);
|
|
|
|
await sendAndExpectReply(page, afterAbortPrompt, afterAbortReply);
|
|
messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(afterAbortReply)),
|
|
'follow-up after abort',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, afterAbortPrompt, abortReply, true);
|
|
expectParent(messages, afterAbortReply, afterAbortPrompt, false);
|
|
|
|
await reloadAndExpectMessages(page, [firstReply, secondReply, abortReply, afterAbortReply]);
|
|
await revisitConversationAndExpectMessages(page, conversationId, [
|
|
firstReply,
|
|
secondReply,
|
|
abortReply,
|
|
afterAbortReply,
|
|
]);
|
|
});
|
|
|
|
test('regenerates assistant siblings, cycles branches, follows up, and forks the visible branch', async ({
|
|
page,
|
|
}) => {
|
|
const label = uniqueLabel('regen');
|
|
const prompt = countedPrompt(label);
|
|
const firstReply = countedReplyText(label, 1);
|
|
const regeneratedReply = countedReplyText(label, 2);
|
|
const followPrompt = replyPrompt(`${label}-follow`);
|
|
const followReply = replyText(`${label}-follow`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, prompt, firstReply);
|
|
const originalConversationId = await conversationIdFromPage(page);
|
|
|
|
await waitForGenerationStart(page, () =>
|
|
clickMessageTitleButton(page, firstReply, 'Regenerate'),
|
|
);
|
|
await expect(messagesView(page).getByText(regeneratedReply)).toBeVisible({ timeout: 30000 });
|
|
|
|
await clickSibling(page, regeneratedReply, 'Previous');
|
|
await expect(messagesView(page).getByText(firstReply)).toBeVisible();
|
|
await expect(messagesView(page).getByText(regeneratedReply)).toBeHidden();
|
|
await clickSibling(page, firstReply, 'Next');
|
|
await expect(messagesView(page).getByText(regeneratedReply)).toBeVisible();
|
|
|
|
await sendAndExpectReply(page, followPrompt, followReply);
|
|
let messages = await waitForMessages(
|
|
page,
|
|
originalConversationId,
|
|
(items) => items.some((message) => messageText(message).includes(followReply)),
|
|
'follow-up after regenerate',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, firstReply, prompt, false);
|
|
expectParent(messages, regeneratedReply, prompt, false);
|
|
expectParent(messages, followPrompt, regeneratedReply, true);
|
|
expectParent(messages, followReply, followPrompt, false);
|
|
|
|
const userMessage = findMessage(messages, prompt, true);
|
|
const assistantSiblings = messages.filter(
|
|
(message) => message.parentMessageId === userMessage.messageId && !message.isCreatedByUser,
|
|
);
|
|
expect(assistantSiblings.map(messageText).sort()).toEqual(
|
|
[firstReply, regeneratedReply].sort(),
|
|
);
|
|
|
|
await reloadAndExpectMessages(page, [regeneratedReply, followReply]);
|
|
await revisitConversationAndExpectMessages(page, originalConversationId, [
|
|
regeneratedReply,
|
|
followReply,
|
|
]);
|
|
await clickSibling(page, regeneratedReply, 'Previous');
|
|
await expect(messagesView(page).getByText(firstReply)).toBeVisible();
|
|
const fork = await clickForkVisibleMessages(page, firstReply);
|
|
const forkedConversationId = fork.conversation.conversationId;
|
|
if (!forkedConversationId) {
|
|
throw new Error('Expected fork response to include a conversation id');
|
|
}
|
|
await expect(page).toHaveURL(new RegExp(`/c/${forkedConversationId}$`));
|
|
|
|
messages = fork.messages;
|
|
expectNoFoldedMessages(messages);
|
|
expect(messages.some((message) => messageText(message).includes(firstReply))).toBe(true);
|
|
expect(messages.some((message) => messageText(message).includes(regeneratedReply))).toBe(false);
|
|
expect(messages.some((message) => messageText(message).includes(followReply))).toBe(false);
|
|
});
|
|
|
|
test('shows the regenerating response immediately when regenerating a non-latest sibling', async ({
|
|
page,
|
|
}) => {
|
|
// A parent with multiple siblings, switched to an older one, then
|
|
// regenerated: the optimistic slice drops the target but keeps the other
|
|
// siblings, so the child count is unchanged and MultiMessage's
|
|
// length-change reset never fires. Without an explicit focus the view stays
|
|
// on the kept (newer) sibling and the streaming response is hidden until the
|
|
// server restores the dropped sibling at finalize. A slow reply keeps the
|
|
// stream open long enough to assert the during-stream state.
|
|
const label = uniqueLabel('regen-nonlatest');
|
|
const prompt = slowCountedPrompt(label);
|
|
const reply1 = slowCountedReplyText(label, 1);
|
|
const reply2 = slowCountedReplyText(label, 2);
|
|
const reply3 = slowCountedReplyText(label, 3);
|
|
const stopButton = page.getByRole('button', { name: 'Stop generating' });
|
|
|
|
await openMockChat(page);
|
|
await sendMessage(page, prompt);
|
|
await expect(messagesView(page).getByText(reply1)).toBeVisible({ timeout: 30000 });
|
|
await expect(stopButton).toBeHidden({ timeout: 30000 });
|
|
|
|
// First regenerate adds a sibling; the parent now has two responses.
|
|
await waitForGenerationStart(page, () => clickMessageTitleButton(page, reply1, 'Regenerate'));
|
|
await expect(messagesView(page).getByText(reply2)).toBeVisible({ timeout: 30000 });
|
|
await expect(stopButton).toBeHidden({ timeout: 30000 });
|
|
|
|
// View the older sibling, then regenerate it (the non-latest one).
|
|
await clickSibling(page, reply2, 'Previous');
|
|
await expect(messagesView(page).getByText(reply1)).toBeVisible();
|
|
await expect(messagesView(page).getByText(reply2)).toBeHidden();
|
|
|
|
await waitForGenerationStart(page, () => clickMessageTitleButton(page, reply1, 'Regenerate'));
|
|
// Still streaming (slow reply): the regenerating response must be visible
|
|
// now — not the kept sibling — and well before finalize.
|
|
await expect(stopButton).toBeVisible({ timeout: 30000 });
|
|
await expect(messagesView(page).getByText(reply3)).toBeVisible({ timeout: 4000 });
|
|
await expect(messagesView(page).getByText(reply2)).toBeHidden();
|
|
|
|
// It stays put after finalize too.
|
|
await expect(stopButton).toBeHidden({ timeout: 30000 });
|
|
await expect(messagesView(page).getByText(reply3)).toBeVisible();
|
|
});
|
|
|
|
test('does not flash the kept sibling before the created event when regenerating a non-latest sibling', async ({
|
|
page,
|
|
}) => {
|
|
// Same hazard at the optimistic (pre-`created`) render: useChatFunctions
|
|
// appends the placeholder and renders it before the request resolves. Gate
|
|
// the regenerate request so only that optimistic frame is on screen — with
|
|
// no `created` event to fall back on — and assert the kept sibling is
|
|
// already gone (the regenerating placeholder took its place). This isolates
|
|
// the optimistic focus; the createdHandler focus cannot mask a regression
|
|
// because `created` never arrives during the assertion.
|
|
const label = uniqueLabel('regen-precreated');
|
|
const prompt = countedPrompt(label);
|
|
const reply1 = countedReplyText(label, 1);
|
|
const reply2 = countedReplyText(label, 2);
|
|
const reply3 = countedReplyText(label, 3);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, prompt, reply1);
|
|
|
|
await waitForGenerationStart(page, () => clickMessageTitleButton(page, reply1, 'Regenerate'));
|
|
await expect(messagesView(page).getByText(reply2)).toBeVisible({ timeout: 30000 });
|
|
|
|
await clickSibling(page, reply2, 'Previous');
|
|
await expect(messagesView(page).getByText(reply1)).toBeVisible();
|
|
await expect(messagesView(page).getByText(reply2)).toBeHidden();
|
|
|
|
// Hold the SSE stream so the `created` event cannot arrive: the optimistic
|
|
// render is all there is. The chat POST returns a stream id; the stream
|
|
// itself is a separate GET (/api/agents/chat/stream/<id>) — gate that one,
|
|
// not the POST, which must complete to hand back the id.
|
|
let release = () => {};
|
|
const gate = new Promise<void>((resolve) => {
|
|
release = resolve;
|
|
});
|
|
await page.route(/\/api\/agents\/chat\/stream\//, async (route: Route) => {
|
|
await gate;
|
|
return route.continue();
|
|
});
|
|
|
|
try {
|
|
await clickMessageTitleButton(page, reply1, 'Regenerate');
|
|
// The stream is gated, so the regenerating response has no text yet.
|
|
await expect(messagesView(page).getByText(reply3)).toBeHidden();
|
|
// Pre-`created` window: the kept sibling must not be the one on screen —
|
|
// the regenerating placeholder has already taken its place.
|
|
await expect(messagesView(page).getByText(reply2)).toBeHidden({ timeout: 5000 });
|
|
await expect(messagesView(page).getByText(reply1)).toBeHidden();
|
|
} finally {
|
|
release();
|
|
}
|
|
|
|
await expect(messagesView(page).getByText(reply3)).toBeVisible({ timeout: 30000 });
|
|
});
|
|
|
|
test('resumes pending OAuth on the selected older branch after reload', async ({ page }) => {
|
|
const label = uniqueLabel('oauth-branch');
|
|
const rootPrompt = countedPrompt(`${label}-root`);
|
|
const firstReply = countedReplyText(`${label}-root`, 1);
|
|
const regeneratedReply = countedReplyText(`${label}-root`, 2);
|
|
const followPrompt = replyPrompt(`${label}-follow`);
|
|
const followReply = replyText(`${label}-follow`);
|
|
const pendingPrompt = replyPrompt(`${label}-oauth`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, rootPrompt, firstReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
await sendAndExpectReply(page, followPrompt, followReply);
|
|
|
|
await waitForGenerationStart(page, () =>
|
|
clickMessageTitleButton(page, firstReply, 'Regenerate'),
|
|
);
|
|
await expect(messagesView(page).getByText(regeneratedReply)).toBeVisible({ timeout: 30000 });
|
|
|
|
await clickSibling(page, regeneratedReply, 'Previous');
|
|
await expectVisibleMessages(page, [firstReply, followPrompt, followReply]);
|
|
await expect(messagesView(page).getByText(regeneratedReply)).toBeHidden();
|
|
|
|
const messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) =>
|
|
items.some((message) => messageText(message).includes(followReply)) &&
|
|
items.some((message) => messageText(message).includes(regeneratedReply)),
|
|
'two-branch conversation',
|
|
);
|
|
const branchOneTail = findMessage(messages, followReply, false);
|
|
|
|
await mockActiveOAuthResumeStream({
|
|
page,
|
|
conversationId,
|
|
parentMessageId: branchOneTail.messageId,
|
|
pendingPrompt,
|
|
pendingUserMessageId: `${label}-pending-user`,
|
|
authUrl: `https://auth.example.test/${label}`,
|
|
});
|
|
|
|
await page.reload({ timeout: 10000 });
|
|
await expectVisibleMessages(page, [firstReply, followPrompt, followReply, pendingPrompt]);
|
|
await expect(messagesView(page).getByText(regeneratedReply)).toBeHidden();
|
|
});
|
|
|
|
test('keeps existing messages visible when resumed OAuth streams post-auth content', async ({
|
|
page,
|
|
}) => {
|
|
const label = uniqueLabel('oauth-post-auth');
|
|
const rootPrompt = replyPrompt(`${label}-root`);
|
|
const rootReply = replyText(`${label}-root`);
|
|
const pendingPrompt = replyPrompt(`${label}-pending`);
|
|
const postAuthText = `E2E post-auth OAuth reply ${label}`;
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, rootPrompt, rootReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
|
|
const messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(rootReply)),
|
|
'root reply before OAuth resume',
|
|
);
|
|
const branchTail = findMessage(messages, rootReply, false);
|
|
|
|
await mockActiveOAuthResumeStream({
|
|
page,
|
|
conversationId,
|
|
parentMessageId: branchTail.messageId,
|
|
pendingPrompt,
|
|
pendingUserMessageId: `${label}-pending-user`,
|
|
authUrl: `https://auth.example.test/${label}`,
|
|
postAuthRunId: `${label}-stable-response`,
|
|
postAuthText,
|
|
});
|
|
|
|
await page.reload({ timeout: 10000 });
|
|
await expectVisibleMessages(page, [rootPrompt, rootReply, pendingPrompt, postAuthText]);
|
|
});
|
|
|
|
test('keeps existing messages visible when OAuth completes before created', async ({ page }) => {
|
|
const label = uniqueLabel('oauth-pre-created');
|
|
const rootPrompt = replyPrompt(`${label}-root`);
|
|
const rootReply = replyText(`${label}-root`);
|
|
const followPrompt = replyPrompt(`${label}-follow`);
|
|
const postAuthText = `E2E pre-created OAuth reply ${label}`;
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, rootPrompt, rootReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
|
|
const messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(rootReply)),
|
|
'root reply before pre-created OAuth stream',
|
|
);
|
|
const branchTail = findMessage(messages, rootReply, false);
|
|
|
|
await mockPreCreatedOAuthStream({
|
|
page,
|
|
conversationId,
|
|
parentMessageId: branchTail.messageId,
|
|
prompt: followPrompt,
|
|
serverUserMessageId: `${label}-server-user`,
|
|
responseMessageId: `${label}-response`,
|
|
authUrl: `https://auth.example.test/${label}`,
|
|
postAuthText,
|
|
});
|
|
|
|
const response = await sendMessage(page, followPrompt);
|
|
expect(response.ok()).toBeTruthy();
|
|
await expectVisibleMessages(page, [rootPrompt, rootReply, followPrompt, postAuthText]);
|
|
});
|
|
|
|
test('long threads retain regenerated and save-and-submit branches after revisit', async ({
|
|
page,
|
|
}) => {
|
|
const label = uniqueLabel('save-submit');
|
|
const rootPrompt = replyPrompt(`${label}-root`);
|
|
const rootReply = replyText(`${label}-root`);
|
|
const firstPrompt = replyPrompt(`${label}-first`);
|
|
const firstReply = replyText(`${label}-first`);
|
|
const middlePrompt = replyPrompt(`${label}-middle`);
|
|
const middleReply = replyText(`${label}-middle`);
|
|
const fourthPrompt = replyPrompt(`${label}-fourth`);
|
|
const fourthReply = replyText(`${label}-fourth`);
|
|
const tailPrompt = countedPrompt(`${label}-tail`);
|
|
const tailReply = countedReplyText(`${label}-tail`, 1);
|
|
const regeneratedTailReply = countedReplyText(`${label}-tail`, 2);
|
|
const editedMiddlePrompt = replyPrompt(`${label}-middle-edited`);
|
|
const editedMiddleReply = replyText(`${label}-middle-edited`);
|
|
const afterEditPrompt = replyPrompt(`${label}-after-edit`);
|
|
const afterEditReply = replyText(`${label}-after-edit`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, rootPrompt, rootReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
await sendAndExpectReply(page, firstPrompt, firstReply);
|
|
await sendAndExpectReply(page, middlePrompt, middleReply);
|
|
await sendAndExpectReply(page, fourthPrompt, fourthReply);
|
|
await sendAndExpectReply(page, tailPrompt, tailReply);
|
|
|
|
await waitForGenerationStart(page, () =>
|
|
clickMessageTitleButton(page, tailReply, 'Regenerate'),
|
|
);
|
|
await expect(messagesView(page).getByText(regeneratedTailReply)).toBeVisible({
|
|
timeout: 30000,
|
|
});
|
|
|
|
await clickMessageTitleButton(page, middlePrompt, 'Edit');
|
|
const editor = page.getByTestId('message-text-editor');
|
|
await expect(editor).toBeVisible();
|
|
await editor.fill(editedMiddlePrompt);
|
|
await waitForGenerationStart(page, () =>
|
|
page.getByRole('button', { name: 'Save & Submit' }).click(),
|
|
);
|
|
await expect(messagesView(page).getByText(editedMiddleReply)).toBeVisible({ timeout: 30000 });
|
|
|
|
let messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(editedMiddleReply)),
|
|
'save-and-submit edited branch',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, firstPrompt, rootReply, true);
|
|
expectParent(messages, firstReply, firstPrompt, false);
|
|
expectParent(messages, middlePrompt, firstReply, true);
|
|
expectParent(messages, middleReply, middlePrompt, false);
|
|
expectParent(messages, fourthPrompt, middleReply, true);
|
|
expectParent(messages, fourthReply, fourthPrompt, false);
|
|
expectParent(messages, tailPrompt, fourthReply, true);
|
|
expectParent(messages, tailReply, tailPrompt, false);
|
|
expectParent(messages, regeneratedTailReply, tailPrompt, false);
|
|
expectParent(messages, editedMiddlePrompt, firstReply, true);
|
|
expectParent(messages, editedMiddleReply, editedMiddlePrompt, false);
|
|
|
|
await clickSibling(page, editedMiddlePrompt, 'Previous');
|
|
await expectVisibleMessages(page, [middlePrompt, fourthReply]);
|
|
await expectCanCycleSiblingTexts(page, tailReply, regeneratedTailReply);
|
|
await clickSibling(page, middlePrompt, 'Next');
|
|
await expectVisibleMessages(page, [editedMiddlePrompt, editedMiddleReply]);
|
|
|
|
await sendAndExpectReply(page, afterEditPrompt, afterEditReply);
|
|
messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(afterEditReply)),
|
|
'follow-up after save-and-submit branch',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, afterEditPrompt, editedMiddleReply, true);
|
|
expectParent(messages, afterEditReply, afterEditPrompt, false);
|
|
expect(messages.some((message) => messageText(message).includes(tailReply))).toBe(true);
|
|
expect(messages.some((message) => messageText(message).includes(regeneratedTailReply))).toBe(
|
|
true,
|
|
);
|
|
|
|
await reloadAndExpectMessages(page, [rootReply, firstReply, editedMiddleReply, afterEditReply]);
|
|
await revisitConversationAndExpectMessages(page, conversationId, [
|
|
rootReply,
|
|
firstReply,
|
|
editedMiddleReply,
|
|
afterEditReply,
|
|
]);
|
|
await clickSibling(page, editedMiddlePrompt, 'Previous');
|
|
await expectVisibleMessages(page, [middlePrompt, fourthReply]);
|
|
await expectCanCycleSiblingTexts(page, tailReply, regeneratedTailReply);
|
|
await clickSibling(page, middlePrompt, 'Next');
|
|
await expectVisibleMessages(page, [editedMiddlePrompt, editedMiddleReply, afterEditReply]);
|
|
});
|
|
|
|
test('error responses remain valid parents for follow-ups', async ({ page }) => {
|
|
const label = uniqueLabel('error');
|
|
const basePrompt = replyPrompt(`${label}-base`);
|
|
const baseReply = replyText(`${label}-base`);
|
|
const errorPrompt = `E2E_FORCED_ERROR:${label}`;
|
|
const errorText = `E2E forced stream error ${label}`;
|
|
const afterErrorPrompt = replyPrompt(`${label}-after-error`);
|
|
const afterErrorReply = replyText(`${label}-after-error`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, basePrompt, baseReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
|
|
await sendAndExpectReply(page, errorPrompt, errorText);
|
|
await expect(messagesView(page).getByText(errorText)).toBeVisible({ timeout: 30000 });
|
|
|
|
await sendAndExpectReply(page, afterErrorPrompt, afterErrorReply);
|
|
const messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(afterErrorReply)),
|
|
'follow-up after error',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, errorPrompt, baseReply, true);
|
|
expectParent(messages, errorText, errorPrompt, false);
|
|
expectParent(messages, afterErrorPrompt, errorText, true);
|
|
expectParent(messages, afterErrorReply, afterErrorPrompt, false);
|
|
|
|
await reloadAndExpectMessages(page, [baseReply, errorText, afterErrorReply]);
|
|
await revisitConversationAndExpectMessages(page, conversationId, [
|
|
baseReply,
|
|
errorText,
|
|
afterErrorReply,
|
|
]);
|
|
});
|
|
|
|
test('rejects normal follow-ups whose parent is a preliminary assistant placeholder', async ({
|
|
page,
|
|
}) => {
|
|
const label = uniqueLabel('placeholder-parent');
|
|
const basePrompt = replyPrompt(`${label}-base`);
|
|
const baseReply = replyText(`${label}-base`);
|
|
const followPrompt = replyPrompt(`${label}-follow`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, basePrompt, baseReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
const token = await getAccessToken(page);
|
|
|
|
const beforeMessages = await fetchMessages(page, conversationId, token);
|
|
const stableParent = findMessage(beforeMessages, baseReply, false);
|
|
const response = await postJsonWithStatus(
|
|
page,
|
|
`/api/agents/chat/${encodeURIComponent(MOCK_ENDPOINTS[0].label)}`,
|
|
token,
|
|
{
|
|
text: followPrompt,
|
|
sender: 'User',
|
|
clientTimestamp: new Date().toLocaleString('sv').replace(' ', 'T'),
|
|
isCreatedByUser: true,
|
|
parentMessageId: `${stableParent.messageId}_`,
|
|
conversationId,
|
|
messageId: `${label}-user-message`,
|
|
endpoint: MOCK_ENDPOINTS[0].label,
|
|
endpointType: 'custom',
|
|
model: MOCK_ENDPOINTS[0].model,
|
|
spec: 'e2e-mock-provider-a',
|
|
isTemporary: false,
|
|
isRegenerate: false,
|
|
error: false,
|
|
},
|
|
);
|
|
|
|
expect(response.status).toBe(409);
|
|
expect(response.json).toEqual(
|
|
expect.objectContaining({
|
|
error: expect.stringContaining('selected parent response is still being saved'),
|
|
}),
|
|
);
|
|
|
|
const afterMessages = await fetchMessages(page, conversationId, token);
|
|
expect(afterMessages.map((message) => message.messageId).sort()).toEqual(
|
|
beforeMessages.map((message) => message.messageId).sort(),
|
|
);
|
|
expect(afterMessages.some((message) => messageText(message).includes(followPrompt))).toBe(
|
|
false,
|
|
);
|
|
expectNoFoldedMessages(afterMessages);
|
|
});
|
|
|
|
test('generation-start failures recover without folding the next follow-up', async ({ page }) => {
|
|
const label = uniqueLabel('start-error');
|
|
const basePrompt = replyPrompt(`${label}-base`);
|
|
const baseReply = replyText(`${label}-base`);
|
|
const failedPrompt = replyPrompt(`${label}-failed-start`);
|
|
const failedText = `E2E generation start failure ${label}`;
|
|
const afterFailurePrompt = replyPrompt(`${label}-after-start-failure`);
|
|
const afterFailureReply = replyText(`${label}-after-start-failure`);
|
|
|
|
await openMockChat(page);
|
|
await sendAndExpectReply(page, basePrompt, baseReply);
|
|
const conversationId = await conversationIdFromPage(page);
|
|
|
|
const failGenerationStart = async (route: Route) => {
|
|
const request = route.request();
|
|
const { pathname } = new URL(request.url());
|
|
const isAgentsChat =
|
|
pathname === '/api/agents/chat' || pathname.startsWith('/api/agents/chat/');
|
|
if (
|
|
request.method() !== 'POST' ||
|
|
!isAgentsChat ||
|
|
pathname.endsWith('/abort') ||
|
|
!request.postData()?.includes(failedPrompt)
|
|
) {
|
|
await route.continue();
|
|
return;
|
|
}
|
|
|
|
await route.fulfill({
|
|
status: 500,
|
|
contentType: 'application/json',
|
|
body: JSON.stringify({ message: failedText }),
|
|
});
|
|
};
|
|
await page.route('**/api/agents/chat**', failGenerationStart);
|
|
|
|
const failure = await submitMessageExpectingGenerationFailure(page, failedPrompt, 500);
|
|
expect(failure.ok()).toBe(false);
|
|
await expect(messagesView(page).getByText(failedText)).toBeVisible({ timeout: 30000 });
|
|
await expect(page.getByRole('textbox', { name: 'Message input' })).toBeEnabled({
|
|
timeout: 30000,
|
|
});
|
|
await page.unroute('**/api/agents/chat**', failGenerationStart);
|
|
|
|
await sendAndExpectReply(page, afterFailurePrompt, afterFailureReply);
|
|
const messages = await waitForMessages(
|
|
page,
|
|
conversationId,
|
|
(items) => items.some((message) => messageText(message).includes(afterFailureReply)),
|
|
'follow-up after generation-start failure',
|
|
);
|
|
expectNoFoldedMessages(messages);
|
|
expectParent(messages, afterFailurePrompt, baseReply, true);
|
|
expectParent(messages, afterFailureReply, afterFailurePrompt, false);
|
|
expect(messages.some((message) => messageText(message).includes(failedPrompt))).toBe(false);
|
|
expect(messages.some((message) => messageText(message).includes(failedText))).toBe(false);
|
|
|
|
await reloadAndExpectMessages(page, [baseReply, afterFailureReply]);
|
|
await expect(messagesView(page).getByText(failedText)).toBeHidden();
|
|
await revisitConversationAndExpectMessages(page, conversationId, [
|
|
baseReply,
|
|
afterFailureReply,
|
|
]);
|
|
await expect(messagesView(page).getByText(failedText)).toBeHidden();
|
|
});
|
|
});
|