LibreChat/e2e/specs/real/usage.spec.ts
Danny Avila 530a935a74
🎨 feat: Color the Context Gauge by Category and Collapse its Breakdown (#14855)
* 🎨 feat: Color the Context Gauge by Category and Collapse its Breakdown

The context window bar becomes a stacked meter — one hue per category — and
the breakdown collapses behind a disclosure so the gauge alone is the default
view. The collapse choice persists per user.

Adds a categorical series scale (`rgb-series-1`…`rgb-series-7`) to the
versioned theme registry, so themes and `REACT_APP_THEME_SERIES_*` can retint
it. Hues are anchored on LibreChat's own brand tokens; every step was computed
rather than picked, by enumerating slot orderings and snapping each step until
all gates passed in both modes:

  worst adjacent CVD ΔE          12.4 light / 13.0 dark  (target 8)
  worst adjacent normal-vision   19.0 light / 19.0 dark  (floor 15)
  contrast                       all 14 steps ≥ 3:1 on both the popover
                                 surface and the meter track

Slot order is the colour-vision-deficiency safety mechanism, not cosmetics.
Reserved status colors are never reused for series identity, and the circular
composer gauge is deliberately untouched — it answers "how close am I to the
limit", which stays a status question.

- `SegmentedMeter` + `MeterSwatch` land beside `Progress` in `@librechat/client`,
  owning the 2px surface gaps, rounded ends, the min-width floor, and the hatch.
  The category-to-slot mapping stays feature-local: the palette is theme data,
  the mapping is not.
- Every present category gets a 2px floor so a 251-token row cannot render as
  0.09px; the shortfall comes out of free space, never another category.
- Deferred tools keep their family's hue and add a 135° hatch, so a hue never
  means two things. Segments are reordered to put each deferred pair beside its
  parent, which is also the adjacency the palette was validated on.
- Messages is drawn as a translucent fill with a solid edge: it is the only
  category the user grows, and the form difference doubles as secondary encoding.
- A row carries a swatch if and only if it is a segment. The estimate path knows
  the total but not the composition, so it keeps a single unsegmented fill.
- Usage totals gain a "Totals" heading, and row text lifts to primary ink on
  hover/focus.
- The popover widens 256px → 288px to absorb the chevron and the legend swatches.

Guardrails: the series scale is held to the 3:1 mark floor on both surfaces, the
app CSS defaults are held in step with the runtime themes, and each slot is
asserted to resolve to a Tailwind utility backed by its CSS variable.

* 🐛 fix: Address Codex Review on the Segmented Context Gauge

Three P2 findings, all confirmed.

**Gaps inflated the fill.** Segment widths were percentages of the whole track
while `gap-[2px]` was added on top, so the gaps ate into the free-space
remainder instead of living inside the filled region. Measured on the real
component: a window at 47.2% painted 55.6% full, and the bar read full at ~94%.
Each segment now surrenders its share of the gap budget, so fills plus gaps span
exactly the used fraction. Same case now paints 50.2%.

The residual 3.0pp is the `SEGMENT_MIN` floor doing its job — five sub-pixel
categories rounded up to 2px each. That overshoot is deliberate and bounded, it
comes out of free space rather than a neighbouring category, and the doc comment
now states the magnitude instead of leaving it implicit.

**No reference-theme test.** The suite only exercised the bundled token tables,
so it could not detect the shared component becoming coupled to LibreChat's
values. Adds a deliberately different reference `ThemeDefinition` and asserts the
registry accepts it, the values reach the applied CSS variables, and every
rendered mark takes its colour from those variables — no literal colours in the
tree. `SegmentedMeter.tsx` also joins the shared-primitive colour guardrail.

**Series tokens missing from the public maps.** `IThemeVariables` and
`IThemeColors` are exported for downstream consumers to type their CSS-variable
and Tailwind maps, and would have rejected the new keys. Adds the series entries
to both, plus a compile-time guard in the registry so a slot added to one map and
missed in another fails the build.

The guard deliberately lives in `registry.ts`, not the spec: `tsconfig.json`
excludes `*.spec.ts`, so an assertion there is never checked by the build —
verified by removing a key from each map in turn and confirming the error.

* ✅ fix: Expand the Breakdown in the Context Gauge e2e Specs

`e2e/specs/mock/usage.spec.ts` asserts on rows that now sit behind the
disclosure, so four tests failed on the collapsed default. My miss — I updated
the component spec and never grepped for e2e coverage.

`openBreakdown` now expands the detail after opening, so every caller that
reads a row keeps working; the helper is idempotent, since a reload restores an
already-expanded preference. The one inline `gauge.click()` that duplicated the
helper now uses it.

Adds the case the regression should have been caught by, and which only e2e can
reach: the popover opens to the gauge alone with no detail mounted, expanding
reveals the labelled Totals section, and the choice survives a real reload
through localStorage without a second click.

`e2e/specs/real/usage.spec.ts` reads the totals the same way. It also hovered
rather than clicked, which never opened the popover at all — hover surfaces only
the compact snapshot tooltip, as the mock spec asserts.
2026-08-14 22:46:10 -04:00

130 lines
5.5 KiB
TypeScript

import { expect, test } from '@playwright/test';
import type { Page, TestInfo } from '@playwright/test';
import { sendMessage, selectMockEndpoint } from '../mock/helpers';
/**
* LOCAL-ONLY real-provider verification: drives multiple turns against real
* Anthropic and asserts the context gauge tracks genuine provider usage.
* Runs only via e2e/playwright.config.real.ts (requires ANTHROPIC_API_KEY).
*/
const REAL_MODEL = process.env.E2E_REAL_ANTHROPIC_MODEL ?? 'claude-haiku-4-5';
const gauge = (page: Page) => page.getByTestId('token-usage');
const gaugeMeter = (page: Page) => gauge(page).getByRole('meter');
const popover = (page: Page) => page.getByRole('region', { name: 'Context usage' });
async function gaugeValue(page: Page): Promise<number> {
const raw = await gaugeMeter(page).getAttribute('aria-valuenow');
const value = Number(raw);
expect(Number.isFinite(value)).toBe(true);
return value;
}
/** Parses formatTokens output ("950", "1.2K", "2M") back to a number. */
function parseTokens(text: string): number {
const match = text.trim().match(/^([\d.]+)([KM])?/i);
if (!match) {
return NaN;
}
const suffix = match[2]?.toUpperCase();
let scale = 1;
if (suffix === 'M') {
scale = 1e6;
} else if (suffix === 'K') {
scale = 1e3;
}
return parseFloat(match[1]) * scale;
}
async function readUsageTotals(page: Page): Promise<{ input: number; output: number }> {
/** Click, not hover: hover only surfaces the compact snapshot tooltip. The
* popover then opens to the gauge alone, with the totals behind a remembered
* disclosure — idempotent, since the preference survives the previous turn. */
await gauge(page).click();
await expect(popover(page)).toBeVisible({ timeout: 15000 });
const toggle = popover(page).getByTestId('context-breakdown-toggle');
if ((await toggle.getAttribute('aria-expanded')) === 'false') {
await toggle.click();
}
const section = popover(page).getByTestId('token-usage-totals');
await expect(section).toBeVisible({ timeout: 15000 });
const rows = section.locator('div');
const input = parseTokens(
(await rows.filter({ hasText: 'Input' }).first().locator('span').last().innerText()) ?? '',
);
const output = parseTokens(
(await rows.filter({ hasText: 'Output' }).first().locator('span').last().innerText()) ?? '',
);
await page.keyboard.press('Escape');
return { input, output };
}
async function sendTurn(page: Page, text: string) {
const response = await sendMessage(page, text);
expect(response.ok()).toBeTruthy();
/** Settle: streaming over once the stop button yields back to send */
await expect(page.getByRole('button', { name: 'Stop generating' })).toBeHidden({
timeout: 90000,
});
await expect(page.getByTestId('send-button')).toBeVisible({ timeout: 30000 });
}
test.describe('context usage gauge with a real provider', () => {
test('grows across turns and reflects real usage', async ({ page }, testInfo: TestInfo) => {
test.setTimeout(300000);
await page.goto('/c/new', { timeout: 15000 });
await selectMockEndpoint(page, { label: 'Anthropic', model: REAL_MODEL } as never);
/** Turn 1 */
await sendTurn(page, 'Reply with exactly one word: alpha');
await expect(gauge(page)).toBeVisible({ timeout: 20000 });
const v1 = await gaugeValue(page);
expect(v1).toBeGreaterThan(0);
const totals1 = await readUsageTotals(page);
expect(totals1.input).toBeGreaterThan(0);
expect(totals1.output).toBeGreaterThan(0);
await gauge(page).hover();
await expect(popover(page).getByText('Messages')).toBeVisible();
await expect(popover(page).getByText('Free space')).toBeVisible();
await expect(popover(page).getByText('Session cost')).toBeVisible();
await expect(popover(page).getByText(/\$\d|<\$0\.01/)).toBeVisible();
await page.screenshot({ path: testInfo.outputPath('turn1-popover.png') });
await page.keyboard.press('Escape');
await expect(page).toHaveURL(/\/c\/(?!new)/, { timeout: 15000 });
/** Turn 2: context must include turn 1 — gauge and totals grow */
await sendTurn(page, 'Reply with exactly one word: beta');
const v2 = await gaugeValue(page);
expect(v2).toBeGreaterThan(v1);
const totals2 = await readUsageTotals(page);
expect(totals2.input).toBeGreaterThan(totals1.input);
expect(totals2.output).toBeGreaterThan(totals1.output);
/** Turn 3 */
await sendTurn(page, 'Reply with exactly one word: gamma');
const v3 = await gaugeValue(page);
expect(v3).toBeGreaterThan(v2);
await gauge(page).hover();
await expect(popover(page)).toBeVisible({ timeout: 15000 });
await page.screenshot({ path: testInfo.outputPath('turn3-popover.png') });
await page.keyboard.press('Escape');
/** Reload: gauge rebuilds from persisted per-message token counts once
* the messages query resolves — retry until the meter leaves zero */
await page.reload({ timeout: 20000 });
await expect(gauge(page)).toBeVisible({ timeout: 20000 });
await expect(gaugeMeter(page)).toHaveAttribute('aria-valuenow', /[1-9]/, { timeout: 20000 });
const vReload = await gaugeValue(page);
expect(vReload).toBeGreaterThan(0);
await gauge(page).hover();
await expect(popover(page)).toBeVisible({ timeout: 15000 });
await page.screenshot({ path: testInfo.outputPath('reload-popover.png') });
testInfo.annotations.push({
type: 'observed',
description: `v1=${v1} v2=${v2} v3=${v3} reload=${vReload} input1=${totals1.input} input2=${totals2.input}`,
});
});
});