LibreChat/scripts/quality/README.md
Atef Bellaaj 715bf1f9e9
🧪 feat: Changed-Line Mutation Selection and Single Quality Report Comment (#18111)
Select mutants from changed lines and ignore logging-only mutants; rank survivors by risk topic from whole-word identifier matches.
Publish one quality report comment, edited in place, with the verdict, change risk (coverage gaps and CRAP refactor hints) and mutation survivors grouped by line.
Bundle the report data into one quality-report artifact; shard artifacts are internal.
Map api/ source aliases into coverage evidence and enforce a per-shard execution budget.
Move Redis cluster runtime files out of the checkout.
2026-11-16 08:00:48 -05:00

448 lines
31 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Pull request quality reports
PR commands and manual runs report change risk for changed lines and added or edited
functions in `packages/data-schemas`, `packages/api`, and the backend `api` folder.
Mutation testing is experimental. It mutates changed code in `packages/data-schemas` only. Support for more scopes is planned.
Only workspaces with changed source get report jobs. PR updates do not start source
reports. PostgreSQL contract and readiness tests still run automatically for changes
to their schema and lifecycle paths. Report jobs also provide PostgreSQL and provision
the MongoDB test binary before tests start. Changes to the quality tooling still run its own verification tests automatically.
Low scores do not fail a check. Failed tests, invalid reports, and tool errors do. A mutant with
a `RuntimeError` or `CompileError` result does not fail the run: it is shown in the report as
invalid and excluded from the scores.
Mutation jobs have a 240-minute limit. Change-risk jobs have a 45-minute limit.
A workflow timeout is a failed run, not a quality score. A Stryker `Timeout` result
means a mutated program timed out; Stryker counts that mutation as detected.
1. Mutation testing uses Stryker 10.1.1. It mutates only changed executable lines, not whole
functions. A changed simple statement (an expression statement, a declaration, a `return` or
`throw`, or the header or test of a compound statement) mutates in full, including every line
of a changed multi-line statement. A block, function body, or class member mutates in full
only when every executable line it spans changed; an edit inside a nested function never pulls
in its unchanged parent, and an edit in the parent never pulls in an unchanged nested function.
A deletion-only edit mutates its enclosing simple statement only when the deletion point sits
strictly inside it; a deletion between statements selects nothing, and is still recorded.
A renamed file with no other edit selects nothing; a new file selects every line. Blank lines
and comment-only lines never count as changed. Standalone, side-effect-free
`logger.<error|warn|info|debug|verbose|silly>(...)` calls are not mutated, including the
mutation that empties a block made up only of such calls; any other call, including one with
a side-effecting argument, mutates normally. A changed file carrying a `// Stryker disable`
comment with no reason or naming `all` is listed in the selection summary as advisory; it
does not fail the run.
It selects related Jest tests and uses per-test coverage to reduce repeated work.
Static mutations are disabled by default. These run outside an individual test,
such as during module loading or a shared test hook. With `static=on`, Stryker
reruns the related tests for each mutated file. `static=off` sets Stryker's
`ignoreStatic: true`. Static-only mutants remain in the report as `Ignored`.
They do not affect the score. Mutants covered both during loading and inside
tests still run with per-test coverage. The exact-plan check includes ignored
results; it never treats a missing result as an ignored mutant. Each shard plan
records the static option. Shards with different options cannot be combined.
Four GitHub jobs run separate mutation ranges, with two Stryker workers per job.
The planner counts mutants before testing and assigns the largest ranges first
to the shard with the fewest mutants. Each original range stays intact: splitting
a block containing nested functions could remove its block-replacement mutation.
A large indivisible range can therefore make one shard slower than the others.
Sharding reduces waiting time but repeats setup and baseline tests on each runner.
It does not split or exclude the tests available to a mutation.
Each shard artifact includes an HTML report and a JSON report with mutation locations.
A shard summary counts the changed functions that its own ranges touch, and the files that
hold them. Files whose mutated ranges are all outside functions are counted separately.
A final job combines the JSON reports and calculates one score from mutant counts. It runs
only when the shard jobs succeeded or failed; when tooling verification fails or the run is
cancelled, the shards do not run and the final job is skipped.
It checks the head, plan, source, and every expected mutation. Missing shards, duplicate
mutations, or incomplete results prevent a combined score. A shard with runtime or compile
errors still combines normally: those mutants are reported as invalid and excluded from
the score, the same as a shard with no such errors.
The live Actions log shows progress every 30 seconds. It separates static mutations
from other mutations and lists each unfinished function. A completion line shows
the result counts for each finished function. Elapsed time and time since the last
result help identify slow work. Pending means unfinished; it does not identify
which mutation a worker is currently running.
`progress.jsonl` saves each result as it arrives. `progress.json` contains the latest
progress snapshot and marks whether the report is complete. Failed runs upload
partial output when the artifact step can still run. Runner loss or a hard timeout
can prevent that upload.
2. Change risk selects whole functions; this selection is separate from mutation's changed-line
selection. The selector compares each function's code with the base version. Comments and
spacing do not count as code changes.
An edit that only deletes code still selects the containing function.
Editing a nested function does not select its unchanged parent or siblings.
An edited parent also excludes unchanged nested function bodies from mutation.
Moving unchanged code within a file does not select it. File renames count as additions.
Deleted functions have no score in the new source. Changed module-level code outside
functions gets no complexity or CRAP row, but its changed executable lines still appear in
the change-risk test-gap section. Mutation testing mutates changed ranges outside functions
as well; the shard summary counts those files separately.
The report includes all function expressions, arrow functions, callbacks, and methods.
3. CRAP combines classic ESLint cyclomatic complexity with statement coverage:
`CC² × (1 − coverage)³ + CC`. Nested functions own their statements.
Cognitive complexity comes from the published `sonarjs/cognitive-complexity` rule.
It is a separate score and does not replace CC in the CRAP formula.
Reports compare the PR head with its merge base. They do not measure the synthetic
GitHub merge commit. Application source and test dependencies come from the PR head.
Quality tooling and the combined-report job use the workflow revision, so older PR
heads do not need to contain the tooling scripts. Jobs that execute PR code have read-only permissions and need no secrets. Separate
jobs accept commands and publish comments and the merge status. They use trusted
workflow code and do not install PR dependencies. The comment publisher downloads only the
small findings file and validates it before use; it never opens the full reports.
Reports appear in the Actions job summary and in the `quality-report` artifact (see
[Report comment and artifact](#report-comment-and-artifact)).
The selector knows these source owners: `api`, `client`, `packages/api`, `packages/data-provider`,
`packages/data-schemas`, `packages/client`, and `packages/chat`. Each uses its own Jest
configuration. Source under the package `src` directories and legacy `api` is eligible.
Tests, fixtures, declarations, generated output, and compatibility harnesses under `misc`
are excluded. Source outside those owners is listed in the selection summary. Build and
CI scripts have separate test commands; they do not have an application Jest owner.
Coverage comes from related tests in the owning workspace, not from end-to-end tests
or every consumer of a built package. No coverage evidence produces `N/A`, never a
zero-risk score. A zero-hit instrumented function is untested. Scores are sorted with
the highest CRAP first. This report style is inspired by
[crapper](https://github.com/unclebob/crapper); its metric definitions are not substituted
for the repository's existing CRAP calculation.
From the repository root, after `npm ci` and `npm run build:packages`:
```sh
node scripts/quality/changes.mts --base origin/dev
node scripts/quality/run.mts --base origin/dev --workspace data-schemas --check risk
node scripts/quality/run.mts --base origin/dev --workspace data-schemas --check mutation
# Run one of the four CI shards (indices 1 through 4).
node scripts/quality/run.mts --base origin/dev --workspace data-schemas --check mutation --shard 1
node --test scripts/crap.test.mts scripts/quality/*.test.mts
```
The default command scope is `all`: change risk for `data-schemas`, `api` (`packages/api`),
and `server` (the backend `api` folder). A mixed change can run three risk jobs and
four data-schemas mutation shards. API-only or backend-only changes run no mutation jobs.
Manual `workflow_dispatch` runs can choose `risk`, `mutation`, or `both`, use `all`
or a comma-separated scope list, and set static mutants on or off. Mutation testing
stays limited to data-schemas in all cases. The selection summary
lists risk workspaces, mutation workspaces, and workspaces outside the scope.
Related Jest tests supply coverage for the changed files. Coverage uses each
workspace's existing worker count and memory recycling. API Redis integration
suites run separately with Redis enabled, using the same groups and serial lanes
as cache CI. The API report runner provides both the single Redis instance and the
repository's three-node Redis cluster. Istanbul merges the groups' coverage before
function scores are calculated. All groups use the same Jest coverage directory so
they can reuse compiled code. After each group, its raw reports move into its own
directory, including when tests fail. Every group still runs its tests with the
workspace's existing timeouts. Tests can exercise unchanged code, but only added
or edited functions appear in the risk report.
`packages/api` changes also get evidence from `api/`'s own specs through a source alias, bounded
by the repository variable `QUALITY_ALIAS_BUDGET_MINUTES` (an integer from 1 to 25 minutes,
default 20; an invalid value fails the run). A pass that exceeds the budget, crashes, or produces
no results file leaves native coverage in place and marks only the affected files incomplete,
without failing the job.
The pass compiles `packages/api/src` to loose CommonJS (plain `exports.x = x` assignments, as
in the built bundle), so specs that spy on or partly mock `@librechat/api` behave as they do
against the bundle.
In normal CI, a spec that mocks or imports a `packages/api/src` file by relative path gets a
module separate from the bundle. The pass keeps that separation: its resolver loads such
relative requests from `api/` through a symlink to `packages/api/src` at
`coverage/source-alias/src`, and the mirror's own relative imports stay in the mirror. A spec's
`jest.mock` of a source file therefore does not replace that file inside
`jest.requireActual('@librechat/api')`. The pass also collects coverage from the mirror path;
Istanbul records it under the real file, so both copies add to one entry. The runner removes the
symlink after the pass. Related-test selection (`--listTests`) does not use the mirror.
Coverage from several runs merges only when each run's entry has the same statement-map hash.
A file with nothing to instrument, such as a type-only module, has an empty entry with no hash
in every run; those entries merge and the file is measured with no executable lines. A file
that is incomplete or unavailable shows that state instead of a percentage in the refactor
hints table, and its CRAP is `N/A`. The test-gap section lists such files after the measured
ones, once for each state, reason, and evidence scope; a group of more than 5 files is
collapsed. Each uncovered changed-line range in the test-gap section
is a fenced code block of up to 8 lines, with each line cut at 160 characters.
## PR commands and merge decision
A user with repository write access, or a trusted bot account matched by exact login and ID
in `comment.mts`, can post one of these commands as a new PR conversation comment. Edited
comments do not start or change a request.
The bot adds an eyes reaction to valid commands from authorized users. This includes
ignore and duplicate requests. If GitHub cannot add the reaction, processing continues.
```text
/quality risk scopes=data-schemas,api,server
/quality mutation scopes=data-schemas static=off
/quality check-all
/quality check-all scopes=data-schemas static=on
/quality ignore
```
`scopes` defaults to `all`. `static` defaults to `off` and only affects mutation
runs. `all` means the supported scopes for the chosen check. Risk supports
`data-schemas`, `api` (`packages/api`), and `server` (the backend `api` folder).
Mutation supports only `data-schemas`. A `check-all` request runs both checks. It uses the selected scopes
for risk and includes mutation only when `data-schemas` is selected.
`/quality both` remains an alias for `/quality check-all`. The manual workflow and
local runner also default to static mutants off.
Unknown scopes, duplicate options, and invalid values are rejected before work starts.
One accepted request can run per PR. Repeating the same effective command for the
same commit while it runs returns its existing link. Different valid options cancel
the previous request before new report jobs start. A finished request can be repeated.
Validation and the repository write-access check happen before cancellation.
Invalid commands and unauthorized users leave the current run and gate unchanged.
Cancellation only targets accepted comment requests from this repository and PR.
Other PRs, unrelated workflows, tooling verification, and standalone manual runs
are not cancelled by a request.
The bot records the exact PR head and base commits when it accepts a request.
One bot comment, edited in place, holds the request and its report. It updates after selection
and tooling verification, then again when the workflow ends. An ignore request shows only the
verdict, commit, requester, and check. The comment layout is in
[Report comment and artifact](#report-comment-and-artifact). Job rows show change risk, then
mutation shards 1 to 4, then the mutation report, each in scope order (`data-schemas`, `api`,
`server`). The comment footer states which outcomes close the gate. Mutation logs provide the
detailed 30-second progress updates.
A failed or cancelled workflow is shown as such. A new PR commit cancels old-commit work and marks earlier
request comments outdated. The current request can be rerun with its original
commit and options. Reruns of superseded requests or older commits are stopped.
Post a new command to measure the current commit or use different options.
`Quality decision` is a commit status. It waits for an explicit decision on each
new PR head; it does not keep a runner idle. The latest accepted request for that
commit owns the status:
4. A selected run clears the gate only after the workflow succeeds. A successful
empty selection also clears it. Low scores remain advisory.
5. The gate stays closed unless the workflow, source selection, tooling verification, and
every report job succeed. A report job fails on a failed test in its workspace's own Jest
run, a tool error, a job timeout, or an incomplete mutation report, including a mutation
shard that reaches its execution budget. A cancelled workflow also keeps the gate closed.
A change-risk file marked incomplete or unavailable, for example by a failed `api/`
source-alias suite, does not close the gate. Retry the run or make a new explicit decision.
6. `/quality ignore` cancels active quality work, then clears the gate and records
who chose to skip the checks. An older run finishing cannot replace it.
7. A new accepted run makes the gate wait again. A duplicate active command does
not change it. A new commit requires a new request or a new ignore decision.
Manual workflow dispatches do not count as explicit comment decisions.
Request admission and status publication share one queued concurrency group per PR. Each reads the latest
head and decisions before it writes. This prevents an older result from reopening
the gate after a newer request has made it pending.
## Report comment and artifact
The `Quality report` job runs after the report jobs, also when one of them failed. It downloads
the intermediate `quality-internal-*` artifacts of the run, reads them as data only, and uploads
one `quality-report` artifact (retention 14 days):
```text
quality-report/
├── findings.json # quality-findings/v2: every section, validated and byte-capped
├── report.md # the full report: the comment sections with nothing cut
├── risk/<workspace>.json # run.mts risk.json for each change-risk workspace
├── risk/<workspace>.md # the change-risk job summary
└── mutation/data-schemas.{json,html,md} # merged Stryker JSON, HTML viewer, mutation report
```
Agents and scripts read `findings.json` first, then `report.md`, and open `risk/` or `mutation/`
for full detail. The intermediate artifacts (`quality-internal-risk-*`,
`quality-internal-mutation-*`, `quality-internal-findings`) are kept for 1 day and are not
linked from the comment. A rerun of failed jobs more than one day later cannot reuse them.
`findings.json` has one section per selected workspace. A missing or invalid workspace report
becomes an `unavailable` section; the bundle does not fail for it. When a change-risk job fails
because its own Jest run (any coverage group) reported failing tests, `run.mts` still writes
`failure.json` beside its other output: the base and head, the number of failed tests, the number
of suites that failed to run, and the first 10 failures (repository-relative suite path and test
name, at most 200 characters). The job still fails, so the gate stays closed. The upload step runs
with `if: always()`, and the bundle turns the record into a `failed` section whose `failure` field
holds these values. A tool error without failing tests writes no record and stays `unavailable`. Risk sections hold the summary
counts, the files with changed lines that no test ran (with code excerpts), and the changed
functions with complexity, coverage, and CRAP. The mutation section holds the status counts, the
`Ignored` reasons, both scores, the survivors, the invalid mutants, and the no-coverage ranges.
A survivor holds its location, mutator, original and replacement code, rank, and tags. It also
holds `source`, the whole source lines from `line` to `endLine`, and `mutated`, the same lines
with the replacement applied. The merge job takes both from the source text in the Stryker
report. Each has at most 12 lines, each line cut at 160 characters; a longer span ends with
`… N more lines`.
The writer cuts lists from their low-priority tail until the file is at most 1 MB and records the
number cut in each section's `omitted` field.
The comment shows, in this order:
1. The verdict: `Gate: open`, `Gate: pending`, or `Gate: closed` with the jobs that block it.
The status line and request facts follow.
2. **Change risk**: a one-line takeaway, one summary row per workspace, then the failing tests
of a `failed` workspace (folded, suite path and test name; the row reads
`failed: N tests failed`), the incomplete or unavailable files (folded), the changed lines with no test (one folded block per file, with
links to the tested commit), and the refactor hints, highest CRAP first. The takeaway gives
the total of changed lines with no test and the changed function with the highest CRAP (name,
link, CRAP, coverage). When a workspace report is unavailable or failed, or a file has no reliable
coverage, it says the data is incomplete instead; with nothing to report it is left out.
The repository variable `QUALITY_REFACTOR_HINTS` sets how many hints show in the table: an
integer from 0 to 100, default 10. The folded list after it holds only the functions with a
CRAP of at least `QUALITY_CRAP_THRESHOLD`: an integer from 0 to 10,000, default 30. A last
line counts the functions below the threshold (or cut for size), which are in `report.md`.
For either variable, any other value logs a warning and uses the default. `report.md` folds
every remaining function. The CRAP formula follows the table.
3. **Mutation testing**: a one-line takeaway, one summary row with the status counts and scores,
then the survivors grouped by file and line. The takeaway gives the survivor total and the
file with the most survivors; a partial or unavailable run, or survivors omitted from the
findings file, makes it say the data is incomplete, and with no survivors it is left out.
Each file is folded. Each line has a link (`Line N` or `Lines N-M`), its risk tags, and its
survivor count, then one fenced `diff` block: the source lines the mutant spans, prefixed
`-`, then the same lines with the mutation applied, prefixed `+`. When a line has more than
one survivor, a `@@ <mutator> @@` line comes before each one, and a survivor that spans other
lines repeats its own `-` lines. The comment shows the 3 highest-ranked survivors of a line,
then `N more survivors on this line in report.md`; `report.md` shows all of them. Lines follow
the survivor rank. The
no-coverage ranges and the invalid mutants are folded. An invalid mutant shows the first
line of its error, with the runner checkout and Stryker sandbox prefixes removed.
4. The job table (folded when the run is complete), an **All data** link to the
`quality-report` artifact, and the run page link.
A comment holds at most 65,536 characters. The fixed parts are never cut. The lists share the
remaining space: a list that fits an equal share shows in full and gives the rest to the other
lists; the others are cut by rank and end with `N more <items> in report.md`.
The publisher in `quality-decision.yml` never checks out or runs PR code. It reads report data
only when all of these are true:
- The run is complete and the `Quality report` job's `Upload report findings` step succeeded.
The **All data** link needs its `Upload quality report` step to succeed. A same-named
artifact from a job that runs PR code therefore cannot stand in for either.
- The `quality-internal-findings` artifact contains exactly one regular `findings.json` file of
at most 1 MB.
- The file passes strict validation: exact keys, types, integer ranges, enum values, safe
relative paths, and text length caps.
- Its repository, PR, base, and head match the bot's request record, and the PR head has not
moved since the run.
If a check fails, the comment says why and shows no report sections. All artifact text is
treated as untrusted. Code, paths, and reasons appear only in code spans or fenced code blocks
with a fence longer than any backtick run in the text, so they cannot add Markdown, HTML, or
mentions. Text in `<summary>` is HTML-escaped. Links are built from the server URL, the
repository, the tested commit, and a percent-encoded path. Scores and risk levels never change
the `Quality decision` outcome.
## Mutation report
Each data-schemas mutation run produces a step summary and a mutation report
(`mutation/data-schemas.md` in the `quality-report` artifact), built from the four shards'
merged JSON report. Both hold the same sections, in this order:
1. **Test gaps on changed lines** — survivors first. A survivor is a mutation a passing test
suite did not detect; the report calls it "not detected by tests", never a bug. Each row
lists the location, mutator, original and mutated code, and risk tags.
2. **No coverage** — changed lines no test executed, grouped per file into contiguous line
ranges.
3. **Invalid mutants** — `CompileError`/`RuntimeError` mutants, listed individually: location,
mutator, original and mutated code, and the reason Stryker gave (`statusReason`, truncated
and escaped the same way as the other code cells). Stryker could not run these to a result
at all; they are excluded from the scores and are never `Survived` or `NoCoverage`.
4. **Status** — one row per result, kept distinct: `Killed`, `Timeout`, `Survived`,
`NoCoverage`, `Ignored` (broken down into `static`, `logging-only`, or the exact
`Stryker disable` reason text), and `Invalid` (`CompileError`/`RuntimeError`). `Timeout`
is never folded into `Killed`: a mutated program that hangs is a distinct outcome from one
a test actively caught.
5. **Scores** — two numbers, each labeled with its formula:
- `Killed by assertion`: `Killed / (Killed + Survived + NoCoverage)`. Excludes `Timeout`
from both sides; only a test that actively failed counts here.
- `Stryker mutation score (counts timeouts as detected)`:
`(Killed + Timeout) / (Killed + Timeout + Survived + NoCoverage)`. Matches Stryker's own
definition, where a timeout counts as a detected mutant.
Both are `N/A (no valid mutants)` when their denominator is zero. Either way, `Invalid`
mutants are excluded from both the numerator and the denominator.
6. **Suppression advisory** — changed `Stryker disable` comments with no reason or naming
`all`, carried from the plan's advisory list. Informational; it does not affect the score.
7. **Source of truth** — a pointer to the merged JSON report (`mutation.json`) and its HTML
report (`mutation.html`, rendered with `mutation-testing-elements`, the same viewer
Stryker's own HTML reporter uses), which hold every mutation location.
Ranking (the `rank` field, lowest first) groups survivors by what they put at risk, not by
how new the surrounding code is. Keyword sets assign one or more tags: `authorization`,
`tenant`, `credentials`, `data-loss`, or the `behavior` fallback.
- Keywords match whole words. The mutated code, its replacement, the enclosing function name,
and the file path are split into words at camelCase and PascalCase boundaries, acronyms,
digits, `_`, `-`, `.`, and `/`, then compared in lower case. `authorizeProviderSession`,
`userRole`, and `refresh_token` match; `authorName` and `dropdown` do not. Simple plural and
verb forms match (`tenants`, `deleted`, `removing`). `author` and a bare `access` are not
keywords; `accessToken` matches through `token`.
- A tag found in the code or the function name is a code match. A tag found only in the file
path, such as `src/auth/`, is a path-only match and adds 5 to that tag's rank.
- Emptying an object literal (`{}`) is always a code match for `data-loss`.
- The highest-priority match sets the leading part of the rank (the minimum over all tags):
`authorization` 0, `tenant` 10, `credentials` 20, `data-loss` 30, `behavior` 40, plus 5 for a
path-only match. A path-only `authorization` match (5) therefore ranks after a code
`authorization` match (0) and before a code `tenant` match (10).
- The mutator adds 0 for an `ObjectLiteral` replaced by `{}`, 1 for a conditional, equality,
logical, or boolean mutator, and 2 for every other mutator, including `StringLiteral`.
- Ties sort by file, then line. Being inside a newly added function is not an input.
`findings.json` carries only `Survived` findings with this rank and their tags; the full
mutation report carries every status.
### Execution budget
The repository variable `QUALITY_MUTATION_BUDGET_MINUTES` caps the wall-clock time one
mutation shard's Stryker run may take. It must be an integer from 1 to 230 minutes; an invalid value fails the run with a clear error instead of falling
back to a default, because a silently wrong budget could hide a shard that is actually
running far longer than intended. Unset defaults to 60. Stryker's JS API has no run
abort, so each shard runs as a supervised child process and is signalled (`SIGTERM`, then
`SIGKILL` after a grace period) if it is still running when the budget elapses.
A shard that hits its budget does not fail outright. It writes a `completion.json` with
`status: 'partial'`, the mutant results already finished (from `progress.jsonl`, flushed as
each result arrives), and the exact ids of the mutants that never ran. The merge step accepts
a partial shard only when its finished and unfinished mutants together exactly cover that
shard's planned set, with no mutant missing, duplicated, or unexpected; any other mismatch is
rejected the same way a missing or malformed complete shard is. A merge that includes a
partial shard sets the mutation section's `status` to `'partial'`, adds an "Execution budget
reached" section to the report naming the unfinished mutants, and exits non-zero with the
distinct message `incomplete: execution budget reached` — separate from a tooling failure, so
a dashboard or future automation can tell "ran out of time" apart from "broke". The mutation
artifact still uploads, because the upload step runs with `if: always()`, and the
`Quality report` job still bundles it.
### Incremental reuse is not enabled
Stryker supports an incremental mode that reuses a prior run's per-mutant results from a
cache file keyed by source hash and configuration, skipping mutants whose covering code and
tests are unchanged. This pipeline does not use it. A comment-
triggered run on a PR branch can be requested again after a force-push, a rebase, or a change
to the merge base; a cache populated by an earlier, different commit on the same branch name
or workflow cache key could then report a mutant as already killed when the code it covers
has since changed, silently hiding a real regression. The four shards also run concurrently
against what would need to be one shared cache, and GitHub Actions cache writes are not
safe for concurrent writers. Reusing results safely would need a cache keyed and invalidated
per exact head commit, which removes most of the reuse incremental mode offers in a
comment-triggered workflow where most requests are for a commit that has never run before.
Each run stays a full run.
## Activation
The command and completion workflows must reach the repository default branch
(`main`) before GitHub can run their `issue_comment` and `workflow_run` triggers.
Merging this PR into `dev` alone does not activate those triggers. Follow the normal
`dev` to `main` promotion; do not backport the workflows separately.
After live command tests pass, an administrator can add `Quality decision` as a
required status in the target branch's protection rule or ruleset. Select GitHub
Actions as its expected source. The status alone does not block merging until that
rule exists. The activation work remains part of PR #18025. Source reports now
require an explicit command or manual dispatch. Tooling verification remains
automatic for changes to the quality scripts, workflows, and dependencies.
Live checks must cover a successful selected run, `static=off`, an invalid command,
a requester without write access, `/quality ignore`, cancellation, a failed test,
a newer commit during a run, and a rerun of an older commit. Confirm that comments
update in place and that only the current commit's latest decision clears the gate.
Also confirm that duplicate requests reuse their run and that replacements never
cancel another PR, unrelated CI, or standalone manual runs.
Rule sources:
8. [Stryker configuration](https://stryker-mutator.io/docs/stryker-js/configuration/)
9. [Stryker Jest runner](https://stryker-mutator.io/docs/stryker-js/jest-runner/)
10. [SonarJS](https://github.com/SonarSource/SonarJS)