QA
/flow-next:qa runs a live-app, real-user QA pass. It drives the running app like an unforgiving customer, derives its test scenarios directly from the spec, files structured findings with evidence, and ends with a YES/NO ship verdict.
It is the one review surface that isn’t static. impl-review, spec-completion-review, quality-auditor, and code-review all read code or specs; /flow-next:qa exercises the deployed app. It is forbidden from marking PASS by reading source - a scenario passes only on captured evidence (screenshot / console / URL), never on inspection.
The doctrine, the spec-as-intent advantage, and the driver ladder are in Live-app QA & driving UIs.
Arguments
Section titled “Arguments”/flow-next:qa <spec-id> [--target <url>] [--base <ref>] [--receipt <path>] [mode:autonomous]QA runs against a spec, never a single task. Each flag takes its value as --flag value or --flag=value.
| Argument | What it does |
|---|---|
<spec-id> | The spec whose acceptance criteria the scenarios come from. |
--target <url> | The URL to test. Without it, QA uses QA_TARGET_URL from the environment, then a deploy URL in the spec, then one in the README, .env.example or deploy config, or a documented dev-server URL and start command. It never assumes localhost; with nothing found it asks, or under mode:autonomous ends BLOCKED. |
--base <ref> | The base the branch is compared against. Default: the first of origin/main, main, origin/master, master that resolves, then the remote’s default branch. |
--receipt <path> | Writes the qa_verdict receipt here instead of REVIEW_RECEIPT_PATH or the default .flow/review-receipts/qa-<spec-id>.json. |
mode:autonomous | Asks nothing; FLOW_AUTONOMOUS=1 turns it on too, and flow --auto dispatches with it. A fact the attended run would ask for (target, base, test account) comes from the spec, config or environment, and when it cannot, the run writes a BLOCKED receipt and exits cleanly. An unknown spec id is the one hard error. |
How it drives the app
Section titled “How it drives the app”QA never re-implements driving - it consumes Flow-Next Drive’s surface-aware driver ladder (agent-browser → chrome-devtools-mcp → Playwright → cursor-ide-browser → manual; native surfaces go Cua Driver → Computer Use, or Cua Sandbox headless). Whatever rung Flow-Next Drive resolves for the surface is what QA inherits.
Findings
Section titled “Findings”Failures are filed immediately as structured P0 / P1 / P2 reports - persona, steps to reproduce, expected vs actual, and evidence (console, screenshots, full URL). Findings feed the bug memory track (track: bug) with overlap dedup, and can be promoted to flow specs or tasks for the fix.
The verdict
Section titled “The verdict”The pass ends with a verdict carried as a proof-of-work receipt (type: qa_verdict), with four outcomes:
qa_outcome | Meaning | Ship? |
|---|---|---|
SHIP | Every scenario that ran passes, zero open P0/P1, every user-observable R-ID live, subtracted or carried | Yes |
NEEDS_WORK | Any open P0/P1, or incomplete coverage | No |
BLOCKED | No live deploy or no driver - could not verify | No |
NA | Spec has no driveable user-visible AC | n/a |
The receipt’s verdict field projects onto the existing review-receipt enum (BLOCKED → NEEDS_WORK, NA → SHIP), so the verdict can feed Spec Completion Review - “does the live app satisfy the AC, not just the code?”
Re-checking after a fix
Section titled “Re-checking after a fix”When the spec already has a QA receipt from an earlier head on the same branch, QA re-checks only what could have changed. A fix for a QA finding, a small review fix or a rebase no longer costs a full live run. QA looks for the prior receipt before it plans scenarios, then diffs the trees between that head and the current one, so a rebase’s base changes count too. It drives:
- every scenario that failed or was blocked last time;
- every scenario whose surface the diff touches, counting the code it drives through and the shared code it depends on;
- one load-bearing scenario that exercises the whole app, as a smoke run, driven first.
Every other scenario that passed carries its result, cited with the head and evidence it was captured at. A carried scenario is never recorded as live at the new head.
QA runs in full, as on a first pass, when the prior receipt belongs to another spec or branch, its head no longer exists, or it lists no per-scenario outcomes. It also runs in full when the diff touches shared code broadly (styles, layout, state stores, routing, auth, a component most pages use, dependencies or build config), when the acceptance criteria changed since the prior pass, when the smoke run fails, or when it cannot tell what a surface depends on.
R3 apply coupon live @ d4e5f6 PASS (screenshot qa/r3-coupon.png)R1 add to cart carried @ a1b2c3 PASS (screenshot qa/r1-cart.png)smoke: full purchase live @ d4e5f6 PASS (screenshot qa/smoke.png)The new receipt is written at the current head, so the freshness gate treats it like any other. Its scenarios rows mark each scenario live or carried, and a criterion whose scenarios all carried reads carried (@<sha8>) in the coverage table. flowctl qa receipt copies a carried scenario’s outcome, R-IDs, evidence and original head from the prior receipt, and refuses a carry of a scenario that did not pass there or came from another spec or branch. See Receipts.
Measured in October 2026 on a small storefront where a full QA pass takes about two and a half minutes, the re-check caught the planted regression in every run, including one planted on a page outside the fix’s diff. It was 10 to 25% faster on average across the runs with a local fix, which never cleared the study’s pre-registered bar of a reliable 20%. When the regression sat outside the diff, every re-check run drove the broken page again instead of carrying it, and that run was no faster than a full pass.
Requirements
Section titled “Requirements”QA needs a live deploy + a driver (Flow-Next Drive). With neither, it surfaces a BLOCKED verdict rather than failing; a spec with no driveable UI yields a clean NA. Opt-in - it adds nothing to the base flow when unused.
As a pipeline stage
Section titled “As a pipeline stage”pipeline.qa is a string enum off | on | auto; any other value is off.
| Value | What runs |
|---|---|
off (default) | QA runs only when you invoke /flow-next:qa <spec>. |
on | One live pass at all-tasks-done, before make-pr, on every spec, under attended flow and flow --auto alike. |
auto | /flow-next:flow runs the pass when the spec’s acceptance describes UI behaviour on a drivable surface and a target can be started (a documented start command, a deploy URL, or .flow/features/). Otherwise the stage records skipped(config: pipeline.qa=auto: <reason>) and the route advances. flow --auto reads the same gate-selection reference, so the skip line and the advance to make-pr are identical unattended. |
Enable it with flowctl config set pipeline.qa auto (or on). Setup asks the question once on an unset key, and separately recommends /flow-next:features whatever the answer, since the live pass reads .flow/features/ to navigate the app. QA reads .flow/features/ itself to find the feature before driving. QA never hard-blocks a loop. NEEDS_WORK and BLOCKED advance to make-pr, and their findings become open items on a draft PR.
Credit
Section titled “Credit”The QA discipline - the P0/P1/P2 taxonomy, evidence rules, and session-hygiene practices - is a lean borrow from Ray Fernando’s running-bug-review-board (Apache-2.0). Thank you, Ray.
Worked example
Section titled “Worked example”/flow-next:qa fn-12-export-json-flagDeriving scenarios from spec: R1-R3 -> 4 scenarios (2 runtime, 1 CLI contract, 1 error path)Driving live app via flow-next-drive... R1 export --json emits valid JSON PASS (captured stdout, parsed) R2 null-owner rows serialize FAIL - P1: row dropped (captured output attached)qa_outcome: NEEDS_WORK (1 P1) - findings filed; PR still advances with a Live QA section.Receipt: qa_verdict (head_sha e4f5a6b, rid_coverage 3/3, open_p0p1: 1)The verdict rests on captured evidence from the running app - the skill is forbidden from passing by reading source.
Dynamic usage
Section titled “Dynamic usage”Recipes that compose with qa in the cookbook:
- Evidence-first - the
qa_verdictreceipt is idempotent per branch head; your scripts can key on it. - Autonomy dial - set
pipeline.qatoonandflow --autoruns the live pass before every PR;autolets attended flow andflow --autodecide per spec from the gate-selection reference.
Next step
Section titled “Next step”/flow-next:make-pr <spec-id>