Flow-Next Drive

The flow-next-drive skill drives any UI surface the way a real user would — a web app, a Chromium-backed desktop app (Electron / Windows WebView2), or a genuinely native app (macOS AppKit/SwiftUI, or a webview exposing no CDP), driven via the Cua Driver (MIT, provider-agnostic, background) / Computer Use, with a Cua Sandbox rung for headless/CI native runs. It detects the surface, picks the highest available driver on a ladder, and degrades gracefully when a richer driver is absent.

It is a router, not a single driver. The default rung — Vercel’s agent-browser CLI — is the only driver assumed present; every other rung is detected and optional. A pass succeeds with whatever the environment actually has: most cloud VMs, Linux, and CI have no Computer Use, so it is never a hard dependency and never on a headless/no-display path.

When to use it

Verifying a deployed UI change matches the spec.
Driving or testing a web app, an Electron / WebView2 desktop app, or a native desktop app.
Reading documentation that has no clean text version.
Capturing baseline screenshots before a redesign.
Logging into a service and pulling structured data.
Light e2e probes that do not warrant a full test framework.

It orchestrates drivers — it does not reimplement them. The full native-desktop QA workflow (scenario authoring, bug filing, verdict) is a downstream /flow-next:qa concern; this skill provides the driver/actuation + the surface conditional.

Step 1 — Detect the surface, then branch

Surface	What it is	Path
Web app	A URL in a browser (localhost dev server, staging, production)	Web ladder
Chromium-backed desktop app	Electron / Windows WebView2 — Chromium under the hood, exposes a CDP debug port	Web ladder, attaching over CDP to the app’s remote-debugging port
True-native / non-CDP surface	macOS AppKit/SwiftUI, Catalyst, or a webview exposing no CDP (macOS WKWebView, which Tauri uses on macOS)	Native rung → Cua Driver / Computer Use / Cua Sandbox

Per-platform caveat: Windows WebView2 is CDP-drivable (web ladder); macOS WKWebView generally is not (native rung). When unsure whether a desktop app exposes CDP, probe for the web ladder first (try to launch/attach with a debug port); if no port is reachable, fall to the native rung.

Step 2 — The universal flow (all surfaces)

Whatever driver the environment has, the work is the same shape:

observe / navigate to the target
snapshot   → fresh element refs (REQUIRED before each act)
act        → click / fill / type / press / scroll
verify     → confirm the expected text / state appeared
capture    → screenshot + console/errors (and on failure)
release    → close the tab / end the session when fully done

Refs (@e1, @e2, …) go stale after any navigation, click, or form submit — always re-snapshot. “ref not found” or “pointer-events: none” almost always means a stale snapshot, not a real bug.

Step 3 — Web ladder (web apps + Chromium-backed desktop apps)

Probe availability top-down and use the highest rung that passes; fail soft to the next; the terminal rung is manual.

Rung	Driver	Use when
1 (default)	agent-browser CLI	Always assumed present. CDP-based, headless-safe, no extra install. Drives web apps; drives Electron / WebView2 over CDP (`--cdp <port>` / `--auto-connect`).
2	chrome-devtools-mcp	You want built-in auto-wait, DevTools-grade network/console inspection, Lighthouse, or to attach to your real signed-in Chrome (`--browser-url`) so bot defenses don’t challenge an automated profile.
3	Playwright	The repo already has Playwright configured, or you need a headless CI-style / cross-browser regression run.
4	cursor-ide-browser MCP	Running inside Cursor with this MCP installed and you want its snapshot YAML + `browser_cdp` control.
5 (terminal)	Manual + screenshot relay	No browser driver available — drive yourself, paste console errors and screenshots into chat.

The same ladder drives Electron / WebView2 apps by attaching to the app’s remote-debugging port. Launch the app with a dedicated debug port and user-data-dir; treat the open debug port as a security exposure (any local app can drive that session).

Step 4 — Native rung: Cua Driver → Computer Use → Cua Sandbox

A genuinely native app (or a non-CDP webview) has no browser tab to attach to — the only way to drive it is to operate the live machine. This rung is provider-agnostic: probe for the best available driver, prefer the highest that passes, degrade to the next. None of these is ever a hard dependency.

Probe	Driver	What it is
`cua-driver` MCP registered / `command -v cua-driver` (real display)	Cua Driver	trycua/cua (MIT) — provider-agnostic, background (no focus steal), macOS / Windows (Linux is pre-release/experimental — Wayland-only limits; fall to the sandbox or documented-limitation there), accessibility-tree-based (drives structured `element_index` elements, not pixels). Preferred when present.
Codex CU available, or a Claude Computer-Use harness present	Computer Use	Codex Computer Use (macOS / Windows) / Anthropic Claude Computer Use (the API `computer` tool via its own harness). Screen-takeover.
Headless / CI (no display) + a sandbox backend (`lume` / Docker / QEMU, or opted-in cloud)	Cua Sandbox	Drives inside an isolated VM/container — the only native option with no real screen. Opt-in per run, torn down each run; local backend default, `cua.ai` cloud explicit opt-in.

The actuation differs from the web ladder, but the universal flow (Step 2) is identical — observe → snapshot → act → verify → capture, described as goal + success state, not pixel coordinates.

Attended vs headless splits the precedence. On a real display, prefer the background Cua Driver → Computer Use (screen-takeover) → documented-limitation. On a headless / CI host (no screen), the Cua Sandbox is the only native option — local Computer Use and the local Cua Driver both need a real display. Web/Chromium surfaces (Electron / WebView2) are unaffected — they stay on the web ladder by CDP-attach even in CI, never on this rung.

Determining headless / CI is empirical, not env-sniffing. $CI ⇒ headless; otherwise ask the driver — cua-driver call get_screen_size reports a real display (positive width and height). Two guards keep it honest: a missing driver is unknown, not headless (don’t route an attended machine without cua-driver to the sandbox), and TCC grants are not a display signal (missing Accessibility → fall to Computer Use; missing Screen Recording → AX-only evidence). $DISPLAY is unset on a fully-displayed Mac, so it is a Linux-only hint.

Cua Driver — install + wiring (detect-and-instruct, never auto-installed)

Flow-Next detects and instructs; it never runs these for you — the same no-auto-install consent rule /flow-next:map applies to clawpatch. The base install stays zero-dependency; agent-browser remains the only assumed-present driver.

1. Install the driver (upstream installers):

# macOS / Linux
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"

# Windows (PowerShell)
irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex

These are upstream’s official installers, piped from the mutable main branch — on sensitive or CI machines, review the script first (drop the pipe and inspect) or pin a release tag instead of main.

2. Register the MCP server with your host (host-specific):

# Claude Code CLI
claude mcp add --transport stdio cua-driver -- cua-driver mcp

# Codex — add to ~/.codex/config.toml
[mcp_servers.cua-driver]
command = "cua-driver"
args = ["mcp"]

Any other MCP host: register a stdio MCP server whose command is cua-driver mcp.

3. Grant macOS permissions (one-time) — the load-bearing step. macOS TCC grants Accessibility and Screen Recording separately, and they unlock different things: Accessibility unlocks driving (launch / click / type / read the live AX tree); Screen Recording unlocks screenshots. Request both:

cua-driver permissions status   # report which grants are present
cua-driver permissions grant    # walk you through both grants

The grant is attributed to com.trycua.driver. The daemon must restart to pick up a newly-granted permission. (Linux / Windows have no TCC split — skip this step there.)

4. (Optional) Run the daemon so the MCP tools share one driver process:

cua-driver serve   # macOS / Linux, and Windows from an interactive desktop session

Without serve, tools fall back to an in-process driver with a warning. On Windows from SSH or a non-interactive shell, a bare cua-driver serve lands in Session 0 (the GUI tools return empty arrays — installed but undriveable); use cua-driver autostart enable then cua-driver autostart kick to run the daemon in the user’s interactive session instead. For sandbox / cloud operations and the full API, see upstream libs/cua-driver/README.md.

The macOS permission split — AX-only evidence

Because Accessibility and Screen Recording grant independently, the driver can drive fully while screenshots stay empty. When Screen Recording is absent, the rung still drives and captures the accessibility tree as structured live-state evidence, and surfaces “Screen Recording not granted ⇒ AX-only evidence, no screenshot” — it never presents a missing screenshot as a captured one. /flow-next:qa (downstream) decides whether AX-only evidence meets its bar.

Cua Sandbox — the headless/CI surface

The Cua Sandbox SDK drives an app inside a disposable VM/container (any OS) — the one surface the local driver and Computer Use can’t reach, because both need a real display. It’s opt-in per run and torn down each run (no leaked VMs). Two backends, very different profiles:

Local (lume / QEMU / Docker) — a VM/container on your own hardware. The default sandbox backend — for a fully offline run, set CUA_TELEMETRY_ENABLED=false (the cua SDK ships anonymous usage telemetry on by default).
Cloud (cua.ai) — a managed VM on Cua’s infrastructure. Explicit opt-in only: it bills and the driven screen + data leave the machine (data egress); requires a CUA_API_KEY. Never auto-selected.

The SDK installs with pip install cua (Python 3.12+ — Flow-Next itself targets 3.8+, so use a dedicated venv: python3.12 -m venv .cua && .cua/bin/pip install cua). For the persistent (non-ephemeral) form, delete explicitly with Sandbox.delete(name, local=True) — the default local=False targets the cloud namespace, so omitting it after a local create leaves the VM running. The first lume pull of a macOS image downloads a multi-GB disk image (~30 GB) — a one-time “coffee break,” not a hang.

Licensing

The default driving path — the background cua-driver MCP — uses only MIT components and needs none of the vision/OmniParser stack. The optional cua-agent[omni] pulls ultralytics (AGPL-3.0) and OmniParser is CC-BY-4.0; the rung documents which extras are MIT-safe vs AGPL/CC-BY and never auto-installs any of them.

Graceful degradation

The native rung never blocks — it falls through, never failing silently:

Cua Driver not installed / MCP not registered → print the install + wiring commands (Flow-Next can’t install for you), fall to Computer Use → documented-limitation.
macOS, Accessibility not granted → only no-grant reads work; guide you through cua-driver permissions grant + a daemon restart, fall to Computer Use meanwhile.
macOS, Screen Recording not granted → drives fully, AX-only evidence (surfaced explicitly, never an empty screenshot).
Headless / CI, no local sandbox backend and no opted-in cloud → no native driver reachable (local driver + Computer Use both need a display); document the limitation and print the sandbox install. Never auto-select the billing cloud.
Chromium-backed app (Electron / WebView2) → not this rung — drives via the web-ladder CDP attach, even headless/CI. (Shell-level integration — system tray, native menus, OS dialogs — can’t be reached over CDP; surface that limitation.)
Genuinely native app with no native driver at all → document the limitation rather than fail.

agent-browser stays the only assumed-present driver. No MCP server, Cua Driver, or Computer Use is ever a hard install dependency, and flowctl never imports any of them.

What this is not

Not a test framework. There is no assertion DSL, no parallel runner, no flake retry.
Not a scraper for restricted sites. Respect robots.txt and the target’s terms of service.
It orchestrates drivers — it does not reimplement Playwright or Computer Use.

Common pitfalls

Forgetting to re-snapshot after a click or form submission — refs become stale (the Cua Driver’s element_index values go stale the same way after any UI mutation; re-read get_window_state before each act).
Routing an Electron / WebView2 app to the native rung — it’s Chromium, drive it over CDP via the web ladder (even though Cua’s launch_app exposes the debug ports).
Treating any native driver as a default — most environments lack one; the Cua Driver / Computer Use / Cua Sandbox is the native-rung fallback, not the common case.
Reaching for the local Cua Driver in CI — it needs a real display; the headless/CI native surface is the Cua Sandbox rung.
Treating a present CUA_API_KEY as consent to use the billing cloud — the cloud sandbox is opt-in per run; the zero-network local backend is the default.

Dynamic usage

Recipes that compose with drive in the cookbook:

Evidence-first - drive is how QA captures live-app evidence instead of narrating it.
Integration tricks - the surface-detection ladder works for your own automation prompts, not just QA.

Driver ladder and universal-flow structure inspired by Ray Fernando’s running-bug-review-board skill (Apache-2.0).