Agentic UI Testing

Validating a user interface by having an agent operate it like a user — reading the page, clicking, typing, and judging pass/fail against a plain-language user story — instead of (or alongside) scripted deterministic tests (Jest/Vitest/Playwright specs). The unit of work is a user story: a small file with a name, a URL, and a natural-language workflow ("Navigate to the front page, verify at least 10 posts load, click 'More', verify page 2 loads"). A QA sub-agent parses the story into steps, drives a browser via the CLI Tools over MCP Servers Playwright CLI, takes a screenshot at every step (00_<step>.png, 01_<step>.png… under a per-run directory <story-kebab>_<8-char-uuid>/), evaluates PASS/FAIL, and on failure captures the JS console and stops. A slash command (/ui-review) fans stories out across parallel agents and merges their reports (Agentic Workflow Patterns — parallelization).

Why do it agentically rather than with a deterministic framework? The pitch: no test-config sea — you write a URL and a story, not selectors and fixtures; agents "act like a user would," so you can spin up arbitrary new stories at a drop of a dime as the UI changes. The screenshot trail doubles as evidence — a walkable record of exactly what the agent saw and did, giving "a trail of success and a trail of failure" when a workflow breaks. The cost is the usual Workflows vs Agents trade: this is a non-deterministic check (flakier, token- costly per run) traded for near-zero authoring cost and human-like coverage — best leaned on for fast-moving UIs, not as the sole gate on a stable critical path.

Claims

Test a UI by having an agent operate it as a user against a natural-language user story, rather than only with scripted deterministic tests. best practice — context: fast-changing UIs where authoring/maintaining selector-based tests is the bottleneck; you trade determinism for near-zero authoring cost, so it complements rather than replaces a deterministic suite on stable critical paths.
Express the test as a user story (name + URL + plain-language workflow), not as test config. best practice — context: agentic UI testing specifically; the low authoring cost — "at a drop of a dime" new stories — is the whole advantage, contingent on an agent reliable enough to translate prose steps into actions.
Have the agent screenshot every step and, on failure, capture the console and stop. best practice — context: agentic browser runs where you need auditable evidence; the per-step screenshot trail is what makes a non-deterministic run debuggable and gives a success/failure record.
Agentic UI tests are non-deterministic; the choice is a deliberate determinism-vs-coverage trade. observation — the operator frames it as the standing balance between an agentic and a deterministic solution, leaning "a little extra agentic."
Run user stories in parallel across agents (via an orchestration command) for speed and independent reports. best practice — context: multiple independent stories; parallelism is affordable because the CLI keeps per-agent token cost low (CLI Tools over MCP Servers).
The same screenshot-and-judge move works as inline self-correction during generation, not only as a separate test pass — an agent building HTML slides screenshots every draft and fixes its own mistakes before moving on. observation — a second, independent source (the Front End Slides skill) shows the loop pointed at the agent's own rendered output mid-build; the agent is both author and QA. Corroborates the "agent reads its rendered output and judges it" mechanism from the generation side rather than a downstream /ui-review.

CLI Tools over MCP Servers — the agents drive a browser through the Playwright CLI; token efficiency is what makes parallel browser agents practical.
Layered Agentic Architecture — agentic UI testing is the concrete workflow the four-layer stack is demonstrated on.
Agentic Workflow Patterns — /ui-review is parallelization (independent stories fanned out and merged); the orchestrator-workers layer over the QA sub-agent.
Workflows vs Agents — the non-deterministic (agent) vs. deterministic (scripted test) trade is the control-locus distinction made concrete for testing.
Agent Supervision — the screenshot trail is the human→agent oversight surface: you review what the agent did, not just its pass/fail verdict.
HTML as Native Output — why the self-correction loop is available here at all: the output is a rendered web page the agent can screenshot and re-read, so native HTML output and visual self-validation compound.
Distillate: My 4-Layer Claude Code + Playwright CLI Skill (Agentic Browser Automation & UI Testing)
Check Gaming — why operate-as-a-user testing matters: a DOM-presence check is gameable (hidden text passes it), but an agent driving the page as a screen-reader user catches the gamed pass.
Distillate: Stop Making PowerPoints: Vibe-Coding HTML Slides as a Skill — the agent screenshots each slide draft and self-corrects during the build (inline, not a separate pass).
Distillate: Claude Fable 5 Bossed 20 Cheap AI Agents. The Whole Site Cost $8. — accessibility validated by operating the site in a real browser (both themes, every route) and by testing "as Maya," a blind VoiceOver/braille-display reader persona whose needs outranked the visual design.

Agentic UI Testing

Claims

Related

Linked from