
atr-skills
Skills: browser control + recording (atr-browser), cross-platform desktop control with in-process agent + multi-monitor support (atr-computer: mouse, keyboard, screen, windows, apps, 'computer ask' for natural-language tasks), run tests with AI analysis (atr-analyze), behavior testing (atr-behavior). MCP Server: 50+ tools (30 browser + 22 desktop) via 'atr mcp serve'
- version
- 1.9.0
- GitHub stars
- 13
Install
/plugin install atr-skills@atr-marketplaceRun it inside Claude Code after adding the marketplace with /plugin marketplace add imyousuf/agentic-test-runner.
Contents
Discovered from sourceSkills 5
ATR Command Analysis Skill
This skill runs commands through ATR (Agentic Test Runner) which provides AI-powered analysis and clean summarized output. Use this as your default way to run tests, linters, type checkers, and builds - the AI analyzes the full output and returns a concise summary, keeping your conversation context clean and focused.
When to Use This Skill vs Direct Commands
Use ATR analyze (RECOMMENDED DEFAULT) when:
- Running test suites (pytest, jest, go test, npm test, make test, cargo test)
- Running linters (eslint, golangci-lint, pylint, flake8, make lint, cargo clippy)
- Running type checkers (mypy, tsc, pyright, make typecheck)
- Running builds that produce verbose output (make build, npm run build, cargo build)
- You want clean, summarized results instead of raw output
- You want automatic failure analysis if something goes wrong
Use direct Bash commands only when:
- Running quick one-off commands with minimal output
- You specifically need the raw, unprocessed output
- Interactive commands that require user input
Basic Usage
atr run --cmd "<command>"
Examples:
# Tests
atr run --cmd "go test ./..."
atr run --cmd "npm test"
atr run --cmd "pytest tests/"
atr run --cmd "make test"
# Linting
atr run --cmd "make lint"
atr run --cmd "eslint src/"
atr run --cmd "golangci-lint run ./..."
# Type checking
atr run --cmd "make typecheck"
atr run --cmd "tsc --noEmit"
atr run --cmd "mypy src/"
# Builds
atr run --cmd "make build"
atr run --cmd "npm run build"
Adding Context
Provide context to help the AI agent focus its analysis:
atr run --cmd "<command>" --context "<context>"
Examples:
atr run --cmd "go test ./..." --context "Tests started failing after refactoring the auth module"
atr run --cmd "npm run build" --context "Added new dependency yesterday"
atr run --cmd "pytest" --context "Testing the new payment integration"
Command Options
| Flag | Description |
|---|---|
--cmd <command> | Command to execute (required) |
--cwd <path> | Working directory |
--context <text> | Additional context for AI agent |
--model flash|pro | Model tier (flash=fast, pro=deep analysis) |
--python-venv <path> | Python virtual environment path |
--nvm-version <version> | Node.js version via nvm |
--no-auto-env | Disable automatic environment detection |
Working Directory
Specify where to run the command:
atr run --cmd "npm test" --cwd "/path/to/project"
Environment Detection
ATR automatically detects and activates appropriate environments:
Python projects:
- Detects
.venv,venv, or Poetry environments - Auto-activates virtual environment
Node.js projects:
- Detects
.nvmrcorpackage.jsonengine requirements - Auto-activates correct Node.js version via nvm
Override automatic detection:
atr run --cmd "pytest" --python-venv /custom/path/.venv
atr run --cmd "npm test" --nvm-version 18
atr run --cmd "make" --no-auto-env
Model Selection
Use different models for different needs:
# Quick analysis (default)
atr run --cmd "make build" --model flash
# Deep analysis for complex issues
atr run --cmd "go test ./..." --model pro
What the AI Agent Does
The ATR agent processes command output and:
- Summarizes results into a clean, concise report
- Identifies test pass/fail status and key metrics
- Analyzes any failures for error patterns and root causes
- Reads relevant source files when failures occur
- Provides actionable recommendations for fixing issues
This keeps your conversation context clean by replacing verbose test output with a focused summary.
Example Output
Executing: go test ./...
Directory: /path/to/project
--- FAIL: TestUserAuth (0.05s)
auth_test.go:42: expected 200, got 401
Command failed (exit code: 1)
Analyzing failure with AI agent...
======================================================================
ANALYSIS RESULTS
======================================================================
Status: FAILURE
Summary:
TestUserAuth fails because the auth middleware expects a JWT token,
but the test doesn't provide one in the request headers.
Root Cause:
Line 38 in auth_test.go creates a request without Authorization header.
The auth middleware (middleware/auth.go:15) rejects it with 401.
Recommendations:
1. Add mock JWT token to test request
2. Or bypass auth middleware in test setup
3. Check if middleware was recently added to the route
Files Examined:
- auth_test.go
- middleware/auth.go
- routes/api.go
Exit Codes
| Code | Meaning |
|---|---|
| 0 | Command passed |
| 1 | The thing under test is broken (analysis provided) |
| 2 | The run could not decide — missing input, stale compiled script, browser or model unavailable |
Configuration
Configure ATR in ~/.atr/config.yaml:
backend: gemini-api # or vertex-ai
model: flash # or pro
gemini:
api_key: "your-key"
# Or for Vertex AI:
vertex:
project: your-project
location: us-central1
Environment variables:
export GEMINI_API_KEY="your-key"
# Or
export GOOGLE_CLOUD_PROJECT="project-id"
Best Practices
- Use as default for running test suites to keep conversation context clean
- Provide context when the failure might be related to recent changes
- Use --model pro for complex, multi-file issues
- Specify --cwd when running from a different directory
- Check environment with
atr test-cmd-env "<command>"to preview detection
Writing ATR behaviour specs
A .test.txt spec is compiled once, by a model driving the real application,
into a JavaScript file that replays with no model in the loop. Running is
solved. Writing a spec that compiles, and that does not lie when it passes,
is the hard part — and it is what this skill is for.
The spec text reaches the compiler verbatim. ATR does not parse sections; it passes the whole file through. Everything you write in it is instruction to the model. That is the whole reason authoring guidance is a skill rather than a manual: the spec is the one input to the compiler that does not ship with the compiler.
Syntax lives in references/test-file-format.md. Start with
references/spec-template.test.txt. What follows is judgement.
Rule 1: never write an assertion that cannot fail
A false pass is the worst thing ATR can produce — worse than a crash, because nobody investigates a green test.
These are all assertions that cannot fail:
Expected Results:
- Steps completed successfully ← the steps succeeded by construction
- No console errors ← true on most broken pages
- The page loads ← it loaded before you clicked anything
Write what would be different if the feature were broken:
Expected Results:
- The order confirmation shows the order number
- The cart badge reads 0
- The "Place order" button is gone
If you cannot name something that would differ, you do not yet have a test.
Rule 2: assert on an element, not on page text
A substring match against the whole page passes for the wrong reason. One real
spec searched for "archiv" and matched three unrelated things on the page.
- Bad: "the page contains 'Archived'"
- Good: "the row's status badge reads 'Archived'"
Name the container. The compiler emits atr.text(selector) when you name one
and a whole-page match when you do not.
Rule 3: assert absence as well as presence
"Read-only" does not mean a notice appeared. It means zero editors and zero send buttons. A spec that only checks for the banner passes on a page that shows the banner and still lets you type.
Absence is only meaningful once the page has settled — wait for the state that removes the thing (the confirmation, the empty list, the new view) and then assert it is gone.
Rule 4: count before and after, never absolutely
"The last entry in the log is X" breaks the second time the test runs into the same log. "One more entry than before, and it reads X" survives.
Tests run repeatedly against shared state. Any assertion phrased as an absolute position or an absolute total is a time bomb.
Rule 5: a submit is request → navigate → repaint
Not one event. Say what state ends the step:
- Bad: "Click Save and wait 2 seconds"
- Good: "Click Save and wait for the confirmation toast"
Never ask for a fixed sleep. The compiler will wait for a state if you name one, and will guess if you do not.
Say it as one expectation, not a wait and then a check. The compiler emits
atr.expectText, which waits and asserts together; the two-call form hands the
diagnosis to whichever call hits the wall first — always the wait — so a page
that stops reaching the state is reported as a timeout rather than as the
feature being broken, and CI retries it instead of escalating.
Rule 6: a destructive test rebuilds its own fixture
A compile drives your spec more than once — once to learn the application, then again to verify what it wrote. A spec that deletes the only record it depends on works once and then fails for ever.
Put the setup in the spec and say it is setup:
Setup:
- Create a project named "atr-fixture-{run}" if one does not exist
The compiler turns that into atr.setup(...), which runs before the steps on
every execution and is not counted as a step. A failure there is reported as
the fixture failing, not the application misbehaving.
This is also the answer to "how do I test deletion": create the thing, delete the thing, assert it is gone.
Rule 7: read the DOM before naming a control
Labels hide in at least three places — placeholder, aria-label, and
<label for> — and two dialogs in the same application will disagree about
which. Use atr browser snapshot and quote what is actually there.
Rich-text editors are the recurring trap: they reject a plain fill and need an input-event recipe. The class is universal even though the recipe is app-specific, so put the recipe in Notes for the compiler (below).
Rule 8: a missing input is nobody's bug
config is the one failure kind the agent will not help with: not
repairable, not retryable, never sent to the model. A person has to decide
what the value should be.
That is deliberate. The tempting "fix" for a missing value is to inline the literal back into the script, which undoes the reason inputs live outside it.
Values live in <spec>.test.properties (committed) beneath
<spec>.test.override.properties (gitignored) and ATR_VALUE_* variables. A
committed properties file executes — $(command) and ${VAR} expand at
read time, on every machine including CI.
Rule 9: when sign-in cannot be scripted, seed the browser
Some applications cannot be logged into by a script — SSO, a hardware token, a one-time code.
atr browser start # start the daemon
# sign in by hand in the window that opens
atr run --behavior tests/ # reuses the running daemon's browser automatically
No flag is needed: atr run picks up a running daemon's CDP endpoint on its
own, and opens an isolated tab for the test.
Name the symptom of forgetting, because it is unrecognisable otherwise: every step times out on a sign-in page, which looks exactly like a hang.
For a credential that can be scripted, use a secret reference rather than a
value — the compiler emits atr.fillSecret, which fetches and types inside one
call so the value never enters the script, the transcript, or a later repair
prompt.
Rule 10: a step is one journey, not one instruction
How you break a spec into steps decides what can ever be shared between specs.
A hoisted operation replaces a run of consecutive statements inside a single step. So if you write:
1. Open the tags page
2. Follow the "rest" tag
the compiler puts a check at the end of step 1 — it arrived, after all — and
now the navigate and the click sit either side of an assertion and either
side of a step boundary. They can never become one shared openTag(), because
gathering them would mean moving the assertion, and an assertion never moves.
Write the journey as one step and the checks as the next:
1. Open the tags page and follow the "rest" tag
2. Confirm the reader is looking at that tag's posts
This is also just better spec writing: a step is a thing the reader does, and Expected Results is what must then be true. The sharing falls out of that.
Rule 11: share operations, never assertions
You do not have to write the shared library. Compile two specs that drive the same journey and ATR hoists it for you: it finds the repetition, names it, rewrites both scripts to call it, and replays them before keeping the change. Nothing is kept unless every rewritten script still claims exactly what it claimed before, character for character.
A compile is also shown what its neighbours wrote, so two specs reaching the same page use the same selector and the same constant name rather than inventing their own. That is what makes the repetition findable at all.
Do not name a library function in a spec. A step that reads "open the
front page using openFirstPost() from the shared library" does not fail when
no such operation exists — the compiler writes a local function of that name
and calls it, and the test passes. It reads as though the script shares code
when it does not: the next spec making the same journey writes it again, and if
a real library operation of that name ever appears, the local one silently wins
over it. The lint reports this as local-helper. Describe what the reader
does; deciding what deserves a name is ATR's job.
You can still write _shared.js by hand, and reading it is worth it either
way — it is loaded into the same scope before every script in the directory,
so its top-level functions are simply in scope:
// tests/e2e/_shared.js
function signIn(username) {
atr.navigate("/login");
atr.fill("#username", username);
atr.fillSecret("#password", {ref: "app_password"});
atr.click("#submit");
atr.waitFor("#dashboard"); // ends on the dashboard
}
The compiler is shown this file verbatim, so it calls signIn() instead of
rediscovering eight steps — which is the real saving. Duplication is the
obvious cost of not having it; compile time is the larger one, and every
rediscovery is a fresh chance to get it subtly different.
Three rules, and the first is enforced rather than requested:
- Operations only.
expectandatr.failrefuse to run from the library. Once a test's assertions live in shared code you can no longer read the test and know what it checks, and one edit can weaken every test in the directory. - Declarations only. No code at the top level: everything there runs before
step 1 of every spec beside it. No
atr.steporatr.setupeither — steps belong to a spec, and a library that declares them renumbers all of them. - Take inputs as parameters.
valuesis per-spec, so a library that readsvalues.get("username")needs every spec in the directory to define it.
A defect in the library is config: never repaired, never retried, never sent
to the model. A person fixes shared code.
atr refactor-ops tests/ --dry-run says what could be hoisted without
changing anything or opening a browser. Turn the automatic pass off with
behavior.extract_operations: on-demand or --no-extract.
Under --no-compile — which is what CI runs — hoisting only ever reports.
Applying it would call the model and rewrite the scripts, and that flag permits
neither, so a replay job never comes back with a modified working tree.
Editing the library does not force a recompile. Scripts carry a second
header, // atr-lib-sha256:, and a change to it means the script is unproven
against the new library, not wrong — so it replays and the header catches up.
Only a genuine signature break reaches the model, and only for the specs that
actually broke.
The honest trade-off: this is Page Object Model, and it carries POM's bargain. You keep "read the script, see exactly what it asserts". You lose "read the script, see exactly how it got there".
Notes for the compiler
The section nobody knows exists, and the one that decides whether a hard spec compiles. It goes at the end of the file and holds everything the model cannot learn by looking at one page:
- app-specific recipes (the rich-text editor; the date picker that ignores typing)
- which of several similar controls is the right one
- state the application carries between pages
- anything you discovered by watching a compile fail
Every note should be a trap that produced a wrong result. If you cannot say what would go wrong without it, leave it out — the notes are prompt, and padding costs iterations.
Do not restate what the compiler already knows. These are in the binary's
prompt and repeating them wastes tokens: use atr.setup for fixtures, wait for
state instead of sleeping, atr.expectExists/atr.expectMissing for presence,
selectors over page text, :has-text() for text-qualified targets.
The loop
-
Bootstrap.
atr browser record --output tests/thing.test.txtgives you the steps. It cannot give you the expectations — it leaves aTODOwhere they belong. Answer it; that is rule 1. -
Compile once.
atr run --behavior tests/thing.test.txt. It drives the real application, so it needs a base URL and takes minutes. -
Read the emitted
.test.jsbefore trusting it. This is the step nobody does. You are checking one thing: could this script fail? A step with no assertion is a step that passes whatever happens. -
Commit the script. It carries a hash of the spec, so a reviewer can see what changed and CI can replay it. A generated file whose behaviour depends on who ran it last is worse than no file.
-
--no-compilein CI. It replays or it fails loudly. Without it a spec edit silently turns a 9-second CI step into a multi-minute model run. -
--prune-valuesoccasionally. Repeated compiles accumulate aliases — one real spec ended up with seven keys for two inputs. -
Backport the compiler's corrections. When a compile discovers your spec was wrong — you wrote
button[type="submit"], the real control isbutton:not([type])— put the correction back in the spec. Nobody does this, and spec and script diverge silently from then on.The reason nobody does it: editing the spec changes its hash and forces a recompile. So batch backports and do them when a recompile is due anyway.
Before you commit
Read the generated .test.js against this list. It is short because it only
contains things that make a compiled test lie:
- Every step either asserts something or performs an action that throws
when it fails. A step of pure
atr.logcannot fail. - No
toContainwith a short needle againstatr.text()with no selector. - No
atr.sleepoutsideatr.retry— a fixed sleep is a race that has not happened yet. - Absence is asserted where the spec says something goes away.
- The spec still describes what the script does.
- The properties file has no key the script does not read, and no secret.
atr run runs the mechanical half of this list for you — a step that can only
succeed, a script with no assertion at all, a short substring matched against
the whole page, a fixed sleep — and refuses to accept a script that cannot
fail. --lint=warn reports without refusing while a suite adopts the check.
The judgement half is yours.
Failure kinds, and what each one asks of you
| kind | means | what you do |
|---|---|---|
assertion | the application is wrong | fix the app — never the assertion |
not_found | the UI moved | let the agent repair it, then check the diff |
timeout | slow, flaky, or hanging | retried automatically; if persistent, the spec is waiting for the wrong thing |
environment | browser or network | not about your test |
config | an input is missing | you must supply it; the agent will not |
script | the generated JavaScript is wrong | the compiler's fault; it repairs itself |
Only assertion turns a suite red (exit 1). Everything else exits 2, so a CI
job can retry rather than escalate.
A worked example
examples/behavior/blog/ is three specs against a live blog, with the inputs,
the shared library and the JavaScript the compiler produced from them. It is
the only example written against a site with no data-testid anywhere, whose
content changes when its author publishes — which is what most real
applications look like.
Worth reading for what the compiler did with the notes section, and for how the expectations are phrased so that publishing a new post does not break them.
Related
atr-behavior— running specs that already exist: flags, exit codes, triage.atr-browser— driving the browser by hand while you work out what the spec should say.references/test-file-format.md— the syntax.references/spec-template.test.txt— start here.
Running ATR behaviour tests
Runs browser tests written as natural-language .test.txt specs. A spec is
compiled once into a sibling .test.js and afterwards replays with no
model in the loop — seconds, and no tokens. The agent comes back only to
diagnose a failure.
Writing a spec is a different job. Load
atr-authorfor that: which assertions are worth writing, what belongs in the notes section, and how to avoid a test that passes without testing anything.
Basic usage
atr run --behavior tests/login.test.txt # compile if needed, then replay
atr run --behavior tests/e2e/ # every .test.txt in the directory
atr run --behavior tests/e2e/ --browser-url http://localhost:3000
Flags
| Flag | Description | Default |
|---|---|---|
--behavior <path> | Test file or directory | (required) |
--browser-url <url> | Base URL for the application | from config or base_url |
--headless | Run the browser headless | false |
--viewport <WxH> | Viewport size | 1920x1080 |
--cdp-endpoint <url> | Connect to an existing browser | - |
--no-compile | Replay only; never call the model. Fails loudly if the script is missing or stale | false |
--recompile | Regenerate the script even if it matches the spec | false |
--no-repair | Diagnose a drifted script but do not rewrite it | false |
--prune-values | Remove inputs the script no longer reads | false |
--lint <mode> | What to do about a script that cannot fail: error, warn, off | error |
--no-triage | Never ask the model why a failure happened, even to classify it | false |
--otel-endpoint <url> | OTLP collector for run telemetry, e.g. http://localhost:4318 | from config or env |
--interpret | Skip compilation; let the agent drive every step | false |
-v, --verbose | Show the script's own atr.log() output | false |
Cost: know which mode you are in
Use --no-compile whenever cost must be predictable — always in CI. It
replays, or it fails and says why. It never calls the model.
Without it, any of these silently turns a 9-second step into a multi-minute model run: the spec was edited, the script is missing, the script has never completed a run, or a failure invites a repair.
atr run --behavior tests/ --no-compile # CI: replay or fail
atr run --behavior tests/login.test.txt # local: compile if needed
--no-compile needs no backend at all — no API key, no project, no
gcloud auth application-default login. A CI job that only replays does not
have to hold credentials, and one that has them spends at most a single call,
only when a run has already failed, to say why (see below). (--interpret
is refused alongside it, since it is nothing but model calls.)
Each run prints how it was serviced (compiled →, repaired →) and how many
model calls it cost. A passing replay costing more than zero is worth
investigating; a failing one costs one call to classify the failure, which
is the point.
A script that cannot fail is refused
Before a script runs, ATR checks statically that it can fail: that the script
asserts something, and that no step is built only from reads. A step of pure
atr.log reports success whatever the application does, and a suite of those
reports that everything works right up until someone opens the application.
A blocking finding stops the run and exits 2 — nothing was learned about the application, so it is not a test failure. The findings never go to the model: a model asked to invent the missing assertions will invent ones that pass.
atr run --behavior tests/ --lint=warn # report but run; for adopting the check
atr run --behavior tests/ --lint=off # skip it
Looser problems — a short substring matched against the whole page, a fixed
atr.sleep outside a retry — are reported and not enforced.
Freshness
The compiled script is committed and carries a hash of the spec:
// atr-spec-sha256: 9f2c…
Edit the spec and the next run recompiles. Reflow whitespace and it does not —
the hash is over normalised text, so sha256sum on the raw file will never
match it. A script that has never completed a run carries
// atr-unverified and is recompiled rather than trusted. Deleting the hash
line marks a script hand-written and off-limits to the compiler: it is replayed
as it is, under --no-compile too, and never recompiled, repaired or hoisted.
A failure is still triaged — against what the script itself checks, not the
spec — and the reason reported; no rewrite is asked for or applied.
--recompile is the one way to replace it with a compiled script.
A compile drives the spec more than once — once to learn the application,
then again to verify what it wrote — so a destructive spec needs a rebuildable
fixture. That is what atr.setup(...) in the compiled script is for, and what
a Setup: section in the spec compiles to.
Test inputs
Beside the spec:
login.test.properties— committed, written by the compiler, editable.login.test.override.properties— gitignored, wins over it.ATR_VALUE_*environment variables — win over both.
base_url here is what lets one spec run against localhost and staging with no
flags. Values support $(command) and ${VAR}, expanded when read — so a
committed properties file executes, on every machine including CI.
--prune-values removes keys the script no longer reads. It reports without
removing unless you pass it, scans the shared library as well as the script,
and says nothing at all when either builds a key at run time, because then no
scan can be sure.
Shared operations
A _shared.js beside the specs is loaded into every script in that directory.
It holds operations — sign in, create the thing — never assertions:
expect and atr.fail are refused from it.
Editing it does not force a recompile. Scripts carry a second header,
// atr-lib-sha256:, and a mismatch means the script is unproven against the
current library rather than stale — so it replays and the header is updated
when it passes. Under --no-compile the update is reported instead of written,
so CI does not leave a dirty tree.
Hoisting follows the same rule. A run normally hoists the operations its specs
keep repeating into _shared.js; under --no-compile it reports what could be
hoisted and changes nothing, because applying it would call the model and
rewrite the scripts — neither of which that flag permits. Run
atr refactor-ops <dir> to act on it.
A defect in the library is a config failure: never repaired, never retried,
never sent to the model.
Exit codes
| code | meaning | what CI should do |
|---|---|---|
| 0 | every test passed | — |
| 1 | the application is broken — an assertion failed | escalate |
| 2 | nothing was learned about the application — missing input, browser or model failure, stale script under --no-compile | retry, or fix the environment |
1 means the app is wrong and nothing else. That separation is the point: a red build that can also mean "the CI box was slow" is a red build people learn to ignore.
Failure kinds
Printed with every failure, and each asks for a different response:
| kind | means | ATR's response |
|---|---|---|
assertion | the application did not do what the spec requires | stops immediately, no model call — never auto-repaired |
not_found | a target is gone; the UI moved | agent repairs the script, budget of 1 |
timeout | slow, flaky, or hanging | retried, then triaged |
environment | browser, daemon or network | retried, then reported as infrastructure |
config | an input this checkout does not define | reported; never repaired or retried — a person must supply it |
script | the generated JavaScript is wrong | repaired |
An assertion failure never reaches the model. That is deliberate: asking a model to confirm a regression spends the tokens compilation exists to save, and risks it "fixing" the assertion that caught the regression.
A regression can present as a timeout. If a script waits for a state and
then asserts it, the wait fails first — so a page that stops reaching that
state is reported as a timeout rather than as the application being wrong.
atr.expectText waits and asserts in one call and cannot be misattributed that
way; the lint reports the two-call form and names it.
For scripts already written the other way, triage covers it: a failure that survives its retries is classified, and a verdict of "the application is at fault" makes the run exit 1.
Triage is not compilation. --no-compile refuses to generate a script;
it no longer refuses to judge one. A replay that fails will spend one model
call to classify it — only on a run that has already gone red, and never to
rewrite anything, since CI asked for a replay. If no backend is configured it
says so and reports the runtime's own classification, so nothing new is
required of a CI job that had none. --no-triage restores the absolute
guarantee.
Debugging a failure
atr run --behavior tests/x.test.txt --headless=false # watch it
atr run --behavior tests/x.test.txt -v # show atr.log output
atr run --behavior tests/x.test.txt --no-repair # diagnose without rewriting
Then read the compiled .test.js — it is ordinary, readable JavaScript, and
the failure names the step and the target.
Signing in by hand
When authentication cannot be scripted (SSO, a hardware token, a one-time code):
atr browser start # start the daemon; a window opens
# sign in by hand
atr run --behavior tests/ # reuses the running daemon's browser automatically
No flag is needed — atr run picks up a running daemon's CDP endpoint on its
own and opens an isolated tab for the test, leaving your tabs alone.
Configuration
~/.atr/config.yaml:
behavior:
base_url: "http://localhost:3000"
browser:
executable: "auto"
headless: true
viewport: { width: 1920, height: 1080 }
page_timeout: "30s"
action_timeout: "10s"
What was run, over time
Every behaviour run is recorded locally in ~/.atr/history.db, including the
runs that never reached the application — a missing input, a browser that would
not start, a stale script under --no-compile. Those are the ones a pass rate
has to exclude to mean anything.
atr history # per spec: runs, pass/fail/infra, flakes, repairs
atr history --spec tests/login.test.txt --runs
atr history --since 7d --json
Two numbers only ATR can give you:
- Repair frequency. A spec repaired again and again is not flaky — the application's DOM is churning underneath it.
- True failure rate. Pass rate over the runs that actually tested the application. FLAKE counts runs that passed only after a retry, which are green in every other report.
The median REPLAY duration is worth watching: a replay is deterministic with no model in the loop, so a suite drifting from 9s to 15s means the application got slower.
The database is plain SQLite and the views (runs, attempts, compiles) are
a stable contract, so anything the command will not tell you is one query away:
sqlite3 ~/.atr/history.db "SELECT spec, outcome, count(*) FROM runs GROUP BY 1,2"
Point ATR at an OTLP collector and the same runs are exported as traces, metrics and logs — which is how a CI run's history survives the container being torn down. Three ways, in precedence order:
atr run --behavior tests/ --otel-endpoint http://localhost:4318 # this run only
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 # the standard variable
telemetry:
endpoint: "http://localhost:4318" # ~/.atr/config.yaml
Give the collector's base URL; the signal path (/v1/traces and friends) is
appended, and a URL that already names a signal is left alone. With no endpoint
anywhere, nothing is emitted and no error is logged.
Turn either off in ~/.atr/config.yaml; both may be off at once:
history:
enabled: true
keep_days: 90
telemetry:
enabled: true
Nothing ATR records lifts a resolved value into a field of its own. A value can appear inside a failure message, because the message quotes the application and your own expectations back at you — so whether messages leave the machine is governed by whether you export the logs signal.
Related
atr-author— writing, recording and reviewing specs.atr-browser— driving the browser by hand.docs/behavior-compilation.md— the full picture.
Companion skill —
atr-computercontrols the OS desktop (mouse, keyboard, screen, windows, app launch, plus an in-process LLM agent viaatr computer ask). Load both when:
- The browser action needs an OS file picker, drag from the file manager, or anything outside the page (e.g. authentication via 1Password desktop app).
- You want a single high-level instruction to span web + desktop —
atr computer ask "<instruction>"can callatr browseralong the way.Browser daemon listens on port 9333; computer daemon on 9334. Both can run simultaneously.
ATR Browser Automation Skill
This skill provides browser automation capabilities through ATR's browser server mode. The browser server runs as a daemon process and accepts CLI commands for browser control.
Architecture
Claude Code --> atr CLI (client) --> ATR Server + Browser
The browser runs in visible (non-headless) mode by default for debugging and verification.
Getting Started
Step 1: Check Browser Status and Start if Needed
Before any browser operations, verify the browser server is running:
atr browser status
If the server is not running, start it:
atr browser start
The server stores state at ~/.atr/browser.state which allows subsequent commands to discover the endpoint automatically.
Step 2: Navigate and Interact
Once running, use navigation and interaction commands to control the browser.
Command Categories
Lifecycle Commands
| Command | Description |
|---|---|
atr browser start [--port PORT] | Start browser daemon (default port: 9333) |
atr browser stop | Stop browser daemon |
atr browser status | Check if browser is running |
Navigation Commands
| Command | Description |
|---|---|
atr browser navigate <url> | Navigate to URL |
atr browser back | Go back in history |
atr browser forward | Go forward in history |
atr browser reload | Reload current page |
Page Management Commands
| Command | Description |
|---|---|
atr browser new-page [url] | Open new tab |
atr browser list-pages | List all tabs |
atr browser select-page <index> | Switch to tab (0-based) |
atr browser close-page <index> | Close tab |
Interaction Commands
| Command | Description |
|---|---|
atr browser click <target> [--double] | Click element (use --double for double-click) |
atr browser fill <target> <value> | Type into input field |
atr browser hover <target> | Hover over element |
atr browser press-key <key> | Press keyboard key (e.g., Enter, Tab, Control+A) |
atr browser drag <from> <to> | Drag element |
atr browser wait <selector> [--timeout] [--visible] | Wait for element to appear |
atr browser scroll --selector "<sel>" [--y N] [--to-bottom] | Scroll inside an element |
atr browser download-images "<sel>" [--output-dir] [--fallback-screenshot] | Download/screenshot images within elements |
atr browser viewport [W H] [--preset mobile|tablet|desktop|wide] | Get or set viewport size |
atr browser batch [--file F] [--on-error stop|continue|retry:N] | Execute multiple commands from stdin/file |
Recording Commands
| Command | Description |
|---|---|
atr browser record [--url URL] [-o FILE] | Record browser interactions as a behavior test |
Inspection Commands
| Command | Description |
|---|---|
atr browser snapshot [--verbose] | Get page elements with UIDs |
atr browser screenshot --file [--full] [-s SELECTOR] | Capture screenshot (saves to /tmp/) |
atr browser screenshot --file --selector-all "<sel>" | Screenshot all matching elements |
atr browser computed-styles "<selector>" [--properties] | Get computed CSS styles for single element |
atr browser computed-styles --selector-all "<sel>" | Get computed CSS styles for all matching elements |
atr browser computed-styles-diff "<sel>" --against N | Compare styles between pages |
atr browser text "<selector>" [--flat|--links|--headings] | Extract text content |
atr browser font-check "<font-family>" | Check if font is loaded and rendering |
atr browser clean-snapshot "<selector>" [--depth N] [--max-length N] | Get cleaned DOM subtree (no noise/tracking attrs) |
atr browser computed-styles --selector "h1" --selector "p" | Batch computed styles for multiple selectors |
atr browser computed-styles-diff --selector "h1" --selector "p" --against 0 | Batch style diff with overall score |
atr browser html | Get page HTML |
atr browser url | Get current URL |
atr browser title | Get page title |
atr browser eval <script> | Execute JavaScript |
atr browser ask "<question>" | Ask a question about the current page |
Screenshot Note: Use --file to save screenshots to /tmp/ with a timestamped filename (e.g., /tmp/atr-screenshot-20240105-103045.png). Add --full for full-page screenshots. Use --selector / -s to screenshot a specific element by CSS selector (e.g., -s "header", -s "#nav", -s "main > section:nth-child(2)"). Combine --selector with --full to capture an element's full scrollable height. Use --selector-all to screenshot every matching element as numbered PNGs — elements that fail or timeout are skipped (use --timeout <ms> to control per-element timeout, default 30s). Without --file, returns base64-encoded image data.
Debugging Commands
| Command | Description |
|---|---|
atr browser console [--limit N] | Get console messages (default: 50) |
atr browser network [--limit N] | Get network requests (default: 50) |
atr browser errors | Get failed requests |
Workflow Pattern
Follow this workflow for browser automation tasks:
-
Ensure Server Running
atr browser status || atr browser start -
Navigate to Target
atr browser navigate https://example.com -
Inspect Page Elements
atr browser snapshotThis returns elements with unique IDs (UIDs) like
e0,e1, etc. -
Interact with Elements Target elements by:
- Text content:
"Sign In" - UID from snapshot:
e5 - CSS selector:
.submit-button
- Text content:
-
Verify Results
atr browser url atr browser title atr browser screenshot --file -
Cleanup When Done
atr browser stop
Asking Questions About a Page
Use atr browser ask when you need specific information from a page without flooding your context with raw HTML or snapshot data. A lightweight sub-agent inspects the page using multiple tools and returns a concise text answer.
atr browser ask "What is the main heading on this page?"
atr browser ask "How many items are in the navigation menu?"
atr browser ask "Is there a login form on this page?"
Prefer ask over html or snapshot when:
- You need a specific fact, not the full page structure
- You want to keep your context clean for subsequent reasoning
- The answer can be expressed as a short text response
Recording Browser Interactions
Use atr browser record to capture user interactions and output a .test.txt behavior test file. A floating overlay appears in the browser showing recorded steps in real time.
# Record interactions on a page
atr browser record --url https://example.com -o repro-steps.test.txt
# Record on the already-open page
atr browser record -o flow.test.txt
Stop recording with Ctrl+C in the terminal or the "Stop" button in the browser overlay.
The recorder captures clicks, form fills, keyboard shortcuts, navigation, scrolling, and select changes. It handles shadow DOM (web components) and generates stable CSS selectors. Password fields are automatically masked.
The output .test.txt file can be replayed with atr run --behavior repro-steps.test.txt.
Using JSON Output
Add --json flag for structured output when parsing is needed:
atr browser snapshot --json
atr browser list-pages --json
atr browser network --json
Element Targeting
The <target> parameter in click, fill, hover, and drag commands accepts:
- Element UID:
e0,e5(from snapshot output) - Visible text:
"Sign In","Submit Form" - aria-label: Elements with matching aria-label attribute
- data-testid: Elements with matching data-testid attribute
- CSS selector:
#login-button,.nav-link,header,footer,nav,main > section,div.hero,li:nth-child(2)
Best practice: Use atr browser snapshot first to see available elements and their UIDs.
Keyboard Keys
For press-key command:
- Named keys:
Enter,Tab,Escape,Backspace - Modifiers:
Control+a,Shift+Tab,Alt+Enter - Arrow keys:
ArrowUp,ArrowDown,ArrowLeft,ArrowRight
Troubleshooting
Browser won't start:
atr browser status
rm ~/.atr/browser.state # If stale state
atr browser start
Port already in use:
atr browser start --port 9334
Element not found:
atr browser snapshot --verbose --json
Additional Resources
For complete command reference with all flags, see references/commands-reference.md.
Companion skill —
atr-browsercontrols Chromium for any in-page operation (navigate, click on web elements, fill forms, scrape). Load both when:
- You need to drag a file from the OS file manager into a browser upload zone.
- Authentication or a native dialog (1Password, system file picker) is part of the flow.
- You're delegating a high-level task that touches both —
atr computer ask "..."can shell out toatr browser ...and vice versa.Computer daemon listens on port 9334; browser daemon on 9333. Both can run simultaneously.
ATR Desktop Computer-Use Skill
This skill provides cross-platform desktop control through ATR's computer daemon. Mouse, keyboard, screen capture, window management, app launch, and an in-process LLM agent (atr computer ask) — all driven from the same atr CLI that powers atr-browser.
Architecture
Claude Code --> atr CLI --> ATR Computer Daemon --> robotgo (X11 / macOS / Windows)
The daemon enforces a configurable safety countdown before every action so the user can intervene with Ctrl+C. Read-only operations (screenshot, position, list windows) skip the countdown.
When the daemon is started without --no-gui it also surfaces the countdown in a small native overlay so MCP/Claude-Code clients (which don't see the daemon's terminal) can still interrupt visually. On Linux the overlay uses zenity (full Cancel button — clicking it aborts the action with ErrAborted); when only notify-send is available it falls back to a one-shot desktop notification (visual only — abort still works via Ctrl+C in the daemon terminal). On macOS it uses osascript display notification; on Windows it uses a PowerShell toast. If no backend can initialize, the daemon logs a warning and continues with terminal-only countdown.
Linux: X11 only in v1. Wayland is not supported yet. GUI overlay: install
zenityfor an abortable dialog (apt install zenity); fall back tolibnotify-binfor visual-only notifications.
Getting Started
Step 1: Check status and start if needed
atr computer status
If not running:
atr computer start # default: per-request 3s countdown
atr computer start --countdown-mode per-app # countdown only on first action per app
atr computer start --countdown-mode off # no countdown (trusted batch runs)
atr computer start --countdown 1 # 1-second countdown
State and logs:
- State:
~/.atr/computer.state - Daemon log (where countdown messages appear):
~/.atr/computer.log
Step 2: Take a screenshot
atr computer screenshot --output /tmp/desk.png
atr computer screenshot --output /tmp/region.png --region 0,0,800,600
Step 3: Drive mouse and keyboard
atr computer click 800 50
atr computer click 800 50 --double
atr computer click 800 50 --button right
atr computer move 100 100
atr computer drag --from 200,300 --to 1200,400
atr computer scroll --dy -3
atr computer hover 500 500
atr computer type "Hello, world"
atr computer key enter
atr computer chord ctrl+shift+t
Step 4: Manage windows
atr computer --json window list # enumerate
atr computer window active # focused window
atr computer window focus --title "Firefox" # by title or app name
atr computer window minimize --title "Slack"
atr computer window maximize --title "Code"
atr computer window restore --id 12345
atr computer window close --title "Calculator"
atr computer window move --id 12345 --to 100,100
atr computer window resize --id 12345 --size 800,600
Step 5: Launch / quit apps
atr computer app launch firefox
atr computer app launch xclock
atr computer app quit firefox
Step 6: Stop the daemon
atr computer stop
Safety countdown modes
| Mode | Behavior |
|---|---|
per-request | Default. Every gated action shows a countdown. |
per-app | First action against an app prompts; subsequent actions auto-approve. |
off | No countdown. Explicit opt-in for trusted batches. |
Aborting: press Ctrl+C in the daemon's terminal during a countdown. The action returns aborted by user.
Combined workflow with atr-browser
ATR can drive both desktop and browser in one flow. Example: end-to-end drag-and-drop between the OS file manager and a browser upload zone.
atr browser start
atr browser navigate https://example.com/upload
atr computer start --countdown-mode off
atr computer app launch nautilus
atr computer window focus --title "Files"
atr computer drag --from 200,300 --to 1200,400 # drag file into browser
atr browser screenshot --output /tmp/dnd.png
For browser-only workflows, see the atr-browser skill. For combined workflows, both daemons can run simultaneously on different ports (browser: 9333, computer: 9334).
Asking the agent (atr computer ask)
Instead of issuing one command at a time, you can describe a desktop task in plain language and let an agent loop drive the screenshots, clicks, and keystrokes:
atr computer ask "open xclock and tell me what time it shows"
atr computer ask "list the open windows that contain 'chrome' in the title"
atr computer ask --max-steps 30 --timeout 10m "open the GNOME calculator and compute 17 * 23"
The agent uses the LLM backend configured for ATR (backend: gemini-api | vertex-ai | vertex-claude | claude-cli in ~/.atr/config.yaml). Default model aliases:
| Backend | Tier | Model |
|---|---|---|
gemini-api / vertex-ai | flash | gemini-3.7-flash |
gemini-api / vertex-ai | pro | gemini-3.1-pro-preview |
vertex-claude | sonnet | claude-sonnet-5 |
vertex-claude | opus | claude-opus-5 |
Override per-run with the global --model flag (e.g. atr --model pro computer ask "..."). With backend: claude-cli, the model flag is ignored and Claude CLI is invoked as a subprocess.
Tools the agent can call (curated subset):
- Perception:
computer_screenshot(image fed back to the LLM),computer_displays,computer_active_window,computer_list_windows,computer_position - Actuation:
computer_click,computer_type,computer_press_key,computer_key_chord,computer_focus_window,computer_window_state(minimize/maximize/restore/close),computer_launch_app
Limits and gotchas:
- The agent cannot type passwords — if a sudo / polkit prompt appears, it stops and reports the blocker.
- Each screenshot sent to the LLM costs vision tokens; with
--model proa 10-step task is measurable spend. - The countdown gate inherits whatever the daemon was started with. For ask runs you usually want
atr computer start --countdown-mode per-appso the first action against an app prompts and subsequent ones auto-approve.
Coordinates and multi-monitor
ATR uses root coordinates as the public API for clicks, moves, drags, and window positions. Root coords have a single bounding-box origin at (0, 0) covering all monitors — every visible coordinate is non-negative, matching xrandr and the X11 root window. atr computer --json window list and atr computer displays both return root coords.
For interactive use it's often easier to think in display-local pixels. Every mouse command accepts --display N, which interprets x and y as pixels relative to that display's top-left:
# Display 0 is primary (whose root origin may NOT be (0, 0) on multi-monitor setups)
atr computer click --display 0 100 200 # display-local
atr computer click 1540 200 # root coords (equivalent on a setup where primary is at root (1440, 0))
atr computer screenshot --region X,Y,W,H is always display-local within the chosen --display.
A common workflow:
atr computer displays # see each display's root bounds
atr computer --json window list # find a window's root bounds
atr computer screenshot --display 0 --region X,Y,W,H # crop using display-local pixels
atr computer click --display 0 X Y # click the spot you cropped
Tips
- Use
atr computer --json window listto enumerate windows, then operate on a specific ID for stability. - For Linux, the per-app cache key is
WM_CLASS(window class), so all Chrome windows share one approval. - Screenshots return PNG bytes; set
--outputto a local path the user expects. - For multi-monitor setups: prefer
--display Nwith display-local pixels for new code; reach for absolute root coords only when relaying values fromwindow list.
MCP servers 1
Sponsors
Details
- Marketplace
- imyousuf/agentic-test-runner
- Category
- Community