Skills: browser control + recording (atr-browser), cross-platform desktop control with in-process agent + multi-monitor support (atr-computer: mouse, keyboard, screen, windows, apps, 'computer ask' for natural-language tasks), run tests with AI analysis (atr-analyze), behavior testing (atr-behavior). MCP Server: 50+ tools (30 browser + 22 desktop) via 'atr mcp serve'

version
1.9.0
GitHub stars
13

Install

/plugin install atr-skills@atr-marketplace

Run it inside Claude Code after adding the marketplace with /plugin marketplace add imyousuf/agentic-test-runner.

Contents

Discovered from source

Skills 5

ATR Command Analysis Skill

This skill runs commands through ATR (Agentic Test Runner) which provides AI-powered analysis and clean summarized output. Use this as your default way to run tests, linters, type checkers, and builds - the AI analyzes the full output and returns a concise summary, keeping your conversation context clean and focused.

When to Use This Skill vs Direct Commands

Use ATR analyze (RECOMMENDED DEFAULT) when:

  • Running test suites (pytest, jest, go test, npm test, make test, cargo test)
  • Running linters (eslint, golangci-lint, pylint, flake8, make lint, cargo clippy)
  • Running type checkers (mypy, tsc, pyright, make typecheck)
  • Running builds that produce verbose output (make build, npm run build, cargo build)
  • You want clean, summarized results instead of raw output
  • You want automatic failure analysis if something goes wrong

Use direct Bash commands only when:

  • Running quick one-off commands with minimal output
  • You specifically need the raw, unprocessed output
  • Interactive commands that require user input

Basic Usage

atr run --cmd "<command>"

Examples:

# Tests
atr run --cmd "go test ./..."
atr run --cmd "npm test"
atr run --cmd "pytest tests/"
atr run --cmd "make test"

# Linting
atr run --cmd "make lint"
atr run --cmd "eslint src/"
atr run --cmd "golangci-lint run ./..."

# Type checking
atr run --cmd "make typecheck"
atr run --cmd "tsc --noEmit"
atr run --cmd "mypy src/"

# Builds
atr run --cmd "make build"
atr run --cmd "npm run build"

Adding Context

Provide context to help the AI agent focus its analysis:

atr run --cmd "<command>" --context "<context>"

Examples:

atr run --cmd "go test ./..." --context "Tests started failing after refactoring the auth module"
atr run --cmd "npm run build" --context "Added new dependency yesterday"
atr run --cmd "pytest" --context "Testing the new payment integration"

Command Options

FlagDescription
--cmd <command>Command to execute (required)
--cwd <path>Working directory
--context <text>Additional context for AI agent
--model flash|proModel tier (flash=fast, pro=deep analysis)
--python-venv <path>Python virtual environment path
--nvm-version <version>Node.js version via nvm
--no-auto-envDisable automatic environment detection

Working Directory

Specify where to run the command:

atr run --cmd "npm test" --cwd "/path/to/project"

Environment Detection

ATR automatically detects and activates appropriate environments:

Python projects:

  • Detects .venv, venv, or Poetry environments
  • Auto-activates virtual environment

Node.js projects:

  • Detects .nvmrc or package.json engine requirements
  • Auto-activates correct Node.js version via nvm

Override automatic detection:

atr run --cmd "pytest" --python-venv /custom/path/.venv
atr run --cmd "npm test" --nvm-version 18
atr run --cmd "make" --no-auto-env

Model Selection

Use different models for different needs:

# Quick analysis (default)
atr run --cmd "make build" --model flash

# Deep analysis for complex issues
atr run --cmd "go test ./..." --model pro

What the AI Agent Does

The ATR agent processes command output and:

  1. Summarizes results into a clean, concise report
  2. Identifies test pass/fail status and key metrics
  3. Analyzes any failures for error patterns and root causes
  4. Reads relevant source files when failures occur
  5. Provides actionable recommendations for fixing issues

This keeps your conversation context clean by replacing verbose test output with a focused summary.

Example Output

Executing: go test ./...
Directory: /path/to/project

--- FAIL: TestUserAuth (0.05s)
    auth_test.go:42: expected 200, got 401

Command failed (exit code: 1)

Analyzing failure with AI agent...

======================================================================
ANALYSIS RESULTS
======================================================================

Status: FAILURE

Summary:
  TestUserAuth fails because the auth middleware expects a JWT token,
  but the test doesn't provide one in the request headers.

Root Cause:
  Line 38 in auth_test.go creates a request without Authorization header.
  The auth middleware (middleware/auth.go:15) rejects it with 401.

Recommendations:
  1. Add mock JWT token to test request
  2. Or bypass auth middleware in test setup
  3. Check if middleware was recently added to the route

Files Examined:
  - auth_test.go
  - middleware/auth.go
  - routes/api.go

Exit Codes

CodeMeaning
0Command passed
1The thing under test is broken (analysis provided)
2The run could not decide — missing input, stale compiled script, browser or model unavailable

Configuration

Configure ATR in ~/.atr/config.yaml:

backend: gemini-api  # or vertex-ai
model: flash         # or pro

gemini:
  api_key: "your-key"

# Or for Vertex AI:
vertex:
  project: your-project
  location: us-central1

Environment variables:

export GEMINI_API_KEY="your-key"
# Or
export GOOGLE_CLOUD_PROJECT="project-id"

Best Practices

  1. Use as default for running test suites to keep conversation context clean
  2. Provide context when the failure might be related to recent changes
  3. Use --model pro for complex, multi-file issues
  4. Specify --cwd when running from a different directory
  5. Check environment with atr test-cmd-env "<command>" to preview detection

Writing ATR behaviour specs

A .test.txt spec is compiled once, by a model driving the real application, into a JavaScript file that replays with no model in the loop. Running is solved. Writing a spec that compiles, and that does not lie when it passes, is the hard part — and it is what this skill is for.

The spec text reaches the compiler verbatim. ATR does not parse sections; it passes the whole file through. Everything you write in it is instruction to the model. That is the whole reason authoring guidance is a skill rather than a manual: the spec is the one input to the compiler that does not ship with the compiler.

Syntax lives in references/test-file-format.md. Start with references/spec-template.test.txt. What follows is judgement.


Rule 1: never write an assertion that cannot fail

A false pass is the worst thing ATR can produce — worse than a crash, because nobody investigates a green test.

These are all assertions that cannot fail:

Expected Results:
- Steps completed successfully      ← the steps succeeded by construction
- No console errors                 ← true on most broken pages
- The page loads                    ← it loaded before you clicked anything

Write what would be different if the feature were broken:

Expected Results:
- The order confirmation shows the order number
- The cart badge reads 0
- The "Place order" button is gone

If you cannot name something that would differ, you do not yet have a test.

Rule 2: assert on an element, not on page text

A substring match against the whole page passes for the wrong reason. One real spec searched for "archiv" and matched three unrelated things on the page.

  • Bad: "the page contains 'Archived'"
  • Good: "the row's status badge reads 'Archived'"

Name the container. The compiler emits atr.text(selector) when you name one and a whole-page match when you do not.

Rule 3: assert absence as well as presence

"Read-only" does not mean a notice appeared. It means zero editors and zero send buttons. A spec that only checks for the banner passes on a page that shows the banner and still lets you type.

Absence is only meaningful once the page has settled — wait for the state that removes the thing (the confirmation, the empty list, the new view) and then assert it is gone.

Rule 4: count before and after, never absolutely

"The last entry in the log is X" breaks the second time the test runs into the same log. "One more entry than before, and it reads X" survives.

Tests run repeatedly against shared state. Any assertion phrased as an absolute position or an absolute total is a time bomb.

Rule 5: a submit is request → navigate → repaint

Not one event. Say what state ends the step:

  • Bad: "Click Save and wait 2 seconds"
  • Good: "Click Save and wait for the confirmation toast"

Never ask for a fixed sleep. The compiler will wait for a state if you name one, and will guess if you do not.

Say it as one expectation, not a wait and then a check. The compiler emits atr.expectText, which waits and asserts together; the two-call form hands the diagnosis to whichever call hits the wall first — always the wait — so a page that stops reaching the state is reported as a timeout rather than as the feature being broken, and CI retries it instead of escalating.

Rule 6: a destructive test rebuilds its own fixture

A compile drives your spec more than once — once to learn the application, then again to verify what it wrote. A spec that deletes the only record it depends on works once and then fails for ever.

Put the setup in the spec and say it is setup:

Setup:
- Create a project named "atr-fixture-{run}" if one does not exist

The compiler turns that into atr.setup(...), which runs before the steps on every execution and is not counted as a step. A failure there is reported as the fixture failing, not the application misbehaving.

This is also the answer to "how do I test deletion": create the thing, delete the thing, assert it is gone.

Rule 7: read the DOM before naming a control

Labels hide in at least three places — placeholder, aria-label, and <label for> — and two dialogs in the same application will disagree about which. Use atr browser snapshot and quote what is actually there.

Rich-text editors are the recurring trap: they reject a plain fill and need an input-event recipe. The class is universal even though the recipe is app-specific, so put the recipe in Notes for the compiler (below).

Rule 8: a missing input is nobody's bug

config is the one failure kind the agent will not help with: not repairable, not retryable, never sent to the model. A person has to decide what the value should be.

That is deliberate. The tempting "fix" for a missing value is to inline the literal back into the script, which undoes the reason inputs live outside it.

Values live in <spec>.test.properties (committed) beneath <spec>.test.override.properties (gitignored) and ATR_VALUE_* variables. A committed properties file executes — $(command) and ${VAR} expand at read time, on every machine including CI.

Rule 9: when sign-in cannot be scripted, seed the browser

Some applications cannot be logged into by a script — SSO, a hardware token, a one-time code.

atr browser start          # start the daemon
# sign in by hand in the window that opens
atr run --behavior tests/  # reuses the running daemon's browser automatically

No flag is needed: atr run picks up a running daemon's CDP endpoint on its own, and opens an isolated tab for the test.

Name the symptom of forgetting, because it is unrecognisable otherwise: every step times out on a sign-in page, which looks exactly like a hang.

For a credential that can be scripted, use a secret reference rather than a value — the compiler emits atr.fillSecret, which fetches and types inside one call so the value never enters the script, the transcript, or a later repair prompt.

Rule 10: a step is one journey, not one instruction

How you break a spec into steps decides what can ever be shared between specs.

A hoisted operation replaces a run of consecutive statements inside a single step. So if you write:

1. Open the tags page
2. Follow the "rest" tag

the compiler puts a check at the end of step 1 — it arrived, after all — and now the navigate and the click sit either side of an assertion and either side of a step boundary. They can never become one shared openTag(), because gathering them would mean moving the assertion, and an assertion never moves.

Write the journey as one step and the checks as the next:

1. Open the tags page and follow the "rest" tag
2. Confirm the reader is looking at that tag's posts

This is also just better spec writing: a step is a thing the reader does, and Expected Results is what must then be true. The sharing falls out of that.

Rule 11: share operations, never assertions

You do not have to write the shared library. Compile two specs that drive the same journey and ATR hoists it for you: it finds the repetition, names it, rewrites both scripts to call it, and replays them before keeping the change. Nothing is kept unless every rewritten script still claims exactly what it claimed before, character for character.

A compile is also shown what its neighbours wrote, so two specs reaching the same page use the same selector and the same constant name rather than inventing their own. That is what makes the repetition findable at all.

Do not name a library function in a spec. A step that reads "open the front page using openFirstPost() from the shared library" does not fail when no such operation exists — the compiler writes a local function of that name and calls it, and the test passes. It reads as though the script shares code when it does not: the next spec making the same journey writes it again, and if a real library operation of that name ever appears, the local one silently wins over it. The lint reports this as local-helper. Describe what the reader does; deciding what deserves a name is ATR's job.

You can still write _shared.js by hand, and reading it is worth it either way — it is loaded into the same scope before every script in the directory, so its top-level functions are simply in scope:

// tests/e2e/_shared.js
function signIn(username) {
  atr.navigate("/login");
  atr.fill("#username", username);
  atr.fillSecret("#password", {ref: "app_password"});
  atr.click("#submit");
  atr.waitFor("#dashboard");     // ends on the dashboard
}

The compiler is shown this file verbatim, so it calls signIn() instead of rediscovering eight steps — which is the real saving. Duplication is the obvious cost of not having it; compile time is the larger one, and every rediscovery is a fresh chance to get it subtly different.

Three rules, and the first is enforced rather than requested:

  • Operations only. expect and atr.fail refuse to run from the library. Once a test's assertions live in shared code you can no longer read the test and know what it checks, and one edit can weaken every test in the directory.
  • Declarations only. No code at the top level: everything there runs before step 1 of every spec beside it. No atr.step or atr.setup either — steps belong to a spec, and a library that declares them renumbers all of them.
  • Take inputs as parameters. values is per-spec, so a library that reads values.get("username") needs every spec in the directory to define it.

A defect in the library is config: never repaired, never retried, never sent to the model. A person fixes shared code.

atr refactor-ops tests/ --dry-run says what could be hoisted without changing anything or opening a browser. Turn the automatic pass off with behavior.extract_operations: on-demand or --no-extract.

Under --no-compile — which is what CI runs — hoisting only ever reports. Applying it would call the model and rewrite the scripts, and that flag permits neither, so a replay job never comes back with a modified working tree.

Editing the library does not force a recompile. Scripts carry a second header, // atr-lib-sha256:, and a change to it means the script is unproven against the new library, not wrong — so it replays and the header catches up. Only a genuine signature break reaches the model, and only for the specs that actually broke.

The honest trade-off: this is Page Object Model, and it carries POM's bargain. You keep "read the script, see exactly what it asserts". You lose "read the script, see exactly how it got there".


Notes for the compiler

The section nobody knows exists, and the one that decides whether a hard spec compiles. It goes at the end of the file and holds everything the model cannot learn by looking at one page:

  • app-specific recipes (the rich-text editor; the date picker that ignores typing)
  • which of several similar controls is the right one
  • state the application carries between pages
  • anything you discovered by watching a compile fail

Every note should be a trap that produced a wrong result. If you cannot say what would go wrong without it, leave it out — the notes are prompt, and padding costs iterations.

Do not restate what the compiler already knows. These are in the binary's prompt and repeating them wastes tokens: use atr.setup for fixtures, wait for state instead of sleeping, atr.expectExists/atr.expectMissing for presence, selectors over page text, :has-text() for text-qualified targets.


The loop

  1. Bootstrap. atr browser record --output tests/thing.test.txt gives you the steps. It cannot give you the expectations — it leaves a TODO where they belong. Answer it; that is rule 1.

  2. Compile once. atr run --behavior tests/thing.test.txt. It drives the real application, so it needs a base URL and takes minutes.

  3. Read the emitted .test.js before trusting it. This is the step nobody does. You are checking one thing: could this script fail? A step with no assertion is a step that passes whatever happens.

  4. Commit the script. It carries a hash of the spec, so a reviewer can see what changed and CI can replay it. A generated file whose behaviour depends on who ran it last is worse than no file.

  5. --no-compile in CI. It replays or it fails loudly. Without it a spec edit silently turns a 9-second CI step into a multi-minute model run.

  6. --prune-values occasionally. Repeated compiles accumulate aliases — one real spec ended up with seven keys for two inputs.

  7. Backport the compiler's corrections. When a compile discovers your spec was wrong — you wrote button[type="submit"], the real control is button:not([type]) — put the correction back in the spec. Nobody does this, and spec and script diverge silently from then on.

    The reason nobody does it: editing the spec changes its hash and forces a recompile. So batch backports and do them when a recompile is due anyway.


Before you commit

Read the generated .test.js against this list. It is short because it only contains things that make a compiled test lie:

  • Every step either asserts something or performs an action that throws when it fails. A step of pure atr.log cannot fail.
  • No toContain with a short needle against atr.text() with no selector.
  • No atr.sleep outside atr.retry — a fixed sleep is a race that has not happened yet.
  • Absence is asserted where the spec says something goes away.
  • The spec still describes what the script does.
  • The properties file has no key the script does not read, and no secret.

atr run runs the mechanical half of this list for you — a step that can only succeed, a script with no assertion at all, a short substring matched against the whole page, a fixed sleep — and refuses to accept a script that cannot fail. --lint=warn reports without refusing while a suite adopts the check. The judgement half is yours.


Failure kinds, and what each one asks of you

kindmeanswhat you do
assertionthe application is wrongfix the app — never the assertion
not_foundthe UI movedlet the agent repair it, then check the diff
timeoutslow, flaky, or hangingretried automatically; if persistent, the spec is waiting for the wrong thing
environmentbrowser or networknot about your test
configan input is missingyou must supply it; the agent will not
scriptthe generated JavaScript is wrongthe compiler's fault; it repairs itself

Only assertion turns a suite red (exit 1). Everything else exits 2, so a CI job can retry rather than escalate.


A worked example

examples/behavior/blog/ is three specs against a live blog, with the inputs, the shared library and the JavaScript the compiler produced from them. It is the only example written against a site with no data-testid anywhere, whose content changes when its author publishes — which is what most real applications look like.

Worth reading for what the compiler did with the notes section, and for how the expectations are phrased so that publishing a new post does not break them.

Related

  • atr-behavior — running specs that already exist: flags, exit codes, triage.
  • atr-browser — driving the browser by hand while you work out what the spec should say.
  • references/test-file-format.md — the syntax.
  • references/spec-template.test.txt — start here.

Running ATR behaviour tests

Runs browser tests written as natural-language .test.txt specs. A spec is compiled once into a sibling .test.js and afterwards replays with no model in the loop — seconds, and no tokens. The agent comes back only to diagnose a failure.

Writing a spec is a different job. Load atr-author for that: which assertions are worth writing, what belongs in the notes section, and how to avoid a test that passes without testing anything.

Basic usage

atr run --behavior tests/login.test.txt              # compile if needed, then replay
atr run --behavior tests/e2e/                        # every .test.txt in the directory
atr run --behavior tests/e2e/ --browser-url http://localhost:3000

Flags

FlagDescriptionDefault
--behavior <path>Test file or directory(required)
--browser-url <url>Base URL for the applicationfrom config or base_url
--headlessRun the browser headlessfalse
--viewport <WxH>Viewport size1920x1080
--cdp-endpoint <url>Connect to an existing browser-
--no-compileReplay only; never call the model. Fails loudly if the script is missing or stalefalse
--recompileRegenerate the script even if it matches the specfalse
--no-repairDiagnose a drifted script but do not rewrite itfalse
--prune-valuesRemove inputs the script no longer readsfalse
--lint <mode>What to do about a script that cannot fail: error, warn, offerror
--no-triageNever ask the model why a failure happened, even to classify itfalse
--otel-endpoint <url>OTLP collector for run telemetry, e.g. http://localhost:4318from config or env
--interpretSkip compilation; let the agent drive every stepfalse
-v, --verboseShow the script's own atr.log() outputfalse

Cost: know which mode you are in

Use --no-compile whenever cost must be predictable — always in CI. It replays, or it fails and says why. It never calls the model.

Without it, any of these silently turns a 9-second step into a multi-minute model run: the spec was edited, the script is missing, the script has never completed a run, or a failure invites a repair.

atr run --behavior tests/ --no-compile   # CI: replay or fail
atr run --behavior tests/login.test.txt  # local: compile if needed

--no-compile needs no backend at all — no API key, no project, no gcloud auth application-default login. A CI job that only replays does not have to hold credentials, and one that has them spends at most a single call, only when a run has already failed, to say why (see below). (--interpret is refused alongside it, since it is nothing but model calls.)

Each run prints how it was serviced (compiled →, repaired →) and how many model calls it cost. A passing replay costing more than zero is worth investigating; a failing one costs one call to classify the failure, which is the point.

A script that cannot fail is refused

Before a script runs, ATR checks statically that it can fail: that the script asserts something, and that no step is built only from reads. A step of pure atr.log reports success whatever the application does, and a suite of those reports that everything works right up until someone opens the application.

A blocking finding stops the run and exits 2 — nothing was learned about the application, so it is not a test failure. The findings never go to the model: a model asked to invent the missing assertions will invent ones that pass.

atr run --behavior tests/ --lint=warn   # report but run; for adopting the check
atr run --behavior tests/ --lint=off    # skip it

Looser problems — a short substring matched against the whole page, a fixed atr.sleep outside a retry — are reported and not enforced.

Freshness

The compiled script is committed and carries a hash of the spec:

// atr-spec-sha256: 9f2c…

Edit the spec and the next run recompiles. Reflow whitespace and it does not — the hash is over normalised text, so sha256sum on the raw file will never match it. A script that has never completed a run carries // atr-unverified and is recompiled rather than trusted. Deleting the hash line marks a script hand-written and off-limits to the compiler: it is replayed as it is, under --no-compile too, and never recompiled, repaired or hoisted. A failure is still triaged — against what the script itself checks, not the spec — and the reason reported; no rewrite is asked for or applied. --recompile is the one way to replace it with a compiled script.

A compile drives the spec more than once — once to learn the application, then again to verify what it wrote — so a destructive spec needs a rebuildable fixture. That is what atr.setup(...) in the compiled script is for, and what a Setup: section in the spec compiles to.

Test inputs

Beside the spec:

  • login.test.properties — committed, written by the compiler, editable.
  • login.test.override.properties — gitignored, wins over it.
  • ATR_VALUE_* environment variables — win over both.

base_url here is what lets one spec run against localhost and staging with no flags. Values support $(command) and ${VAR}, expanded when read — so a committed properties file executes, on every machine including CI.

--prune-values removes keys the script no longer reads. It reports without removing unless you pass it, scans the shared library as well as the script, and says nothing at all when either builds a key at run time, because then no scan can be sure.

Shared operations

A _shared.js beside the specs is loaded into every script in that directory. It holds operations — sign in, create the thing — never assertions: expect and atr.fail are refused from it.

Editing it does not force a recompile. Scripts carry a second header, // atr-lib-sha256:, and a mismatch means the script is unproven against the current library rather than stale — so it replays and the header is updated when it passes. Under --no-compile the update is reported instead of written, so CI does not leave a dirty tree.

Hoisting follows the same rule. A run normally hoists the operations its specs keep repeating into _shared.js; under --no-compile it reports what could be hoisted and changes nothing, because applying it would call the model and rewrite the scripts — neither of which that flag permits. Run atr refactor-ops <dir> to act on it.

A defect in the library is a config failure: never repaired, never retried, never sent to the model.

Exit codes

codemeaningwhat CI should do
0every test passed—
1the application is broken — an assertion failedescalate
2nothing was learned about the application — missing input, browser or model failure, stale script under --no-compileretry, or fix the environment

1 means the app is wrong and nothing else. That separation is the point: a red build that can also mean "the CI box was slow" is a red build people learn to ignore.

Failure kinds

Printed with every failure, and each asks for a different response:

kindmeansATR's response
assertionthe application did not do what the spec requiresstops immediately, no model call — never auto-repaired
not_founda target is gone; the UI movedagent repairs the script, budget of 1
timeoutslow, flaky, or hangingretried, then triaged
environmentbrowser, daemon or networkretried, then reported as infrastructure
configan input this checkout does not definereported; never repaired or retried — a person must supply it
scriptthe generated JavaScript is wrongrepaired

An assertion failure never reaches the model. That is deliberate: asking a model to confirm a regression spends the tokens compilation exists to save, and risks it "fixing" the assertion that caught the regression.

A regression can present as a timeout. If a script waits for a state and then asserts it, the wait fails first — so a page that stops reaching that state is reported as a timeout rather than as the application being wrong. atr.expectText waits and asserts in one call and cannot be misattributed that way; the lint reports the two-call form and names it.

For scripts already written the other way, triage covers it: a failure that survives its retries is classified, and a verdict of "the application is at fault" makes the run exit 1.

Triage is not compilation. --no-compile refuses to generate a script; it no longer refuses to judge one. A replay that fails will spend one model call to classify it — only on a run that has already gone red, and never to rewrite anything, since CI asked for a replay. If no backend is configured it says so and reports the runtime's own classification, so nothing new is required of a CI job that had none. --no-triage restores the absolute guarantee.

Debugging a failure

atr run --behavior tests/x.test.txt --headless=false   # watch it
atr run --behavior tests/x.test.txt -v                 # show atr.log output
atr run --behavior tests/x.test.txt --no-repair        # diagnose without rewriting

Then read the compiled .test.js — it is ordinary, readable JavaScript, and the failure names the step and the target.

Signing in by hand

When authentication cannot be scripted (SSO, a hardware token, a one-time code):

atr browser start          # start the daemon; a window opens
# sign in by hand
atr run --behavior tests/  # reuses the running daemon's browser automatically

No flag is needed — atr run picks up a running daemon's CDP endpoint on its own and opens an isolated tab for the test, leaving your tabs alone.

Configuration

~/.atr/config.yaml:

behavior:
  base_url: "http://localhost:3000"
  browser:
    executable: "auto"
    headless: true
    viewport: { width: 1920, height: 1080 }
    page_timeout: "30s"
    action_timeout: "10s"

What was run, over time

Every behaviour run is recorded locally in ~/.atr/history.db, including the runs that never reached the application — a missing input, a browser that would not start, a stale script under --no-compile. Those are the ones a pass rate has to exclude to mean anything.

atr history                          # per spec: runs, pass/fail/infra, flakes, repairs
atr history --spec tests/login.test.txt --runs
atr history --since 7d --json

Two numbers only ATR can give you:

  • Repair frequency. A spec repaired again and again is not flaky — the application's DOM is churning underneath it.
  • True failure rate. Pass rate over the runs that actually tested the application. FLAKE counts runs that passed only after a retry, which are green in every other report.

The median REPLAY duration is worth watching: a replay is deterministic with no model in the loop, so a suite drifting from 9s to 15s means the application got slower.

The database is plain SQLite and the views (runs, attempts, compiles) are a stable contract, so anything the command will not tell you is one query away:

sqlite3 ~/.atr/history.db "SELECT spec, outcome, count(*) FROM runs GROUP BY 1,2"

Point ATR at an OTLP collector and the same runs are exported as traces, metrics and logs — which is how a CI run's history survives the container being torn down. Three ways, in precedence order:

atr run --behavior tests/ --otel-endpoint http://localhost:4318   # this run only
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318          # the standard variable
telemetry:
  endpoint: "http://localhost:4318"    # ~/.atr/config.yaml

Give the collector's base URL; the signal path (/v1/traces and friends) is appended, and a URL that already names a signal is left alone. With no endpoint anywhere, nothing is emitted and no error is logged.

Turn either off in ~/.atr/config.yaml; both may be off at once:

history:
  enabled: true
  keep_days: 90
telemetry:
  enabled: true

Nothing ATR records lifts a resolved value into a field of its own. A value can appear inside a failure message, because the message quotes the application and your own expectations back at you — so whether messages leave the machine is governed by whether you export the logs signal.

Related

  • atr-author — writing, recording and reviewing specs.
  • atr-browser — driving the browser by hand.
  • docs/behavior-compilation.md — the full picture.

Companion skill — atr-computer controls the OS desktop (mouse, keyboard, screen, windows, app launch, plus an in-process LLM agent via atr computer ask). Load both when:

  • The browser action needs an OS file picker, drag from the file manager, or anything outside the page (e.g. authentication via 1Password desktop app).
  • You want a single high-level instruction to span web + desktop — atr computer ask "<instruction>" can call atr browser along the way.

Browser daemon listens on port 9333; computer daemon on 9334. Both can run simultaneously.

ATR Browser Automation Skill

This skill provides browser automation capabilities through ATR's browser server mode. The browser server runs as a daemon process and accepts CLI commands for browser control.

Architecture

Claude Code --> atr CLI (client) --> ATR Server + Browser

The browser runs in visible (non-headless) mode by default for debugging and verification.

Getting Started

Step 1: Check Browser Status and Start if Needed

Before any browser operations, verify the browser server is running:

atr browser status

If the server is not running, start it:

atr browser start

The server stores state at ~/.atr/browser.state which allows subsequent commands to discover the endpoint automatically.

Step 2: Navigate and Interact

Once running, use navigation and interaction commands to control the browser.

Command Categories

Lifecycle Commands

CommandDescription
atr browser start [--port PORT]Start browser daemon (default port: 9333)
atr browser stopStop browser daemon
atr browser statusCheck if browser is running

Navigation Commands

CommandDescription
atr browser navigate <url>Navigate to URL
atr browser backGo back in history
atr browser forwardGo forward in history
atr browser reloadReload current page

Page Management Commands

CommandDescription
atr browser new-page [url]Open new tab
atr browser list-pagesList all tabs
atr browser select-page <index>Switch to tab (0-based)
atr browser close-page <index>Close tab

Interaction Commands

CommandDescription
atr browser click <target> [--double]Click element (use --double for double-click)
atr browser fill <target> <value>Type into input field
atr browser hover <target>Hover over element
atr browser press-key <key>Press keyboard key (e.g., Enter, Tab, Control+A)
atr browser drag <from> <to>Drag element
atr browser wait <selector> [--timeout] [--visible]Wait for element to appear
atr browser scroll --selector "<sel>" [--y N] [--to-bottom]Scroll inside an element
atr browser download-images "<sel>" [--output-dir] [--fallback-screenshot]Download/screenshot images within elements
atr browser viewport [W H] [--preset mobile|tablet|desktop|wide]Get or set viewport size
atr browser batch [--file F] [--on-error stop|continue|retry:N]Execute multiple commands from stdin/file

Recording Commands

CommandDescription
atr browser record [--url URL] [-o FILE]Record browser interactions as a behavior test

Inspection Commands

CommandDescription
atr browser snapshot [--verbose]Get page elements with UIDs
atr browser screenshot --file [--full] [-s SELECTOR]Capture screenshot (saves to /tmp/)
atr browser screenshot --file --selector-all "<sel>"Screenshot all matching elements
atr browser computed-styles "<selector>" [--properties]Get computed CSS styles for single element
atr browser computed-styles --selector-all "<sel>"Get computed CSS styles for all matching elements
atr browser computed-styles-diff "<sel>" --against NCompare styles between pages
atr browser text "<selector>" [--flat|--links|--headings]Extract text content
atr browser font-check "<font-family>"Check if font is loaded and rendering
atr browser clean-snapshot "<selector>" [--depth N] [--max-length N]Get cleaned DOM subtree (no noise/tracking attrs)
atr browser computed-styles --selector "h1" --selector "p"Batch computed styles for multiple selectors
atr browser computed-styles-diff --selector "h1" --selector "p" --against 0Batch style diff with overall score
atr browser htmlGet page HTML
atr browser urlGet current URL
atr browser titleGet page title
atr browser eval <script>Execute JavaScript
atr browser ask "<question>"Ask a question about the current page

Screenshot Note: Use --file to save screenshots to /tmp/ with a timestamped filename (e.g., /tmp/atr-screenshot-20240105-103045.png). Add --full for full-page screenshots. Use --selector / -s to screenshot a specific element by CSS selector (e.g., -s "header", -s "#nav", -s "main > section:nth-child(2)"). Combine --selector with --full to capture an element's full scrollable height. Use --selector-all to screenshot every matching element as numbered PNGs — elements that fail or timeout are skipped (use --timeout <ms> to control per-element timeout, default 30s). Without --file, returns base64-encoded image data.

Debugging Commands

CommandDescription
atr browser console [--limit N]Get console messages (default: 50)
atr browser network [--limit N]Get network requests (default: 50)
atr browser errorsGet failed requests

Workflow Pattern

Follow this workflow for browser automation tasks:

  1. Ensure Server Running

    atr browser status || atr browser start
    
  2. Navigate to Target

    atr browser navigate https://example.com
    
  3. Inspect Page Elements

    atr browser snapshot
    

    This returns elements with unique IDs (UIDs) like e0, e1, etc.

  4. Interact with Elements Target elements by:

    • Text content: "Sign In"
    • UID from snapshot: e5
    • CSS selector: .submit-button
  5. Verify Results

    atr browser url
    atr browser title
    atr browser screenshot --file
    
  6. Cleanup When Done

    atr browser stop
    

Asking Questions About a Page

Use atr browser ask when you need specific information from a page without flooding your context with raw HTML or snapshot data. A lightweight sub-agent inspects the page using multiple tools and returns a concise text answer.

atr browser ask "What is the main heading on this page?"
atr browser ask "How many items are in the navigation menu?"
atr browser ask "Is there a login form on this page?"

Prefer ask over html or snapshot when:

  • You need a specific fact, not the full page structure
  • You want to keep your context clean for subsequent reasoning
  • The answer can be expressed as a short text response

Recording Browser Interactions

Use atr browser record to capture user interactions and output a .test.txt behavior test file. A floating overlay appears in the browser showing recorded steps in real time.

# Record interactions on a page
atr browser record --url https://example.com -o repro-steps.test.txt

# Record on the already-open page
atr browser record -o flow.test.txt

Stop recording with Ctrl+C in the terminal or the "Stop" button in the browser overlay.

The recorder captures clicks, form fills, keyboard shortcuts, navigation, scrolling, and select changes. It handles shadow DOM (web components) and generates stable CSS selectors. Password fields are automatically masked.

The output .test.txt file can be replayed with atr run --behavior repro-steps.test.txt.

Using JSON Output

Add --json flag for structured output when parsing is needed:

atr browser snapshot --json
atr browser list-pages --json
atr browser network --json

Element Targeting

The <target> parameter in click, fill, hover, and drag commands accepts:

  1. Element UID: e0, e5 (from snapshot output)
  2. Visible text: "Sign In", "Submit Form"
  3. aria-label: Elements with matching aria-label attribute
  4. data-testid: Elements with matching data-testid attribute
  5. CSS selector: #login-button, .nav-link, header, footer, nav, main > section, div.hero, li:nth-child(2)

Best practice: Use atr browser snapshot first to see available elements and their UIDs.

Keyboard Keys

For press-key command:

  • Named keys: Enter, Tab, Escape, Backspace
  • Modifiers: Control+a, Shift+Tab, Alt+Enter
  • Arrow keys: ArrowUp, ArrowDown, ArrowLeft, ArrowRight

Troubleshooting

Browser won't start:

atr browser status
rm ~/.atr/browser.state  # If stale state
atr browser start

Port already in use:

atr browser start --port 9334

Element not found:

atr browser snapshot --verbose --json

Additional Resources

For complete command reference with all flags, see references/commands-reference.md.

Companion skill — atr-browser controls Chromium for any in-page operation (navigate, click on web elements, fill forms, scrape). Load both when:

  • You need to drag a file from the OS file manager into a browser upload zone.
  • Authentication or a native dialog (1Password, system file picker) is part of the flow.
  • You're delegating a high-level task that touches both — atr computer ask "..." can shell out to atr browser ... and vice versa.

Computer daemon listens on port 9334; browser daemon on 9333. Both can run simultaneously.

ATR Desktop Computer-Use Skill

This skill provides cross-platform desktop control through ATR's computer daemon. Mouse, keyboard, screen capture, window management, app launch, and an in-process LLM agent (atr computer ask) — all driven from the same atr CLI that powers atr-browser.

Architecture

Claude Code --> atr CLI --> ATR Computer Daemon --> robotgo (X11 / macOS / Windows)

The daemon enforces a configurable safety countdown before every action so the user can intervene with Ctrl+C. Read-only operations (screenshot, position, list windows) skip the countdown.

When the daemon is started without --no-gui it also surfaces the countdown in a small native overlay so MCP/Claude-Code clients (which don't see the daemon's terminal) can still interrupt visually. On Linux the overlay uses zenity (full Cancel button — clicking it aborts the action with ErrAborted); when only notify-send is available it falls back to a one-shot desktop notification (visual only — abort still works via Ctrl+C in the daemon terminal). On macOS it uses osascript display notification; on Windows it uses a PowerShell toast. If no backend can initialize, the daemon logs a warning and continues with terminal-only countdown.

Linux: X11 only in v1. Wayland is not supported yet. GUI overlay: install zenity for an abortable dialog (apt install zenity); fall back to libnotify-bin for visual-only notifications.

Getting Started

Step 1: Check status and start if needed

atr computer status

If not running:

atr computer start                           # default: per-request 3s countdown
atr computer start --countdown-mode per-app  # countdown only on first action per app
atr computer start --countdown-mode off      # no countdown (trusted batch runs)
atr computer start --countdown 1             # 1-second countdown

State and logs:

  • State: ~/.atr/computer.state
  • Daemon log (where countdown messages appear): ~/.atr/computer.log

Step 2: Take a screenshot

atr computer screenshot --output /tmp/desk.png
atr computer screenshot --output /tmp/region.png --region 0,0,800,600

Step 3: Drive mouse and keyboard

atr computer click 800 50
atr computer click 800 50 --double
atr computer click 800 50 --button right
atr computer move 100 100
atr computer drag --from 200,300 --to 1200,400
atr computer scroll --dy -3
atr computer hover 500 500

atr computer type "Hello, world"
atr computer key enter
atr computer chord ctrl+shift+t

Step 4: Manage windows

atr computer --json window list                          # enumerate
atr computer window active                               # focused window
atr computer window focus --title "Firefox"              # by title or app name
atr computer window minimize --title "Slack"
atr computer window maximize --title "Code"
atr computer window restore --id 12345
atr computer window close --title "Calculator"
atr computer window move --id 12345 --to 100,100
atr computer window resize --id 12345 --size 800,600

Step 5: Launch / quit apps

atr computer app launch firefox
atr computer app launch xclock
atr computer app quit firefox

Step 6: Stop the daemon

atr computer stop

Safety countdown modes

ModeBehavior
per-requestDefault. Every gated action shows a countdown.
per-appFirst action against an app prompts; subsequent actions auto-approve.
offNo countdown. Explicit opt-in for trusted batches.

Aborting: press Ctrl+C in the daemon's terminal during a countdown. The action returns aborted by user.

Combined workflow with atr-browser

ATR can drive both desktop and browser in one flow. Example: end-to-end drag-and-drop between the OS file manager and a browser upload zone.

atr browser start
atr browser navigate https://example.com/upload

atr computer start --countdown-mode off
atr computer app launch nautilus
atr computer window focus --title "Files"
atr computer drag --from 200,300 --to 1200,400   # drag file into browser

atr browser screenshot --output /tmp/dnd.png

For browser-only workflows, see the atr-browser skill. For combined workflows, both daemons can run simultaneously on different ports (browser: 9333, computer: 9334).

Asking the agent (atr computer ask)

Instead of issuing one command at a time, you can describe a desktop task in plain language and let an agent loop drive the screenshots, clicks, and keystrokes:

atr computer ask "open xclock and tell me what time it shows"
atr computer ask "list the open windows that contain 'chrome' in the title"
atr computer ask --max-steps 30 --timeout 10m "open the GNOME calculator and compute 17 * 23"

The agent uses the LLM backend configured for ATR (backend: gemini-api | vertex-ai | vertex-claude | claude-cli in ~/.atr/config.yaml). Default model aliases:

BackendTierModel
gemini-api / vertex-aiflashgemini-3.7-flash
gemini-api / vertex-aiprogemini-3.1-pro-preview
vertex-claudesonnetclaude-sonnet-5
vertex-claudeopusclaude-opus-5

Override per-run with the global --model flag (e.g. atr --model pro computer ask "..."). With backend: claude-cli, the model flag is ignored and Claude CLI is invoked as a subprocess.

Tools the agent can call (curated subset):

  • Perception: computer_screenshot (image fed back to the LLM), computer_displays, computer_active_window, computer_list_windows, computer_position
  • Actuation: computer_click, computer_type, computer_press_key, computer_key_chord, computer_focus_window, computer_window_state (minimize/maximize/restore/close), computer_launch_app

Limits and gotchas:

  • The agent cannot type passwords — if a sudo / polkit prompt appears, it stops and reports the blocker.
  • Each screenshot sent to the LLM costs vision tokens; with --model pro a 10-step task is measurable spend.
  • The countdown gate inherits whatever the daemon was started with. For ask runs you usually want atr computer start --countdown-mode per-app so the first action against an app prompts and subsequent ones auto-approve.

Coordinates and multi-monitor

ATR uses root coordinates as the public API for clicks, moves, drags, and window positions. Root coords have a single bounding-box origin at (0, 0) covering all monitors — every visible coordinate is non-negative, matching xrandr and the X11 root window. atr computer --json window list and atr computer displays both return root coords.

For interactive use it's often easier to think in display-local pixels. Every mouse command accepts --display N, which interprets x and y as pixels relative to that display's top-left:

# Display 0 is primary (whose root origin may NOT be (0, 0) on multi-monitor setups)
atr computer click --display 0 100 200      # display-local
atr computer click 1540 200                  # root coords (equivalent on a setup where primary is at root (1440, 0))

atr computer screenshot --region X,Y,W,H is always display-local within the chosen --display.

A common workflow:

atr computer displays                                # see each display's root bounds
atr computer --json window list                      # find a window's root bounds
atr computer screenshot --display 0 --region X,Y,W,H # crop using display-local pixels
atr computer click --display 0 X Y                   # click the spot you cropped

Tips

  • Use atr computer --json window list to enumerate windows, then operate on a specific ID for stability.
  • For Linux, the per-app cache key is WM_CLASS (window class), so all Chrome windows share one approval.
  • Screenshots return PNG bytes; set --output to a local path the user expects.
  • For multi-monitor setups: prefer --display N with display-local pixels for new code; reach for absolute root coords only when relaying values from window list.

MCP servers 1

Sponsors

Details

Marketplace
imyousuf/agentic-test-runner
Category
Community