oaustegard avatar

ai-and-reasoning

AI model invocation, multi-agent orchestration, prompt distillation, and structured reasoning frameworks.

version
2.9.1
GitHub stars
404

Install

/plugin install ai-and-reasoning@oaustegard-claude-skills

Run it inside Claude Code after adding the marketplace with /plugin marketplace add oaustegard/claude-skills.

Contents

Discovered from source

Skills 20

Agent Routing — model, effort, and cascade selection

Current prices, and what changed in 2026-10

model$/MTok inoutcache readone Agent-tool spawn, small seeded repoone spawn, SWE-bench Verified
Haiku 5.5$0.10$0.50$0.01$0.005–0.011$0.024–0.037
Sonnet 5.5$2$10$0.20$0.17$0.23–0.35
Opus 5.5$4$20$0.20~$0.34 (2× Sonnet, not measured)—
Fable 5.1$10$50$0.25——

Haiku 5.5 is 20× under Sonnet 5.5 on every token class (Haiku 4.5 was 2–5×), and its prices hold for prompts up to 100K tokens ($0.50/$2.50 beyond). On the 14-repo seeded-bug battery it solved 11/14 at rung 1, which is the pass set Sonnet 5.5 at low gets. With the same-model informed retry it solved 14/14 for $0.12 in total, less than one Sonnet 5.5 spawn (oaustegard/experiments → temporal-routing-headroom, 2026-10-07). Where the Haiku 4.5 figures below say Haiku loses on cost, rerun the arithmetic at 5.5 prices before trusting them.

On real issues the picture holds with a smaller margin. Over 295 SWE-bench Verified tasks (django and sympy) Haiku 5.5 resolved 256 (86.8%) at rung 1 for $7.06 in total. On its 39 misses, rung 2 with the failing tests' output resolved 30 retrying Haiku ($1.43) and 34 escalating to Sonnet 5.5 ($13.53). Sonnet solved every task the Haiku retry did and 4 more (p = 0.125) (oaustegard/experiments → swe-ladder, 2026-10-09). Real repair spawns run longer than seeded ones, so a spawn costs 3–5× the small-repo figure on Haiku and 1.5–2× on Sonnet; the ratio between the tiers stays near 10×.

In Claude Code, price a subagent per spawn first. A fresh Agent-tool subagent writes a ~55K-token prefix (system prompt, tools, skills list) to the 5-minute cache on its first request. On short repair tasks that prefix outweighed everything the subagent produced: a Sonnet 5.5 spawn cost $0.17 input-side, 14× the $0.0121 output-only figure this skill used to quote per completed task. Same-model spawns in sequence share the cached prefix (reads cost 0.1×), so serialise spawns of one model when they can wait, and count spawns before counting output tokens. Every table further down prices output tokens only. That is still right for comparing tiers on long outputs, but it understates what a spawn costs.

The rule that decides everything

Cost is output tokens × output price, plus the spawn's prefix. Prices span ~5× across tiers. Token counts span up to 7× within a single tier depending on task shape. The shape therefore decides more than the tier does, and routing on the per-token discount gets the answer backwards.

Measured 2026-08-17, 14 spec-dense Python modules graded by hidden tests, all tiers at equal quality where noted:

armtok/taskpass$/taskvs opus
haiku-solo20,05114/14$0.10071.30×
haiku + concision13,34212/14$0.06721.01×
opus-solo3,00114/14$0.07741.00×
sonnet-base4,68714/14$0.04780.62×
sonnet + concision2,95113/14$0.03050.42×
sonnet cascade (below)—14/14$0.03150.41×

Haiku 4.5 was 5× cheaper per token and cost 30% more per solved task than Opus, because it emitted 6.7× the tokens. Prices then: Haiku 4.5 $1/$5, Sonnet 5 $2/$10, Opus 5 $5/$25 per MTok. At Haiku 5.5's $0.50 output price the same 20,051 tokens would cost $0.010 a task, a fifth of Sonnet's $0.048. Whether Haiku 5.5 is as verbose as 4.5 on generation is unmeasured. It would have to emit about 95K tokens a task to lose on output cost.

Two questions before spawning

  1. Is the output short or long? Short = a schema instance, a label, an answer, a small patch. Long = a module, a document, a plan, a review.
  2. Is it mechanically checkable, or does it need judgment?
short outputlong output
checkablehaiku + verifierhaiku + verifier, informed retry on haiku, then sonnet @ high
judgmentsonnet @ mediumsonnet @ high, or opus when the work must be discovered

On Haiku 4.5, output length was the discriminator, because it is what the verbosity multiplier multiplies: Haiku 4.5's premium was invisible on a 200-token JSON object and ruinous on a 700-token module that cost it 13,000 tokens of thinking. At 5.5 prices verbosity no longer decides it, so the checkable column now starts on Haiku at any output length. The long-output cell is projected from price and from the repair battery, and has not been measured on generation.

The one measurement that has separated Haiku 5.5 from Sonnet 5.5 is judgment without a stated rule: triage of real private email, 84.7% against 91.7% (McNemar p = 0.0005), in the same external study whose two synthetic benchmarks with stated rules did not separate them. That the missing rule is the cause is the study's hypothesis, not a measurement. Until someone tests it, treat "the prompt states the rule and the answer is checkable" as the condition for the checkable row, and route rule-less sorting, triage and prioritisation to the judgment row.

The judgment row splits on whether the prompt names its deliverable or leaves the work to be found. Sonnet 5.5 finishes a named deliverable well. Given an audit-and-update goal, it listed findings, raised doubts and stopped to ask, and Oskar moved that session to Opus 5.5 mid-task (2026-10-07; one session, an observation, not a measurement).

Routing table

Task shapeModelEffortVerify with
Extraction, classification, format transforms, schema-bound outputhaikun/aschema / spot-check
Closed-form computation, state tracking, multi-hop lookuphaikun/adeterministic check
Constraint-bound generation (exact counts, required tokens, lipograms)haikun/amechanical checker
Bulk scans/greps, per-file summaries, fan-out readshaikun/asample audit
Code edits with tests availablehaiku → haiku informed retry → sonnet on a second failuresessionrun the tests (the orchestrator's, not the worker's)
Code generation from a spec; any long structured artifacthaiku → sonnet @ high on a second failuremediumrun the tests (Haiku 5.5 unmeasured here)
Judging / scoring another model's outputsonnet+medium— (judge ≠ worker)
Ambiguity resolution, novel synthesis, architecture, tastesonnet/opushighhuman or panel
Long-horizon multi-step agentic work, cross-file reasoningsonnet/opushigh/xhighmilestone checks
Improving a program against an automatic scorer, small budgethaiku swarm: proposers + writer (evolving-with-swarms)proposers low, writer mediumthe scorer, plus a bound it cannot exceed
Same, when the last 0.1% matters and $3+ and wall-clock time are both on the tablesonnet loopper problem: measure medium and highthe scorer

Haiku holds the top four rows on merit: 240/240 measured across nested modular arithmetic, 30-hop chains, 25-operation state tracking, trap-laden word math, and 5-constraint generation, some with CoT suppressed (references/calibration-2026-07-15.md). Those were Haiku 4.5 runs, which requested effort: low and ignored it, so the column says n/a. Haiku 5.5 does take effort (see below); until it is measured, leave it at its default. Do not up-tier short checkable work "to be safe"; there is no measured benefit and it costs 3–5×. The burden of proof is on routing up.

An independent run on 5.5 agrees (drknowhow/experiments → haiku-delegation-bench-hard, 2026-10-18, pre-registered, 60 items, three reps). Across log joins, multi-hop repo navigation, subtle review bugs, multi-file edits, threads where later messages supersede earlier values, and unanswerable questions with near-miss distractors, Haiku 5.5 scored 0.968 against Sonnet 5.5's 0.994 and Opus 5.5's 1.000, for about 1/16 of Sonnet's cost. The gap is 0.026 (95% CI 0.006–0.055); five of six categories routed to plain Haiku and the sixth to Haiku with a check. Every item stated its rule and had one checkable answer, which is the condition the top rows assume.

Haiku 4.5 lost the generation rows on cost alone, not capability: it scored 14/14 on the same suite Opus swept. At 5.5 prices that cost argument is gone (see the top of this file), so the generation rows now start on Haiku behind a test verifier, pending a generation-suite run on 5.5.

Search against a scorer: the Haiku swarm wins small budgets, Sonnet catches up by $3

Measured 2026-11-09 (oaustegard/experiments → haiku-swarm-poc, 48 runs, three seeds per arm, $3 cap or 60 minutes). Haiku 5.5 swarms (4 swarms per round, 8 proposers each at low writing 45-word ideas, one writer at medium per swarm) against stock OpenEvolve driving Sonnet 5.5, on three construction problems with exact scorers. Dollars until a program first scored 99% of the reference:

problemHaiku swarmSonnet loop, best effortSonnet @ low
circle packing, n=26$0.013–0.019$0.32–0.44 (medium)$0.36–0.37
hexagon packing, n=11$0.035–0.060$0.53–0.58 (high)$2.39–2.55, one seed never
Heilbronn triangle, n=11$0.09–0.50$0.77–1.02 (medium)$0.46–1.17, one seed never
  • At $0.25 every swarm seed beat its paired Sonnet seed on all three problems.
  • Given $3, Sonnet reached the same finals in 7–42 minutes; the swarm spends about $1 an hour, so it is the cheap route, not the fast one.
  • Sonnet 5.5's effort is problem-dependent in search. No difference on circle packing; medium alone reached the reference on all three Heilbronn seeds; on hexagon low never reached 0.999 while high did on every seed. Do not carry an effort level from one search problem to the next without a seed or two of evidence.
  • The Haiku writer at low is a gamble: better on the easy problem, worse on both harder ones.
  • Proposers took 30–39% of the swarm's spend, and on one circle-packing seed the writer alone nearly tied. Run an N=0 control on a new problem.

Effort is model-specific — verify per model before reusing a level

Measured 2026-08-17 via per-message output_tokens_details.thinking_tokens, thinking as a share of output on identical prompts:

modellowmedium
Sonnet 52.9%47.7% (61.7% without concision)
Haiku 4.588–91%88–91%

low was a near kill-switch on Sonnet 5: it dropped 14/14 → 10/14. Sonnet 5.5 is different. On the 14-repo seeded-bug battery (2026-09-29, two replicates, temporal-routing-headroom):

armsolved (r1, r2)output tokens, 14 tasks$/completed
Sonnet 5 @ low (2026-09-03)9/1416,779—
Sonnet 5.5 @ low10, 1111,466$0.0104
Sonnet 5.5 @ medium11, 1210,853$0.0090
Sonnet 5.5 @ high12, 1214,488$0.0121
Opus 5.5 @ high11, 1318,464$0.0284
Opus 5 @ high (2026-09-03)10/1455,674$0.1392
  • low and medium cost the same on Sonnet 5.5 for this work, and low missed only trap tasks. Claude Code's <reasoning_effort> tag agrees: 4 at low, 5 at medium.
  • high bought the paired traps: lru_ttl and wrap_text went from 0 of 2 at low to 2 of 2 and 1 of 2.
  • Sonnet 5.5 @ high matched Opus 5.5 @ high at 24 of 28 each, for 0.43× the cost per completed task. Seeded repair does not separate tiers, so this says the two are close on this family and nothing about harder work.
  • The token counts are one replicate (budget deltas from r2); thinking share could not be measured, because Claude Code omits thinking text.

Effort does not reach Haiku 4.5. The Messages API rejects effort on it; Claude Code's Workflow agent() accepts the option and drops it (0 of 12 Haiku probes carried an effort tag or a transcript effort field). The identical 88–91% thinking share at low and medium above is that fact seen from the token side, and the ~26% token drop once attributed to low sits inside the 23% run-to-run gap measured below. Tune Haiku with the prompt; the routing table lists its effort as n/a.

Haiku 5.5 takes effort. The API accepts low through max on it, defaults to medium, and thinks adaptively by default; budget_tokens and non-default temperature are 400s. Through claude -p its thinking cannot be switched off: MAX_THINKING_TOKENS=0 and alwaysThinkingEnabled: false are ignored, so --effort low is the floor; short-idea calls at low thought 0 tokens 65% of the time (mean 95, 2026-11-09). Whether Claude Code's Workflow agent({effort}) reaches a Haiku 5.5 subagent, and what low does to its pass rate, are both unmeasured. The 2026-10-07 ladder ran it at the session's level through the Agent tool.

  • Tune Sonnet with the prompt as well as the knob. On Sonnet 5, medium was the working floor and low overshot into thinking-off; on Sonnet 5.5 low is a usable rung 1 and high is the rung that caught the traps.
  • Buy depth only for judgment-heavy roles; drop triage and formatting roles to low without touching the expensive role's budget.

Setting effort — which channel reaches which agent

Measured in Claude Code on 2026-09-28 (muninn.austegard.com/blog/effort-levels-in-claude-code-subagents.html):

channelreachesnotes
Workflow agent(prompt, {effort})one subagent24/24 Sonnet and Opus probes ran at the requested level while the session sat elsewhere. The only way a parent sets a subagent's effort.
Agent tool, SendMessagenothing directlyNo effort parameter on the call. A plain subagent runs at the session's level when it starts or resumes.
Agent tool with a named agent typeone subagentMeasured 2026-10-12 (tip from Craig Rungie): an agent definition with effort: in its frontmatter (.claude/agents/<name>.md, or --agents JSON) sets the effort of every Agent-tool call to that subagent_type; the subagent's transcript records it. With none set the request ran at medium. Sonnet 5.5 thinking on a reasoning puzzle, two runs each: low 481/666, medium 567/624, high 676/703 tokens. Keep one agent file per model-effort pair you use, e.g. sonnet-low.md, and pass subagent_type. Agent files load at session start.
claude -p --effort LEVELone headless childMeasured 2026-10-11 on Sonnet 5.5: low gave 0 thinking tokens, high gave 112–197 on the same prompt (2 runs each). Needs no workflow opt-in; dispatching-subagents runs these children in bulk.
Session setting (/effort)main loop, and every subagent at its next turnA resumed subagent picks up the new level. The model cannot change this itself; it recommends a level and the human sets it.
Managed Agentsagent configSet on the agent, not per session: an effort inside a per-session model override is silently ignored. If the create response echoes effort: None, the org's beta header (managed-agents-2026-04-01) doesn't carry the feature and the field was dropped.

A Workflow run needs the user's opt-in (ultracode, an explicit request, or a user-invoked skill that names it). Without one, a subagent spawned through the Agent tool gets the session's level, and the routing table's effort column is advisory, unless the call names an agent type defined with effort: (row above) or the task can run as a headless claude -p child, which takes --effort directly.

This skill picks the model and effort. Which mechanism launches the child (Agent tool, Workflow, or headless children), and what the launching costs the parent, is dispatching-subagents.

The concision lever, and its limit

Adding one instruction — this is routine work; do not deliberate at length, do not enumerate test cases or weigh alternative designs; write it directly — cut output 37% on Sonnet and 27% on Haiku, at no quality cost. It composes with effort. Use it on every long-output generation spawn.

It does not reach work whose output is small. On agentic bug repair — a patch plus a paragraph — the same instruction cut Sonnet output 2.9%, at no quality change either way. The lever acts on deliberation the model would have written down, so a task that emits little has little to cut. Measure before carrying it to a new task family; "every long-output generation spawn" is the scope, and repair work is not in it.

Then stop. Thinking below a model's natural level is load-bearing, and cutting into it buys tokens with correctness:

  • An engineered suppression prompt (positive framing, bounded budget, n-shot exemplar) cut Haiku 35% and halved its pass rate, 8/9 → 4/9. Within that arm, passing runs thought 1.9× more than failing runs.
  • Sonnet at low (2.9% thinking) fell 14/14 → 10/14.
  • Priced per passing result the suppressed arms were more expensive: 22,143 tokens vs 17,126 for the un-engineered prompt.

A targeted checklist ("enumerate the spec's rejection rules first") helps only when it names the actual failure mode: it took one validation-heavy task from 15,220 to 9,634 tokens at equal quality, and took a semantics-heavy task from 3/3 to 0/3. Misnaming the failure mode is worse than not intervening.

Cascade

Precondition, checked first: is the cheap tier actually cheaper per task? The first rung is never free, so a cascade pays only when the cheap tier's measured cost per completed task is below the destination's. Verbosity can erase a price discount outright — Haiku 4.5 at $0.067/task against Sonnet's $0.031 made haiku → sonnet worse than Sonnet alone regardless of p_fail: the attempt cost 2× the destination's entire job. Compute this before designing the ladder.

Haiku 5.5 passes this precondition. A Haiku 5.5 rung-1 spawn cost $0.0077 against $0.17 for a Sonnet 5.5 spawn, and it solved the same 11 of 14 repairs Sonnet 5.5 solves at low. A Haiku first rung now pays even when it fails often. A failed attempt is about 5% of the Sonnet job it precedes.

Second precondition: no verifier ⇒ no cascade. Route by the table instead; silent cheap-tier errors compound with nothing to catch them.

The verifier's holder makes the escalation call. Never the worker. A subagent asked whether it finished says yes: across 58 graded runs carrying an explicit "did you finish" field, 58 said yes and 44 had passed the held-out suite. Every one of the 14 failures self-reported success. The workers were not lying — they had passed the tests they could see, and those tests stay satisfiable while the task is unfinished. "Try it, and ask for help if you fail" therefore fails on exactly the tasks that need escalation. Put the decision wherever the stronger check lives; in a fan-out that is the orchestrator.

The shape that worked (measured, 14/14 at 0.41× Opus):

result = sonnet(task, effort=low, concise)          # rung 1: 10/14, $0.0155
if verify(result) fails:
    result = sonnet(task, effort=medium, concise,   # rung 2: fixed 12/12
                    prior=result, failure=test_output)

Haiku 5.5 ladder (measured 2026-10-07, 14 seeded-bug repos, one replicate):

result = haiku(task)                                   # rung 1: 11/14, $0.108 for 14 spawns
if verify(result) fails:                               # the orchestrator's hidden suite
    result = haiku(task, prior=diff, failure=output)   # rung 2: rescued 3/3, $0.015

14/14 for $0.123. Sonnet 5.5 at rung 2 rescued the same 3/3 for $0.511, 35× the per-spawn cost, and bought nothing more. All three rung-1 misses named the second defect in their own summaries and left it as "outside the reported bug". The informed retry supplies the evidence that the shortcut was wrong; the tier adds nothing to that.

On SWE-bench Verified the tier adds a little (measured 2026-10-09, 295 tasks, one sample per arm):

on Haiku's 39 rung-1 missesresolvedrung cost
haiku, issue text only (blind re-roll)8/39$1.05
sonnet, issue text only17/39$10.34
haiku, prior diff + failing tests' output30/39$1.43
sonnet, prior diff + failing tests' output34/39$13.53

Over the whole ladder, haiku → haiku resolved 286/295 for $8.49 and haiku → sonnet 290/295 for $20.59, so Sonnet's 4 extra tasks cost about $3 each. Sonnet's set contained the Haiku retry's, so Sonnet belongs at the third rung: haiku, haiku informed, then sonnet only on what the retry still fails. That sends Sonnet 9 tasks instead of 39. The three-rung order is a projection, not a measurement: the Sonnet arm started from the rung-1 diff, not the rung-2 one.

The failing tests' output in that run was SWE-bench's hidden suite, an oracle no deployed ladder has. Read the informed-retry rows as the ceiling for whatever verifier the orchestrator holds; the Haiku-versus-Sonnet gap between them is a fair comparison, because both arms got identical input.

Rung 2 is the same model one effort step up. A tier jump is the exception you justify. Measured twice. On a second battery (14 seeded-bug repos, 2026-09-03) rung 2 ran from an identical failed attempt at both settings: sonnet @ medium and opus @ high rescued the same 4 of 5 tasks and both missed the same fifth, at 11,691 against 32,504 output tokens. Composed over the same rung 1, the same-model cascade cost 0.31× always-opus and the tier jump 0.76×. The tier jump costs 2.5× and buys nothing.

Those rungs are Sonnet 5. On Sonnet 5.5, low and medium solved the same tasks at the same cost, so the step that changes anything is low → high: high is where the paired traps started getting caught (see the effort table above). The cascade itself has not been run on 5.5.

A cascade can beat the frontier solo arm on correctness, not only on cost. In that run the sonnet→sonnet cascade solved 13/14 where always-opus solved 10/14. opus starting from the issue text fell into the same stop-early trap as sonnet on three tasks; opus starting from the failed patch and the failing assertions fixed all three.

How to run rung 2 in Claude Code. For a subagent, launch a fresh Workflow agent() one effort step up and pass it the diff and the test output; Agent and SendMessage cannot change a subagent's effort. For the main loop, recommend the next level and let the human set /effort; the session keeps its cache and carries on.

Caching pushes the same way. Caches are model-scoped, so a tier jump discards rung 1's prefix. An effort change does not: in Claude Code on 2026-09-28, five subagent resumes across a level change (Low→Medium, High→Extra, Extra→Max) all read their full prefix from cache, and the parent read 97,839 tokens from cache on its first request after a change. On the API the per-message effort message ({"role": "system", "content": [], "output_config": {"effort": …}}, beta mid-conversation-output-config-2026-07-01) keeps the cache on Fable 5.1, Mythos 5.1, Opus 5, Opus 5.5 and Sonnet 5.5 with thinking on; a change to the top-level effort still invalidates the messages cache.

The TTL is what misses. Every miss in that experiment followed a gap longer than five minutes. Subagents write only to the 5-minute tier and the parent to the 1-hour tier, so a subagent resumed after five idle minutes rebuilds its whole prefix (about 54k tokens for a bare general-purpose agent). Resume a subagent for a quick retry; after a longer gap, start a fresh one, since same-model agents share a cached prefix (32 of 36 fresh probes read from cache on their first request).

Every figure in this skill prices output tokens only; input and cache effects sit outside its cost model.

Carry the prior attempt and the raw failure output into the retry. Informed retry fixed 12/12; a blind re-attempt fixed 9/12 and failed one task identically across all three replicates — a systematic blind spot re-rolling never escapes. On SWE-bench Verified the gap is wider: a Haiku retry with the failing tests' output fixed 30/39 of Haiku's misses, a blind Haiku re-roll 8/39, and Sonnet from the issue text alone 17/39. Neither blind arm solved a task its informed counterpart missed. The failure output moved 22 tasks for Haiku; switching to Sonnet moved 9 without it and 4 with it. The extra input averaged 866 tokens, 5.9% of the retry's cost. Input is 1/5 the price of output, so context is nearly free relative to thinking.

The artifacts, not the prior model's account of itself. Adding rung 1's stated diagnosis on top of the patch and the assertions did nothing: 13/15 against 12/15 over three replicates, 1% fewer output tokens, and the whole difference was one replicate of one unstable task. It does not help and it does not anchor — SWE-Router (arXiv 2607.00053) restarts its strong model from the task description to avoid an anchoring effect that is not there. Pass the diff and the test output; skip the rationale.

Don't pay a frontier model to write guidance. An Opus diagnosis added zero over raw test output in two independent tests, at ~$0.15/task. The failing test already says what the orchestrator would say.

Verify content, not envelope. Strip fences, preambles, and trailing commentary before checking; hard-fail only on semantic content and log envelope deviations as soft. Two Haiku runs returned 7/7 and 6/6 correct fields while both wrapping output in a markdown fence the prompt forbade — a verifier keying on raw.startswith('{') would have escalated both for zero content error. Spurious escalation is a cascade failure mode, not a safety margin.

Check sentinel answers for premise rejection. When the contract says to return a sentinel such as NOT_FOUND for an unanswerable question, Haiku 5.5 sometimes answers in prose that rejects the question's premise instead ("A canary never pages at any p99 latency"). In the external benchmark above, that one item produced all three of Haiku's inventions and Sonnet's single one; no reply stated a fact the source lacked. A verifier that tests answer == "NOT_FOUND" scores these as confident answers, and code that branches on the sentinel takes the wrong branch. Say in the prompt that a false premise also gets the sentinel, and have the verifier send any non-sentinel answer to an unanswerable-looking question to a check rather than straight to escalation.

Judgment tasks fail in a shape checkers miss. Asked to rebut a stakeholder's "spend is down 66%" off a partial-month extract, Haiku killed the bad conclusion but normalized per calendar day across a 40%-weekend window and missed a model-mix confound — while passing every mechanical check available (word count, prose form, internal arithmetic consistency). The cheap tier fails as right headline, missed confound. This is why judgment rows route up rather than cascade.

Context handoff — routing picks the tier; the prompt carries the context

Subagents inherit nothing: not the conversation, not loaded skills, not the existence of artifacts already on disk. Every index, scan output, artifact path, or tool recipe must be serialized into the prompt (or a file the prompt points at). Otherwise the agent falls back to blind rediscovery and the tier premium is spent on crawling. A Sonnet with no handoff wastes more than a Haiku with a good procedure.

Per spawn: (1) artifact paths + how to query them, (2) tool commands verbatim, interpreter path included — subagents don't know your venv, (3) explicit anti-patterns ("no ls/glob discovery"), (4) an output spec.

Evidence: 2026-07-16, four Sonnet Explore agents launched onto a 2,300-file repo without the handoff opened with ls crawls despite a full tree-sitter symbol index sitting on disk; relaunched with per-agent index slices, the verbatim command, and anti-crawl rules, discovery cost dropped to ~zero.

To convert a judgment-shaped task into a cheap-tier-executable one (explicit procedures, n-shot examples), use the sibling down-skilling skill. This skill decides the routing; that one engineers the prompt.

Shared-prefix caching cuts the fan-out multiplier (unmeasured, conditional). When N subagents share a byte-stable prefix — the fixed handoff, not the per-agent slices — prefix caching can pull that portion toward a read-discount rate where the orchestration surface exposes it. Keep per-agent content at the tail. Verify your surface caches subagent prefixes before relying on it.

Loop discipline

Never blind-loop. Re-applying a prompt to a model's own output is the identity at best — an LLM call already unrolls its reasoning internally — and regression-then-freeze at worst: a re-looped haiku broke its own middle line on iteration 2 and froze on the broken text for every iteration after.

  1. Loop only with an out-of-band evaluator — ground truth, mechanical checker, or an up-tier judge scoring every iteration.
  2. Select, don't trust the last: final = argmax_r eval(answer_r).
  3. Stop on first regression. If eval(r) < eval(r-1), stop; loops froze on degraded output rather than recovering.
  4. Loop for diversity, not depth. Vary the angle per iteration; identical re-application converges instantly.
  5. "Improve this" with no headroom is the danger zone. It pressures the model to change something; without a selector, that change ships.

Judge rules

  • Judge model ≠ worker model; judge at least one tier up. Same-model self-assessment is untested.
  • Prefer mechanical checkers wherever a spec can be executed (counts, schemas, tests, regex): free, deterministic, zero judge tokens.
  • Judges are for rubric quality, not arithmetic — don't ask a model to verify a sum a Python one-liner can check.

Escalation triggers (route up despite the table)

  • The prompt names a goal rather than a deliverable (audit-and-update, follow-through across repos, "find what's outdated and fix it"). Sonnet 5.5 tends to stop and ask where Opus 5.5 carries on (observed 2026-10-07).

  • The verifier fails twice at the same tier. Route up for capability, not for thoroughness — opus @ high fell into the same stop-early trap as sonnet @ low on three of four tasks built to reward a second look. A verifier catches that; a bigger model does not.

  • The task requires weighing trade-offs with no checkable ground truth.

  • Output ships verbatim to a human without review.

  • The subagent must plan its own multi-step tool strategy over many turns.

  • The task spans multiple sources that may disagree and must be reconciled.

Not a trigger: a tail of failures the harness made. In a long test-fixing swarm, tests parked after failed attempts look like a capability gap. On 2026-10-12 (haiku-cc, 54 clusters over three runs) cold Sonnet 5.5 solved 2 clusters cold Haiku 5.5 did not, at about 7x the cost per solve, and Haiku's own retry ladder (notes, feature budget, effort step) fixed both later; the tail the swarm abandoned was harness-made and Haiku solved it cold once un-parked. Fix the harness and retry before escalating the tier. Procedure and measurements: swarming-to-green ("Which model").

Observing the fan-out — you can't govern what you can't watch

Stop-on-regression and "verifier failed twice" assume you can see a subagent's work while it runs. By default you can't: the session stream previews only the primary thread, and a subagent's output lands only after its whole turn buffers.

Attach one stream per thread. Read the session stream for the coordinator; on every session.thread_created (carrying session_thread_id and agent_name), attach a watcher to GET /v1/sessions/{id}/threads/{thread_id}/stream with event_deltas.

  • Preview is a scratch buffer; the buffered event is the record. Deltas are best-effort and shed under load, so concatenated deltas are a prefix of the final text. Reconcile by a single replace when the buffered agent.message arrives; the SDK's accumulate_managed_agents_event folds start/delta/record into one snapshot. One accumulator per connection. (The same trap appears offline: per-message usage records in transcripts include streaming partials — take the max per message id, or you undercount tokens ~2×.)
  • No replay. A stream opened after a request started gets no deltas for it, and reconnects never replay — attach on thread_created or miss the first response.
  • Coordination events live on the primary thread — session.thread_created, agent.thread_message_sent, agent.thread_message_received. Child tool calls cross-posted to the primary carry session_thread_id; skip them.
  • Terminate cleanly. Watchers exit on session.thread_status_idle; the main loop on session.status_idle — print the stop reason when it isn't end_turn, and break on terminated-status events.

Operational, not calibrated. Source: Anthropic Managed Agents notebook CMA_watch_subagents_live (beta managed-agents-2026-04-01); contract in events and streaming.

Measure before trusting this

Everything above is measured on four batteries: a 300-call deterministic calibration (references/calibration-2026-07-15.md), a 14-task hidden-test coding suite (2026-08-17, ~190 subagent runs), and a 14-repo seeded-bug agentic battery (2026-09-03, ~120 subagent runs, oaustegard/experiments → temporal-routing-headroom) that measured the cascade rungs, the escalation signal, and the tier gap against each other, rerun on 2026-10-07 with Haiku 5.5 at both rungs and priced per spawn from the subagent transcripts (harness/subagent_cost.py there), and 295 SWE-bench Verified django and sympy tasks graded Docker-free (2026-10-09, ~530 subagent runs, oaustegard/experiments → swe-ladder). Re-measure when:

  • A model or price revision lands. Both the verbosity multipliers and the cost table above invert on either. Sonnet 5.5 (2026-09-29) and Haiku 5.5 (2026-10, 10× cheaper than Haiku 4.5) are such revisions. Haiku 5.5 is measured here on the seeded-bug battery and SWE-bench Verified, and externally on six stated-rule delegation categories (haiku-delegation-bench-hard); its verbosity on the generation suite, and what effort does to it, are not. That study found Haiku 5.5 thinking about twice as many tokens per call as Sonnet 5.5 (540 against 261) on short structured answers while still finishing faster. Sonnet 5.5's effort levels are measured above on the seeded-bug battery only; the other Sonnet figures in this skill are Sonnet 5 data, and the verbosity multipliers and the generation-suite costs have not been re-run on the 5.5 generation.
  • The task family is off all four batteries. No deterministic task has made Haiku fail on correctness yet, so the capability cliff is past what's been probed. Seeded-bug repair in a small module is now measured as not tier-separating: three probe shapes aimed at thoroughness, at ambiguity the tests underdetermine, and at a repo with no test suite at all, and sonnet @ low solved all six cells against opus @ high. Real repair separates the tiers modestly: on SWE-bench Verified, Sonnet 5.5 alone is estimated at 92.5% against Haiku 5.5's 86.8%, with all of the gap on the tasks Haiku misses (Sonnet solved 50/50 sampled Haiku successes).
  • Output length differs materially from what was measured. The whole cost model keys on token volume; a 10× longer artifact re-opens the tier question.
  • You need pass-rate differences of 1–2 tasks. Run-to-run variance swamps them: two runs of the same model on the same 14 tasks produced disjoint failure sets and a 23% token gap. Token deltas are trustworthy; small pass-rate deltas are not.

Challenging — Adversarial Review

Adversarial review before shipping. Three paths, each with distinct trade-offs:

  • Subagent path (Claude Code, primary). Native sub-Claude via the Task tool — zero API keys, fresh context window, same model. Best when available.
  • External API path (claude.ai, Codex, headless). Gemini (cross-model + cross-context) or Anthropic API (cross-context). Costs an incremental API call but gives genuine outside perspective.
  • Self path (any environment). The caller assistant inhabits the adversary persona in a dedicated response. Zero cost, retains full subject-matter context from the conversation. Weaker at catching same-session confabulations than fresh-context adversaries, stronger at catching local-convention and factual errors the artifact glosses over. Not a strict downgrade — a different failure-mode profile.

The adversary='auto' resolution (default) picks gemini → claude → self based on available credentials. Callers in Claude Code still use prepare() + Task tool explicitly (subagent is strictly better than self in that environment, and auto-detection of Claude Code is brittle).

Inspired by VDD (dollspace.gay) and Grainulation's anti-rationalization patterns. The drill helper adopts the 5 Whys pattern from Tim Kellogg's open-strix writeup. The self-path persona-inhabitation move is kin to generative-thinking's inversion — commit to the mode before evaluating.

Profiles

Pick the profile matching your artifact. Read only the profile you need — each is self-contained with persona, anti-rationalization table, evaluation criteria, and adversary system prompt.

ProfileUse ForIteration strategyFile
proseBlog posts, essays, articles — generic prose competenceparallel replayreferences/prose.md
prose-registerProse with a named voice signature — fidelity checkparallel replayreferences/prose-register.md
analysisResearch briefs, comparisons, synthesisparallel replayreferences/analysis.md
codeScripts, implementations, PRsparallel replayreferences/code.md
recommendationTechnical decisions, architecture choicesparallel replayreferences/recommendation.md
philosophersArguments, position pieces, design rationales — conceptual-layer auditparallel replayreferences/philosophers.md
drill5 Whys on one finding from a reviewsequential deepenreferences/drill.md

One engine, one surface, two iteration strategies. Review profiles iterate in parallel replay — each pass independent, novelty tracked for confabulation. Drill iterates in sequential deepen — each pass takes the chain so far and produces one more why-level until bedrock or max depth, followed by a synthesis pass that extracts root causes.

philosophers and analysis are complements on argument-heavy artifacts: analysis audits the evidentiary layer (source independence, cherry-picking, calibration); philosophers audits the conceptual layer via Socratic elenchus and Aristotelian division (definitional stability, inference validity, is/ought slippage, taxonomy-before-verdict). An artifact can pass one and fail the other.

prose and prose-register are siblings, not redundant. prose evaluates generic prose competence (claims, logic, structure, performed insight) and is explicitly told not to comment on style. prose-register evaluates fidelity to a named voice signature passed via voice=... (positive markers + anti-patterns) and ignores generic competence. Run both passes on voiced prose — they catch different failure modes.

Usage — Claude Code (subagent path, primary)

Two-step protocol: a Python helper builds the prompt, you spawn a subagent via the Task tool, then a parser turns its response into structured findings.

import sys
sys.path.insert(0, '/mnt/skills/user/challenging/scripts')
from challenger import prepare, parse_response

job = prepare(
    artifact=open('/home/claude/draft.md').read(),
    profile='prose',
    context='Blog post about RAG scaling laws',
)

Then invoke the Task tool — subagent_type='general-purpose', prompt=job['prompt'], description='Adversarial review (prose)'. The subagent runs in a fresh context, applies the persona, and returns a JSON message. Pass that message text to the parser:

result = parse_response(subagent_text)
print(result['verdict'])    # SHIP | REVISE | RETHINK
print(result['findings'])   # List of specific issues
print(result['strengths'])  # What to preserve

Why subagents (not the API): no key, no network dependency, fresh context, and the same Claude that's reviewing your work is reviewing it again with no prior bias — but in a clean window. For cross-model diversity (genuinely different blind spots), use Gemini below.

Voiced prose — prose-register (subagent path)

For prose written in a named voice, prose will miss register-specific failure modes by design (its persona is instructed not to comment on style). Use the prose-register profile with a voice=... signature instead — or in addition, since the two profiles target orthogonal failure modes.

voice = """
Corvid voice signature.
Positive markers: short sentences when certainty is high, dry observations,
lead with the answer then context, no throat-clearing.
Anti-patterns: heroic-narrator framing, drama-line-breaks ("Then the part that
almost killed it." floating alone), cliche tells ("shot itself in the foot",
"footgun"), time-scale inflation ("a month ago" when it was yesterday),
performed-significance setups ("What's interesting about this is..."),
RTFM-performed-as-revelation, precious sentimentality in closings.
"""

job = prepare(
    artifact=open('/home/claude/draft.md').read(),
    profile='prose-register',
    context='Blog post for muninn.austegard.com',
    voice=voice,
)
# Task tool → parse_response() as usual.

voice is required for prose-register and rejected for every other profile — pass the signature once, fail loud if it leaks to the wrong call. The richer the signature (with both positive markers and explicit anti-patterns), the more precise the review. A thin signature returns verdict: RETHINK with a finding describing what to sharpen rather than a confident review against an ambiguous spec.

Drill — 5 Whys on a systemic finding (subagent path)

Drill uses the same prepare() / Task / parse_response() protocol as review profiles, but you run the loop yourself — each pass takes the chain so far and produces exactly ONE new {why, because}. When the adversary sets bedrock=true (or you hit max depth), run a final synthesis pass.

from challenger import prepare, parse_response

suspect = next(f for f in result['findings'] if f['severity'] in ('high', 'critical'))
artifact = open('/home/claude/draft.md').read()
ctx = 'Blog post about RAG scaling laws'

chain = []
for depth in range(1, 6):
    job = prepare(artifact, 'drill', context=ctx, finding=suspect, chain=chain)
    # Task tool: subagent_type='general-purpose', prompt=job['prompt']
    step = parse_response(subagent_text)   # {why, because, bedrock, reasoning}
    chain.append({'why': step['why'], 'because': step['because']})
    if step.get('bedrock'):
        break

# Final synthesis pass over the completed chain
job = prepare(artifact, 'drill', context=ctx, finding=suspect, chain=chain, synthesize=True)
# Task tool again, then:
diagnosis = parse_response(subagent_text)
print(diagnosis['chain'])        # [{why, because}, ...]
print(diagnosis['root_causes'])  # usually 3-4 distinct systemic issues
print(diagnosis['direction'])    # compass heading for the process fix

finding accepts either a dict from parse_response() or a free-text description. Patches fix the instance; drills fix the class. Why sequential, not parallel? A single-shot drill lets the model shortcut the whole tree and produces renames instead of explanations; one level per pass forces each "because" to earn its depth. See references/drill.md for when to drill and the anti-patterns to reject.

Usage — claude.ai, Codex, headless scripts (API path)

Where subagents aren't available, call an external model directly.

from challenger import challenge

result = challenge(
    artifact=open('draft.md').read(),
    profile='prose',
    context='Blog post about RAG scaling laws',
    adversary='gemini',     # default — cross-model diversity
)

adversary accepts:

  • auto (default) — resolves to gemini > claude > self based on available credentials. Logs the choice.
  • gemini — Gemini 3.8 Flash at thinking_level='high'. Cross-model + cross-context. Requires Gemini credentials. (Was 3.1 Pro until 2026-09-03; the Pro tier is off routing, its price/quality is dominated by the later Flash models.)
  • claude — Anthropic API. Cross-context, same model family. Do not use this in Claude Code — use the subagent path instead.
  • self — NOT runnable via challenge() (raises with a pointer to prepare_self()). Self-challenge requires the caller assistant to produce the adversary response, which a synchronous function cannot do.

Drill uses the same challenge() call with profile='drill'. challenge() runs the whole sequential-deepen loop internally and returns the synthesized diagnosis:

diagnosis = challenge(
    artifact,
    profile='drill',
    context=ctx,
    finding=suspect,          # required for drill
    max_iterations=5,         # optional — defaults to 5 for drill, 3 for review
)
print(diagnosis['chain'], diagnosis['root_causes'], diagnosis['direction'])

Blocking mode (API path, review profiles only)

result = challenge(artifact, profile='analysis', mode='blocking', max_iterations=3)

Loops the adversary until: (a) no actionable findings, (b) novelty rate > 75% (adversary inventing problems — artifact is clean), or (c) max iterations. mode is ignored when profile='drill' — drill always iterates until bedrock or max depth. Subagent-path callers can replicate blocking mode by looping prepare() / Task / parse_response() themselves and tracking findings across iterations.

Usage — self path (same-context adversary)

When neither subagents nor external API credentials are available — or when subject-matter context from the current conversation is load-bearing for the review — use prepare_self(). The caller assistant inhabits the adversary persona in a dedicated response.

from challenger import prepare_self, parse_response

job = prepare_self(
    artifact=open('draft.md').read(),
    profile='analysis',
    context='Cross-domain claim that depends on codebase-specific IEEE-754 conventions',
)
# job is {'system': <adversary system prompt>, 'user': <artifact + context>, ...}

The caller then:

  1. Reads job['system'] — the prompt opens with SELF-INVOCATION MODE instructing a full persona switch. Commit to it.
  2. In a dedicated response, produces JSON matching the schema described in the system prompt.
  3. Passes that JSON string to parse_response().
# After generating the adversary JSON in a dedicated response:
result = parse_response(adversary_json_text)
print(result['verdict'], result['findings'])

When self beats external: the artifact depends on conventions visible only from inside the conversation (codebase invariants, prior decisions, domain terminology established earlier). External adversaries with generic priors issue confident-but-wrong findings in these cases; self retains the context.

When external beats self: the artifact contains confabulations or blind spots the caller already committed to. Fresh context catches these; self inherits them.

When possible, run both. They have orthogonal failure modes.

Drill via self path uses the same loop as the subagent drill: iterate prepare_self(profile='drill', finding=..., chain=...) — produce one {why, because} per dedicated response — append to chain — until bedrock or max depth — then a final synthesize pass.

Verdicts

  • SHIP: Clean. Deliver.
  • REVISE: Real issues, sound core. Fix and deliver.
  • RETHINK: Structural problems. Reconsider approach.

Severity Levels

  • critical/high/medium/low: Standard severity — actionable findings that block in blocking mode.
  • unverifiable: Adversary flagged something it doesn't recognize (API, pattern, model name) but can't confirm is wrong. Surfaced for awareness but does not block SHIP. Use context to ground the adversary on APIs/patterns it may not know.

Credentials (API path only)

The subagent path needs no credentials. The API path loads from environment or project files:

  • Gemini via Cloudflare Gateway (preferred): CF_ACCOUNT_ID, CF_GATEWAY_ID, CF_API_TOKEN from env or proxy.env
  • Gemini direct: GOOGLE_API_KEY from env
  • Claude API (claude.ai fallback only): ANTHROPIC_API_KEY or API_KEY from env or claude.env

No external skill dependencies. requests is loaded lazily — only the API path requires it.

SURFACE ROUTING — read first

This skill hand-rolls subagent orchestration via raw Anthropic API calls. A managed runtime now does the same job. Which one to use depends on your surface:

  • In Claude Code (incl. CCotw): use the native runtime, NOT this skill. If you can invoke /deep-research, trigger a run with the workflow keyword, set /effort ultracode, or spawn Task subagents — do that instead. The runtime gives 16-concurrent / 1000-agent ceilings, an approval gate, adversarial cross-review, and in-session resume that this skill would otherwise reimplement badly. Dynamic workflows shipped in research preview (Claude Code v2.1.154+, 2026).
  • In claude.ai chat or the bare API (no workflow runtime): use this skill. Parallel API instances over httpx is the only fan-out path here. muninn_utils.dispatch (17 pre-built lenses) already implements this panel over orchestrating-chat-agents' execution model — prefer it over rebuilding the panel by hand. Proceed below.

Discriminator: do you have a native subagent/Task tool or a workflow command? Yes → native. No → this skill. Never reimplement the runtime where it already exists.

Convening Experts

Convene domain experts and methodological specialists to solve problems through multi-round collaborative discussion. Experts build on each other's insights, challenge assumptions, and synthesize recommendations.

Panel Format

Single-Round Consultation

For simpler problems requiring multiple viewpoints:

  1. Assemble panel (3-5 experts based on problem domain)
  2. Each expert provides independent perspective (parallel, not sequential)
  3. Synthesize recommendations with attribution

Multi-Round Discussion

For complex problems requiring collaborative reasoning:

  1. Round 1: Each expert analyzes problem independently
  2. Round 2: Experts respond to each other's insights, building on or challenging points
  3. Round 3 (if needed): Converge on synthesis, resolve disagreements
  4. Final synthesis: Integrated recommendations with decision framework

Expert Roles

Available expertise spans:

  • MSD domain experts (life sciences, engineering, manufacturing, quality, corporate functions)
  • Consulting framework specialists (strategic, process improvement, innovation, systems analysis, root cause)
  • Ancient philosopher method experts (elenchus, division-before-judgment, dichotomy of control, power analysis)

See references/msd-domain-experts.md, references/consulting-frameworks.md, and references/ancient-philosophers.md for complete role catalogs.

Claude loads relevant references based on problem domain.

Panel Convening Logic

Claude selects 3-5 experts based on problem characteristics:

Problem type → Primary expert + Supporting experts

  • Technical troubleshooting → Domain expert + Systems Thinker + Five Whys Facilitator
  • Strategic decision → McKinsey Consultant + relevant domain experts + SWOT Analyst
  • Process improvement → Six Sigma Black Belt + Lean Practitioner + domain Manufacturing Engineer
  • Product innovation → Design Thinking Facilitator + Jobs-to-Be-Done Specialist + relevant engineers
  • Root cause analysis → Domain expert + Five Whys Facilitator + Systems Thinker
  • Market positioning → Porter Framework Expert + Marketing Specialist + BCG Consultant
  • Cross-functional problem → Relevant domain experts + Bain Consultant (RAPID) + Systems Thinker
  • Definitional / conceptual dispute → Socratic Examiner + Aristotelian Taxonomist + domain expert
  • Governance / institutional design → Aristotelian Taxonomist + Thucydidean Realist + Systems Thinker
  • Ideal-vs-pragmatic tension → Platonic Idealist + Aristotelian Taxonomist (native adversarial pair, per Politics IV Part II) + domain expert

Response Format

Name the panel up front, then give each expert's view before synthesizing.

Single-round: list the panel members, then each expert's independent analysis (in any clear order — parallel, not sequential), then a synthesis with attribution.

Multi-round: initial analysis from each expert, then a round where experts respond to specific points from named others (building on or challenging them), then convergence — resolving disagreements or flagging which remain productive — and a final synthesis. Skip rounds that would add nothing (e.g. no round 3 if round 2 already converged).

Headers separating experts and rounds help the reader; the exact heading text doesn't matter.

Expert Behavior Guidelines

Domain Experts:

  • Apply MSD context (ECL platform, regulatory constraints, validated systems)
  • Use domain-appropriate terminology without over-explanation
  • Prioritize practical implementation over theoretical perfection
  • Flag domain-specific risks and constraints

Philosopher Experts:

  • Apply the named method (elenchus, division, dichotomy of control), showing its structure — method, not costume; no period voice
  • Persona-only by default; corpus-injected grounding (verbatim source text + cite-every-claim) when output must be auditable or turns on a work's fine structure — mechanics and routing table in references/ancient-philosophers.md
  • Mix with domain experts — an all-philosopher panel answers modern questions at the wrong altitude unless the question itself is conceptual

Framework Experts:

  • Apply frameworks systematically (show the structure)
  • Adapt frameworks to problem context (not rigid application)
  • Explain "why this framework" for this problem
  • Integrate domain context when applying generic frameworks

Cross-Panel Interaction:

  • Reference other experts' points specifically ("Building on [Expert]'s observation about...")
  • Challenge constructively ("I see it differently because...")
  • Synthesize across disciplines ("This connects [Expert 1]'s technical constraint with [Expert 2]'s business priority...")
  • Flag tensions between perspectives explicitly

Disagreement Handling:

  • Make disagreements productive (what assumptions differ?)
  • Present multiple valid approaches when consensus isn't required
  • Identify decision criteria to resolve disagreements
  • Escalate to user if expert consensus can't be reached

Decision Frameworks

When panel must recommend action:

RAPID (Bain)

  • Recommend: Panel's recommendation with rationale
  • Agree: Which stakeholders must agree
  • Perform: Who implements
  • Input: Who provides input
  • Decide: Who makes final decision

Weighted Decision Matrix

  • Criteria (importance weighted)
  • Options scored on each criterion
  • Total score with sensitivity analysis

Risk-Benefit Analysis

  • Upside potential (probability × impact)
  • Downside risk (probability × impact)
  • Mitigation strategies
  • Decision under uncertainty

MSD Integration

Apply MSD-specific context automatically:

Technical constraints:

  • ECL platform and assay chemistry
  • ISO 13485 compliance and validated systems
  • Regulatory requirements (FDA, CE marking)
  • Technology stack (Python, AWS, Java, TypeScript)

Business context:

  • Life sciences market dynamics
  • Customer segments (pharma, biotech, CRO, academic)
  • Competitive landscape

Cultural factors:

  • Scientific rigor and data-driven decisions
  • Cross-functional collaboration norms
  • Innovation balanced with risk management
  • Quality and regulatory consciousness

Examples

Example 1: Technical Troubleshooting

User: Our new assay is showing high background signal in serum samples

Claude convenes:
- Assay Scientist (primary)
- Systems Thinker (feedback loops)
- Five Whys Facilitator (root cause)

Format: Multi-round (technical nuance requires collaboration)

Example 2: Strategic Decision

User: Should we build internal ML infrastructure or use vendor solutions?

Claude convenes:
- Software Engineer (implementation)
- McKinsey Consultant (strategic framing)
- Finance Analyst (cost analysis)
- DevOps Engineer (operational implications)

Format: Single-round → RAPID framework synthesis

Example 3: Process Improvement

User: Manufacturing yield dropped 8% after equipment upgrade

Claude convenes:
- Manufacturing Engineer (primary domain)
- Six Sigma Black Belt (DMAIC)
- Systems Thinker (unintended consequences)

Format: Multi-round (root cause needs collaborative analysis)

Constraints

Skip the panel entirely when the problem has one clearly correct answer — give a direct answer instead of manufacturing perspectives around it.

Four constraints have no other source of truth in this document, so they stay explicit. Don't invent fictional names for experts (role titles only — "Software Engineer", not "Dr. John Smith, Software Engineer"), and don't invent MSD-specific details beyond general domain knowledge; both would misrepresent a generic panel as grounded in specifics it doesn't have.

The other two are failure modes the panel format itself invites: manufacturing consensus where the experts genuinely disagree (flag the disagreement and say what would settle it), and ending on "here are the perspectives, you decide" instead of a synthesis someone can act on.

Deciding with confidence

scripts/decide.py takes an OpenAI Decisions API request (input plus questions of type predicate, choice or score) and returns its response shape: a probability per option, the chosen value or score, and confidence. Claude Haiku 5.5 answers. Its logits are not exposed, so the distribution is estimated: a sample states a probability per option, and the result is temperature-scaled from a labelled eval.

The default is one sample at low effort: one call, the answer, and the probability that call stated. That confidence is weaker than a pooled one, because nothing checks it. When the confidence will gate an action, run --stack: three parallel samples, each under a different angle (plain, literal, sceptic) and option order, averaged. Their disagreement lowers confidence and shows up in diagnostics.agreement.

Every sample answers each question with a cue first: the few words of the evidence that decide it, quoted. Quoting before committing to probabilities was worth more than any other prompt change tested.

Measured 2026-11-11 through the cli transport at low effort, on four labelled sets: 104 short banking-intent, guardrail and severity items; the held-out half of two life-science sets (77 whole research articles, 289 labelled questions, and 204 article sections, 1,457); and the 2,000 test decisions of the public Typed Decisions benchmark, sent as System One requests. Jev, TypeSafe's purpose-built decision model, is scored on the same items, or taken from the benchmark's card:

short itemswhole articlessectionsTyped Decisions
one sample (default)0.8560.9690.8700.741
--stack0.8940.9620.9060.7465
Jev0.8940.9720.8940.727
one sample without the cue0.8270.9310.857—

Among the 43 general models the Typed Decisions card scores zero-shot, the stack would rank 10th and one sample 13th; Jev is 18th. The life-science labels come from Sonnet 5.5 and Opus 5.5 agreeing, which may favour Haiku, and half of the short items were used to choose the prompt. The stack's main value is its agreement signal: answers its three samples agreed on were 83% to 98% right, split ones 52% to 73%. One sample costs $0.0005 to $0.0024 a request at list price, the stack three times that; a stacked request took a median 8.8 s through the CLI, where Jev answers in under a second. references/method.md has the full tables.

When NOT to use this skill

SituationUse instead
The label set is too large to list in a prompt (a taxonomy, hundreds of tags)hallucinating-labels
The answer needs reasoning, multi-step work or a written explanationa normal Claude call; this skill forbids explanation
You need extracted fields or free-form JSONstructured outputs on the Messages API
A real decision model is reachable (OpenAI /v1/decisions, Strands Decider, Jev) and latency mattersthat model: 0.1–0.3 s against ~0.7–4 s here
Picking which Claude tier runs a subagentagent-routing
The evidence is an imagenot supported; the transports send text only

Abandon a run when diagnostics.agreement is below 1 on most of a batch: the questions or option descriptions are ambiguous, and more samples will not fix them. Rewrite the options (distinct, observable criteria) before sampling more.

Procedure

  1. Write the request as JSON in the OpenAI shape. Give every question a unique name. Options need descriptions that say when each applies; add an other choice when the set is not exhaustive. Score levels go lowest first. When the evidence is a file on disk (a paper, a log, a transcript), give "input_file": "path" in place of "input": the script or the subagent reads it, and you do not write the document out again. When the evidence is this conversation, give "input_transcript": true (or {"last_turns": 20, "max_chars": 60000}): the script reads the transcript Claude Code keeps on disk for the session, renders the last turns with tool calls clipped, and redacts secrets. The file can trail the turn in progress, so do not ask it about a tool result from this same turn.

    {"input": "I was charged twice for my order.",
     "questions": [
       {"type": "choice", "name": "department", "instructions": "Which department should handle this?",
        "choices": [{"value": "billing", "description": "Payments, invoices, refunds."},
                    {"value": "shipping", "description": "Delivery and tracking."},
                    {"value": "other", "description": "Anything else."}]},
       {"type": "predicate", "name": "angry", "instructions": "Is the customer expressing anger?"},
       {"type": "score", "name": "severity", "instructions": "How severe is this?",
        "levels": [{"label": "Cosmetic", "description": "No lost functionality."},
                   {"label": "Workaround", "description": "Fails, another way works."},
                   {"label": "Blocked", "description": "Fails with no workaround."}]}]}
    

    A predicate may carry "criteria": {"true": "...", "false": "..."} to say what each answer means; without it the defaults are "the condition holds" and "does not hold".

    A System One request (the body of TypeSafe's POST /v1/systemone, which Jev and its clones answer) works as it is: state (text, or any JSON, which the prompt shows as indented JSON) plus questions keyed by id, of type noul, choice or score, each with instructions and criteria. The response then comes back in System One's shape: per question id, probabilities keyed by option, label, confidence, and probability_true (noul) or score (score), plus diagnostics. A request written for Jev needs no change.

  2. Pick the transport. --transport auto (the default) takes the first that works:

    TransportNeedsWall time per request
    apipip install anthropic and ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN~0.7 s expected (the API time measured inside claude -p)
    clithe claude CLI, logged in~4 s, measured
    bedrockpip install anthropic, AWS_REGION, AWS credentials; explicit onlyas api

    With none of them, use the subagent path in step 3b.

  3. Run it.

    a. With a transport:

    S=/mnt/skills/user/deciding-with-confidence/scripts   # or wherever the skill lives
    python3 $S/decide.py run request.json            # one low-effort sample
    python3 $S/decide.py run request.json --stack    # three angles, pooled: use when confidence gates an action
    echo '{...}' | python3 $S/decide.py run -
    python3 $S/decide.py batch requests.jsonl --out responses.jsonl   # one request per line, resumable
    

    From Python: sys.path.insert(0, S); import decide; decide.decide(req) returns the response dict, or None when no sample was usable; pass k=3, angles=decide.STACK for the stack. --angles takes any comma-separated list, and a request's "angles" may hold instruction text of your own.

    b. In a Claude Code session, the plugin's decider subagent is the cheaper path: it is pinned to claude-haiku-5-5 (the model the calibration was fitted on) and carries its instructions inline, so it answers without a tool call. Install the deciding-with-confidence plugin, or copy agents/decider.md to .claude/agents/; agent definitions load at session start. Then:

    python3 $S/decide.py emit request.json --stack --agent deciding-with-confidence:decider > batch.json
    

    Use the name the Agent tool lists: deciding-with-confidence:decider from the plugin, decider from .claude/agents/. Send each samples[i].prompt verbatim as an Agent call with that subagent_type, all in one message. Collect the reply texts into a JSON list in replies.json, then python3 $S/decide.py aggregate batch.json replies.json. A single reply carries no agreement signal and no calibration until aggregate has run. With input_file the emitted prompt names the file and the subagent reads it, so you send a short prompt however long the document is.

    Without the agent installed, omit --agent: the system prompt then rides inside each prompt for a stock general-purpose subagent with model: haiku. That works and costs about 58K tokens a sample against 9K.

  4. Read the answer. Predicates give probability (P true). Choices give choice, probabilities and confidence. Scores give score (the probability-weighted level index, so 1.4 sits between levels 1 and 2), probabilities and confidence. Every answer carries diagnostics: agreement, spread, the pre-calibration raw pool, per_sample, and cues, each sample's quote of the evidence it decided on. Read the cues when an answer looks wrong: a cue of "none", or a quote from the wrong part of the evidence, says the sample was guessing.

  5. Act on it with thresholds, never on the top answer alone.

    • With one sample there is no agreement; use the top-probability rule alone, or rerun the doubtful ones with --stack.
    • agreement < 1 (the samples' top answers differ): escalate. With --stack, unanimous answers were 90% to 98% right across the three sets and split ones 63% to 73%.
    • Top probability under 0.7 (after calibration): escalate. On article sections that sent 17% of one-sample answers onward and left 5.6% of all answers wrong; with the stack, adding the split rule sent 19% and left 3.0% wrong. Both assume the escalation target gets every escalated answer right. Treat 0.7 as a starting point.
    • Set the final thresholds from labelled examples of your own traffic, weighing the cost of a false positive against a false negative.
  6. Calibrate on your own labels when you have 50 or more per question type:

    python3 $S/decide.py eval my_labelled.jsonl --out results.jsonl   # {id, input, questions, labels}
    python3 $S/decide.py calibrate results.jsonl --out my-cal.json
    export DECIDER_CALIBRATION=$PWD/my-cal.json
    

    Run eval with --stack so the file holds both temperature sets. One sample and a pool of three need different temperatures, because averaging three samples flattens the distribution. calibrate fits single-sample temperatures from each request's first sample and pooled ones from the pools, and a type with fewer than 50 labels keeps T = 1. The bundled assets/calibration.json was fitted through the cli transport on the 1,798 labelled answers above. Every temperature is below 1, so Haiku's stated probabilities are underconfident and the calibration sharpens them. Single-sample T is 0.74 for predicates, 0.60 for choices and 0.81 for scores; the pooled T values are 0.44, 0.52 and 0.50.

Earned exceptions

RuleBanned whenEarned when
One samplethe confidence decides whether something is escalated, blocked or sent onyou need the answer, and a wrong one is cheap or gets reviewed anyway
--stackcost is the constraint and nothing reads the confidencethe confidence gates an action: parallel samples cost no wall time and 3x the tokens
Bundled calibrationyour transport or domain differs from the eval's (cli; banking, support and life-science text)you have not yet labelled 50 items of your own
Escalate on agreement < 1nothing stronger sits behind ita stronger model or a human can take the split items

Common failure modes

  • decide: no usable sample; first error: TypeError: "Could not resolve authentication method..." → the api transport found no key. Set ANTHROPIC_API_KEY, or pass --transport cli.
  • decide: no transport → no key and no claude on PATH. Use step 3b.
  • diagnostics.samples below k → some replies did not parse. Usually a transport timeout under heavy concurrency; lower DECIDER_WORKERS (default 8) or raise DECIDER_TIMEOUT (default 60 s).
  • Every answer near uniform (confidence around 0.1–0.2 throughout) → options without descriptions, or descriptions that overlap. Rewrite them so each has distinct, observable criteria; more samples will not help.
  • decide: input_transcript: no session id or no transcript for session → the script is not running inside a Claude Code session, or the transcript lives elsewhere. Pass {"path": "/path/to/session.jsonl"}.
  • A subagent reply with prose around the JSON → aggregate takes the outermost {...} and copes. A reply with no JSON at all counts as a failed sample.
  • Transcript text showing up as evidence in the subagent path → the [no-context] line was dropped from the prompt. Send the emitted prompt verbatim.

Verification

Run the tests, then one live request:

python3 -m pytest /mnt/skills/user/deciding-with-confidence/tests -q    # 56 collected, no network
echo '{"input":"Nobody can log in since the 2pm deploy.","questions":[{"type":"predicate","name":"outage","instructions":"Is a service down for users?"}]}' \
  | python3 $S/decide.py run -

The live answer should have probability above 0.9 and diagnostics.samples equal to 1 (3 with --stack). A bad success is an answer with samples: 1 from a --stack run: it looks normal but has lost the agreement signal. Check samples when a batch runs under load.

Diagnosed failures

  • 2026-10-09, first eval: 14 of 520 samples were discarded because Haiku left the options it ruled out of 8-way choices. Missing keys now share the probability mass the sample left unassigned, and the re-run lost none. The prompt also asks for ruled-out options at 0.001.
  • 2026-10-09: claude -p --json-schema doubled API time (2.2 s against 0.8 s) with a second turn and three times the output tokens. The cli transport sends the JSON contract in the system prompt instead.
  • 2026-10-09: the claude-workspace PreToolUse hook that appends parent-transcript chunks to Agent prompts would have put them in the evidence. emit appends [no-context], which that hook honours.
  • 2026-11-10: the cli transport passed the sample prompt as a command-line argument, which caps a single argument at about 128 KB and so cannot carry a long document. The prompt now goes in on stdin.
  • 2026-11-10: a session that pasted a document into each subagent prompt paid for the document in its own output tokens, once per sample. A request with input_file is emitted as a path for the subagent to read.
  • 2026-11-10: a decision about the conversation itself meant restating the conversation in the request. input_transcript reads it from the session's transcript file; on a long session one call over the last six turns read 38,580 characters and cost $0.0036.
  • 2026-11-11: one temperature per question type, fitted on pools, applied to single samples and pools alike. With the cue prompt the stack's pools were further from calibrated (ECE 0.10 to 0.15) than single samples were, because averaging flattens them. Pools now have their own temperatures, and cross-validated ECE is 0.01 to 0.05 for both.

Delegating with context

Writing a background brief for a subagent costs output tokens, generation time, and fidelity: exact paths, error strings and numbers get paraphrased or dropped, and the brief omits what the orchestrator did not know mattered. The context_hook.py PreToolUse hook removes the need. It reads the session transcript, asks Jev which chunks the task needs, and appends those chunks to the subagent's prompt verbatim. The hook fires after the prompt is written, so it cannot save a brief you already wrote. The saving happens here, before the tool input exists.

Procedure

  1. Check the hook is live. test -s ~/.claude/context-filter-log.jsonl && tail -1 ~/.claude/context-filter-log.jsonl shows the last filtered delegation. No file on a first delegation is normal; the hook's reply after your first Agent call confirms it (step 5). Where the hook is not wired (claude.ai, a machine without the settings entry), stop here and write a normal brief.

  2. Preview what the filter would pass. One call, all tasks at once:

    python3 /mnt/skills/user/delegating-with-context/scripts/preview.py \
      --task "Write the xr latency note for the follow-up memory" \
      --task "Draft the closing status for both PRs"
    

    Each task prints the kept chunk ids with a one-line gist. This is the check: find the chunk that holds the fact the subagent must not miss. It takes one to two seconds and costs a fraction of a cent.

  3. Write only the task. What to do, the deliverable, its format, and any constraint that was never written down in the session. Name things the way the transcript names them ("the Sniff Test benchmark table", "PR #812"): Jev reads literally, and a named thing matches its chunk while "that thing we discussed" matches nothing.

  4. Add only what the preview missed. If a needed fact is in no kept chunk, state that one fact in the prompt, or raise the budget with [context-budget: 40000] in the prompt text. Do not restate facts the preview already showed.

  5. Spawn, then read the hook's reply. The hook answers with context-filter: appended N of M parent-session chunks (...) and the kept ids. context-filter FAILED (...) means the subagent got your prompt alone: send it the missing facts with SendMessage or re-delegate with them stated.

The hook also covers create_session from any MCP server (mcp__*__create_session). The spawned session receives the task, a line saying the chunks below are reference data from the parent session, the chunks, and the task again at the end. Give it provenance anyway (issue link, parent session id) per docs/delegation.md rule 1 in claude-workspace: a long first message from a parent still looks like injection to a careful delegate.

Markers the hook reads

In the promptEffect
[no-context]Hook skips; the prompt goes as written
[context-budget: N]Token budget for appended context (default 20000)
subagent_type: "fork"Hook skips; a fork already inherits the parent context

When NOT to use this skill

SituationUse instead
The subagent needs nothing from this session (fresh research, a standalone lookup)Add [no-context] and write the prompt normally
The subagent must reason over the whole session (review everything we did)subagent_type: "fork"; it inherits the full context
Many small per-item judgments (label 400 rows)Jev or Gemini directly, per docs/delegation.md in claude-workspace; a subagent costs ~32k tokens before it reads its prompt
Choosing which model or agent type should run the workagent-routing
Parallel API fan-out from claude.ai, where hooks do not runorchestrating-chat-agents
A long job relayed through many fresh subagents that must carry a running result (a tally, a register, a board)Have each step maintain a compact state file and check its invariants in the harness after every step. Kept chunks hold facts, not a running count: once the events outgrow the budget the count is gone (oaustegard/experiments/clm-relay-needles)

Earned exceptions to "write only the task"

Restating background iswhen
bannedthe fact appears in the transcript as a tool result, a user message or your own reply
earnedthe fact exists only in your reasoning (thinking is not in the transcript), a decision you made silently, or a constraint nobody wrote down
earnedthe preview shows the chunk holding it was not kept

Abandon the procedure when

  • The hook reports FAILED twice in a row: the transport or key is broken, and every further delegation silently runs without context. Write briefs and fix the transport (Failure modes below).
  • The subagent's reply says a fact was missing that the preview showed as kept: the chunk was clipped (a kept chunk is capped at ~6k tokens). Send the fact directly.

Common failure modes

  • No context-filter: line after an Agent call → the hook is not wired or crashed before printing. Check .claude/settings.json for the PreToolUse entry and run echo '{}' | python3 .../context_hook.py; it must exit 0 silently.
  • no Jev transport → neither CF_ACCOUNT_ID + CF_API_TOKEN (the TypeSafe key is stored in the Cloudflare AI Gateway, CF_GATEWAY_ID routes to it) nor TYPESAFE_API_KEY is set in the hook's environment.
  • Preview keeps many chunks that all mention the session's main topic → the scores are compressed because everything looks related. Rename the task with the distinctive nouns of the thing you want; rank, not threshold, is what selects.
  • HTTP 429 Rate limited, code 2003 → the AI Gateway's own limit. The filter runs 2 calls at a time and honours Retry-After; parallel delegations each run their own filter, so stagger a fan-out of more than ~4 spawns.
  • blocked_chunks above 0 in the log, or HTTP 402 "Payment error from model using BYOK" → not billing. TypeSafe's edge WAF rejected the request bytes (shell commands and curl lines in a transcript read as attack payloads). The filter bisects the window and scores each blocked chunk 0.5, so it can still be kept on rank. If a chunk you need is blocked, state its fact in the prompt.
  • A kept chunk states something a later chunk corrected → each chunk is scored on its own merits. User messages are force-kept (clipped) because they carry corrections; if a correction lives in a tool result, state it.

Verification

After the subagent returns: its reply must not say a needed fact was missing, and tail -1 ~/.claude/context-filter-log.jsonl must show the delegation with kept_ids and no error key. A bad success looks like a fluent deliverable with a plausible but wrong number: compare one specific figure against the chunk it came from before relaying it.

Diagnosed failures

  • 2026-09-22: the first selection rule force-kept every user-role message. Skill bodies and stop-hook feedback are user-role (isMeta) turns, and on a 181-chunk session they filled the whole 20k budget before any tool result was considered; the chunk holding the answer was dropped. Fixed: isMeta turns are scored like tool results, and real user messages are clipped and capped at a quarter of the budget.

  • 2026-09-22: a single condensed window (every chunk clipped to ~1k chars) ranked the key chunk 6th because the fact sat mid-result, outside the clip. Paging at 4x view (five ~27k windows, 1.6 s) ranked it 1st. The filter defaults to the paged view.

  • 2026-11-02, hierarchical selection (after DecisionRAG, arXiv 2610.35868): score transcript segments first, then only the chunks inside the top 40% of segments, at 8x view. Over 52 delegations from 16 archived sessions (19 to 581 chunks) it was never worse than the flat 4x filter and better on 3 (+2.7% fact recall, CI +0.0% to +6.4%), at a median 31% fewer Jev tokens. It is the default; CONTEXT_FILTER_MODE=flat restores the flat filter.

  • 2026-11-02: on sessions of ~500+ chunks the global-state loop never terminated and the hook timed out at 120 s, passing no context (fixed in 0.2.3).

  • 2026-09-23, eval over 32 delegations from 8 archived sessions: written briefs lost every fact on 5 tasks by telling the subagent to look the fact up ("check the pr-workflow config entry") instead of stating it; neither the filter nor the full transcript ever did. That is what step 3 prevents.

  • 2026-09-23, first real create_session probes: two Haiku cloud sessions received the appended context (their first-turn token counts match) and stalled at need-input, one saying the message looked truncated, the other that no request had arrived. The context ended the message and the task sat above ~20k tokens of transcript. Labelling the block as reference data and repeating the task after it, the next probe answered correctly first time.

Scripts

  • scripts/chunking.py: transcript JSONL to chunks (user, assistant, harness, tool call + result), with secret values and token patterns redacted before anything leaves the machine.
  • scripts/jevfilter.py: windows under Jev's limits (32k state plus longest question, 64k state plus all questions), one Noul per chunk, rank into the budget.
  • scripts/hierfilter.py: the default selector. Level 1 scores segments (a new one at each user message and every 8 chunks) from short digests; level 2 scores the chunks of the kept segments with jevfilter's windows and packs them with its select.
  • scripts/context_hook.py: the PreToolUse hook. Fails open and says so. CONTEXT_FILTER_MODE=flat switches it to jevfilter.build_context.
  • scripts/preview.py: step 2.

Install as a plugin (the hook wires itself; hooks/hooks.json):

claude plugin marketplace add oaustegard/claude-skills
claude plugin install delegating-with-context@oaustegard-claude-skills

Or wire it by hand in .claude/settings.json, guarded so a missing file exits 0 (python3 on a missing path exits 2, which blocks the tool call):

{"matcher": "Agent|Task|mcp__.*__create_session", "hooks": [{"type": "command", "timeout": 120,
  "command": "f=/mnt/skills/user/delegating-with-context/scripts/context_hook.py; test -f \"$f\" || exit 0; exec python3 \"$f\""}]}

Evidence: with ~20k tokens of selected chunks a fresh subagent found 99/105 required facts, against 102/105 with the whole ~81k-token transcript and 82/105 with a written brief; method and caveats in oaustegard/experiments/subagent-context-filter/RESULTS.md.

Related evidence, 2026-10-01: relaying a long stream through one fresh call per chunk, Jev keeping lines verbatim kept 835/835 required lines exactly, while a subagent copying them into its own notes retyped one with an invented hash; for a running register the subagent's in-place table won (oaustegard/experiments/clm-relay-needles).

Dispatching subagents

Claude Code has three ways to start a subagent. They differ most in what the launching costs the parent.

Agent toolWorkflow toolclaude -p children (scripts/headless_fanout.py)
parent costone parent turn per launch and one per completionone tool call per runone Bash call per run
concurrency20, and a 21st launch errors (it does not queue)min(16, vCPUs − 2): 2 on a 4-vCPU CCotw containerRAM-bound, about 210 MB per child: 40 ran clean on 4 vCPU / 15 GB
child contextsession prefix (~32–55K tokens), plus parent transcript chunks if a context hook is installedsession prefixwhat you pass: about 1.2K tokens with --system-prompt and no tools, 39K with the default prompt
toolsagent type's tools, the session's MCP connectors, hooksas Agentbuilt-in tools named in --tools, write paths allowlisted; no MCP, hooks or skills
effortsession's level, or the effort: in a named agent type's frontmatteragent(prompt, {effort})--effort (measured: low gave 0 thinking tokens, high gave 112–197)
needs user opt-innoyes: an explicit request, ultracode, or a skill that calls itno
cost visibilitynone per childnone per childtotal_cost_usd and tokens per child, in a ledger
output checkyoursin-scriptfile plus JSON Schema, or CLI-validated --json-schema

The headless runner is the default for volume. On 2026-10-11 a codex build dispatched 314 Haiku children through the Agent tool. The children cost $11.61. The 1,458 Opus parent turns the dispatching took (each re-reading a context of about 400K tokens) cost roughly $160–170, about 14× the work. Five of the same tasks through headless_fanout.py cost $0.10, $0.02 each against $0.033 per Agent-tool child, and one parent call.

Choosing

Answer in order. The first yes picks the mechanism.

  1. Does the child need what only the session has? That means MCP connectors (Muninn, Gmail, GitHub MCP), a context hook's transcript chunks, a worktree, or a custom agent type with guard hooks such as auditor. If so, use the Agent tool. Keep the count small, or batch many items into each child, because every launch costs a parent turn.
  2. Do you need to read each result before deciding the next launch? That is iterative work where task N+1 depends on N's answer. Use the Agent tool.
  3. Did the user opt in to a workflow, and does the procedure have phases a script should enforce (find → verify → fix, with retries and gates)? Use Workflow, and load workflow-authoring first. Accept 2-way concurrency on a 4-vCPU container, or move the bulk phase to headless children.
  4. Are there ten or more independent tasks whose inputs fit in a prompt or in files, and whose output a script can check? Use headless children. This covers judges, graders, extractors, page writers, verifiers and per-file reviews.
  5. Fewer than ten small independent tasks: use the Agent tool in one message. The parent-turn cost is still small at that size.

Earned exceptions

rulebanned whenearned when
Use headless for volumethe child needs an MCP connector or the parent's transcriptthe inputs can be written into the prompt or a file first
Agent tool for small countsthe parent context is already past ~300K tokens, so each launch turn is expensivethe parent context is small, or the results feed the next launch
Judges in headless children—always: a child with no hooks cannot see the parent discussing the items it scores

Headless fan-out procedure

  1. Write one task per line to tasks.jsonl: {"id", "prompt"} plus any of out (absolute path of a JSON file the child writes), json_schema (structured output, no file), model, tools, allow, deny, system_prompt, timeout. {out} in a prompt expands to the out path. This is the same {id, prompt, out} shape claude-workspace's scripts/fanout.py dispatches over the Agent tool, so one file runs on either engine.
  2. Put everything the child needs into the prompt or into files it can Read. The child has no hooks, no CLAUDE.md, no skills, no MCP and no memory of this session. For no-tool tasks (judges, classifiers) inline the inputs and pass "tools": "". Put a large prefix shared by every task in system_prompt, since that is the cacheable part.
  3. Dry-run. This prints the first three commands:
    S=/mnt/skills/user/dispatching-subagents/scripts/headless_fanout.py
    python3 $S run tasks.jsonl --work RUN --schema schema.json --dry-run
    
    Check --model, --tools and the Edit(//...) rule against the out path.
  4. Launch as a background Bash command, never as a waiting loop. Run the job itself in the background and end the turn. The completion notification brings you back.
    python3 $S run tasks.jsonl --work RUN --schema schema.json \
        --model haiku --concurrency 16 --max-usd 5 > RUN.log 2>&1; python3 $S status RUN
    
    (Bash run_in_background: true.) Defaults are --tools Read,Grep,Glob,Write, 1 retry, 900 s per child, concurrency 16, and 5-minute prompt-cache writes (--cache-ttl 5m).
  5. Read status. It reports attempts, ok, failing ids with reasons, total cost, tokens and permission denials. Rerunning the same command skips done tasks and retries the rest.

What the runner enforces per child:

  • a fresh --session-id and --no-session-persistence;
  • the parent's session-binding variables removed, because without that a child reports the parent's session id;
  • --setting-sources "" and --strict-mcp-config, so no hooks and no MCP;
  • --permission-mode dontAsk with an allowlist: Read/Grep/Glob when listed, Edit(//<out>) for the out file, and any --allow rules; deny rules (task field or --deny) go to --disallowedTools and win over the allowlist, so a child can hold Bash with Bash(curl:*) refused;
  • done means verified: the out file parses and passes the schema, or structured_output is present. The child's claim of success does not count.

Permission rule syntax under dontAsk

Measured 2026-10-11, Claude Code 2.1.296:

  • A write is allowed by Edit(//abs/path/**) or Edit(rel/**) (relative to the child's cwd). Write(path) and a bare Edit are both refused.
  • Bash(grep:*) style prefixes work. A prefix rule matches the start of the command only, so Bash(curl -s https://api.github.com/*) still admits -X DELETE appended after it. Never allow Bash for a child that holds a token or that could reach a write API. Fetch live data yourself and put it in the prompt instead.
  • Denied calls appear in the child's permission_denials, and the ledger counts them. A child that hits a denial usually reports the step as failed and stops (it does not route around).

Agent tool procedure

  • Launch all children of a batch in one message, run_in_background: true, at most 20 in flight. A launch over 20 errors and is lost.
  • Batch items per child. The per-spawn prefix (32–55K tokens) is paid once per invocation, however many items it carries.
  • In claude-workspace, scripts/fanout.py keeps the ledger and validates outputs. Its next can report a slot free before the agent stops. Dispatch one replacement per completion notification instead (diagnosed 2026-10-11, the codex build).
  • Subagents that must talk to each other use SendMessage / ListAgents; the measured traps (reply by agentId, never by the envelope's from; subagents have no ListAgents) are in references/native-messaging.md.
  • Put [no-context] in judge and eval prompts where the delegating-with-context hook is installed, or the judge sees the parent discussing what it scores.

Workflow procedure

Load workflow-authoring before writing the script, and run only with the user's explicit opt-in. Give agent() its effort where it matters. For a bulk phase on a small container, a Workflow step can shell out to headless_fanout.py instead of awaiting 100 agent() calls two at a time.

When NOT to use this skill

situationuse instead
which model or effort a child should get, cascades, escalationagent-routing
fan-out from claude.ai chat, where no Agent tool or claude binary existsorchestrating-chat-agents
per-item classification at large volume, where a cheap non-Claude model is enoughinvoking-gemini
a deterministic multi-step pipeline with no model judgment between stepsflowing
work that must outlive this session or containercreate_session (claude-workspace docs/delegation.md)
hundreds of workers fixing failing tests for hours, each fix merged behind a no-regression gateswarming-to-green (it runs this skill's headless runner per worker)
improving one program against an automatic scorer with rounds of cheap proposers and a writerevolving-with-swarms
one subagent for one questionthe Agent tool directly, with no ledger

Abandon a headless run mid-way when the first wave's ledger shows the same failure reason on most tasks (a prompt or permission bug, not bad luck), or when denials is non-zero on tasks meant to need no tools. Stop it, fix the task file, and rerun; done tasks are kept.

Common failure modes

  • A child's session_id equals yours. The session variables leaked (CLAUDE_CODE_SESSION_ID and the rest). Run children only through the runner, or strip SESSION_ENV yourself.
  • The output file is missing and the ledger shows denials > 0. The write rule did not match: the Edit(//...) path differs from the path in the prompt, or someone used Write(...). Dry-run and compare.
  • modelUsage lists claude-haiku-5-5 on a Sonnet or Opus run. This is Claude Code's own side call, about 3% of the cost. It is expected. Read the per-model split before attributing cost.
  • A judge or extractor reads only part of a large input file. Inline the input, or split files so each fits one Read. Check what the child read before trusting the scores (oaustegard/experiments METHODS.md).
  • Spend climbs past expectation. Every child writes its own cache, since nothing is shared across --no-session-persistence children. Move the common prefix into system_prompt, and cap the run with --max-usd. The runner stops launching once the cap is reached.
  • Many children fail with "transient" reasons at once. The run hit a rate or overload limit. Lower --concurrency. The runner backs off and retries.
  • A short child costs several times its prompt. Every claude -p call carries about 1,370 tokens of the CLI's own, even with --system-prompt, --tools "" and --setting-sources "": about 550 read from cache and about 810 written fresh on every call (they hold per-session details, so no later call can read them). On a 60-token Haiku answer that overhead is most of the bill. Batch small items into one child, or account for it per call (measured 2026-11-09, oaustegard/experiments haiku-swarm-poc).
  • cacheCreationEphemeral1hInputTokens is non-zero in modelUsage. The CLI writes prompt cache at the 1-hour TTL by default, at 2x input against 1.25x for the 5-minute one. FORCE_PROMPT_CACHING_5M=1 (or CLAUDE_CODE_PROMPT_CACHE_TTL=5m) switches it; the runner sets it unless --cache-ttl 1h. Use 1h only when children sharing a prefix start more than five minutes apart.
  • A Haiku 5.5 child keeps thinking at effort low. Its thinking is adaptive and the CLI cannot switch it off: MAX_THINKING_TOKENS=0 and --settings '{"alwaysThinkingEnabled": false}' are ignored, and CLAUDE_CODE_MAX_OUTPUT_TOKENS below what the model wants fails the call with "exceeded the N output token maximum" while billing the tokens. Effort low is the floor; there it thought 0 tokens on 65% of short-idea calls (mean 95, top tenth 302 or more).
  • A large fan-out stops on the account's usage window. With --output-format stream-json --verbose each child emits a rate_limit_event whose unifiedWindows carry five-hour and seven-day utilization. Read it to pause before the window runs out instead of failing mid-run; ten parallel lanes used most of a five-hour window in about 70 minutes on 2026-11-09.

Verification

python3 $S status RUN          # exit 0 only when every task's last attempt is ok
python3 /mnt/skills/user/dispatching-subagents/scripts/test_headless_fanout.py   # offline, fake claude binary

A bad success is a run where every task passes the schema check but the outputs are copies of the example in the prompt, or are empty lists. Read three outputs before using any aggregate.

Diagnosed failures

  • 2026-10-11, codex build. 314 Agent-tool launches from an Opus parent cost about 14× the children's own bill in parent turns. The fix is this skill's headless runner. On five of the same verifier tasks it cost $0.10 in one call.
  • 2026-10-11, probe. claude -p children with no flags inherited the parent's session id and wrote transcripts under the parent's id. Fixed by the runner's environment strip and --session-id.
  • 2026-10-11, probe. --allowedTools "Write(/abs/out/**)" was refused under dontAsk and the child wrote no file. Edit(//abs/out/**) works.

Down-Skilling: Opus → Haiku Distillation

Translate your reasoning capabilities into explicit, structured instructions that Haiku 5.5 can execute reliably. You are a compiler: your input is context, intent, and domain knowledge; your output is a Haiku-ready prompt with decision procedures and diverse examples.

Core Principle

Opus infers from WHY. Haiku executes from WHAT and HOW.

Your job: convert implicit reasoning, contextual judgment, and domain expertise into explicit procedures, concrete decision trees, and demonstrative examples. Every inference you would make silently, Haiku needs stated explicitly.

Economics: Why Examples Are Free

Opus 5.5 costs 40× Haiku 5.5 on both sides ($4/$20 vs $0.10/$0.50 per MTok for prompts up to 100K tokens; 2026-10 pricing), and Sonnet 5.5 costs 20×. A task that costs $1.00 on Opus costs ~$0.025 on Haiku, so retries no longer erase the saving: Haiku can fail and retry a dozen times and still cost less than one Opus attempt. (At Haiku 4.5's $1/$5 it was 5×, and one retry halved the saving.) What a misfire costs now is mostly silent error, a plausible wrong answer no verifier caught, not dollars.

The math that matters:

  • Input tokens are cheap (Haiku 5.5: $0.10/MTok input vs $0.50/MTok output)
  • Adding 2,000 tokens of examples costs ~$0.0002 per call
  • Give the orchestrator a verifier and an informed retry (the prior output plus the check's failure output). On a 2026-10-07 repair battery, Haiku 5.5 rescued every rung-1 miss that way (agent-routing)
  • What prevents the misfires a verifier cannot see is a test set run before shipping, not a larger example block (see Start Bare below)

What this means for prompt design:

  • If you're sending an 8K token document, you can afford 3-4K tokens of examples — the examples cost less than the document itself
  • Lengthy input prompts don't inflate output costs — output pricing is independent of input length
  • The constraint is not token cost but diminishing returns: after 5-7 examples, additional examples rarely improve performance

Bottom line: At 5.5 prices, tokens are no longer the constraint. The expensive mistakes are a judgment rule shipped without the example that calibrates it, and examples standing in for a test set. Both cost you a wrong answer that ships.

Haiku 5.5: Start Bare, Then Add Only What a Miss Asks For

Measured 2026-10-07 on three of this skill's own distilled prompts (feedback extraction, moderation, SQL), each run as shipped, with its <examples> stripped, and bare (task, label names and output schema only); two replicates (oaustegard/experiments → downskill-shots):

armrule-determined itemsitems an example settles
shipped (rules + examples)40/4022/24
bare (no rules, no examples)38/4020/24
rules without examples40/4015/24

Three things follow. Haiku 5.5 does not need examples to apply a rule. Rules written for an example-heavy prompt mislead it once the examples are gone, because it reads them literally: "surface content hostile → flag" flagged a harsh opinion about an article, which the example had shown as APPROVED. And a house convention the model would not guess (spending counts completed orders only) transferred through its example in 1 of 2 runs, so state it as a rule.

Procedure on Haiku 5.5:

  1. Write the bare prompt: role, task, every valid label, the output schema.
  2. Write 10–20 test inputs with expected outputs, at least a third of them at decision boundaries. Run the bare prompt on them.
  3. For each miss, add the smallest thing that fixes it:
    • an unstated default or house convention ("some" means LIMIT 10; spending counts completed orders) → one explicit rule;
    • a judgment boundary (opinion versus harassment, MIXED versus NEGATIVE) → one example at that boundary, with its <reasoning>. Never a judgment rule without the example that calibrates it.
  4. Rerun. Stop when the test set passes; typical-case examples add nothing here.
  5. For rewriting and summarisation, keep an explicit no-invention rule: without one, 3 of 8 rewrites stated facts the source does not ("We tested it across a range of workloads"). Don't count on a fact-listing step against invention. On 5.5, 0 of 64 rewrites invented a technical detail whether that step was hidden in the process, a visible <facts> block, a list supplied in the prompt, or absent (retested 2026-10-08). The anti-invention examples (model the silence) are optional too: the set that drove Haiku 4.5 to 19 of 20 produced 0 of 8 on 5.5.
  6. A fact list decides what the rewrite carries over, so choose it on purpose:
    • A list Haiku writes itself records the source's claims about itself as facts ("the team describes it as a paradigm shift"), and the rewrite repeats them (8/8). If those should go, say so as a rule.
    • A list supplied in the prompt is followed omissions and all: a supplied list that left out "it is launching" produced 2 of 8 rewrites calling the product "in beta". Supply one only when it has been checked against the source.
  7. Don't instruct Haiku 5.5's reasoning ("think in 3-5 steps", "in your reasoning, first list..."). Anthropic's system card measures it near 0% at following instructions about the content of its own thinking (figure 6.4.2.4.A). Put any step whose result you need into the visible output, where it can be checked, and set reasoning length with the effort parameter.

Typically this ends at 0–3 examples, not 4–7. The rest of this file is the Haiku 4.5 method. Use it for Haiku 4.5, and for the boundary examples step 3 calls for.

Before Distilling: Check Whether the Task Needs It (2026-07 calibration)

This skill's gap catalog was originally derived from model-card priors. A 2026-07-15 empirical calibration (300 measured Haiku 4.5 calls; data in the agent-routing skill's references/calibration-2026-07-15.md) found Haiku 4.5 substantially stronger than those priors on mechanically checkable work: 240/240 on nested modular arithmetic (16-leaf expression trees), 30-hop function chains, 25-operation state tracking, trap-laden word math, and 5-simultaneous-constraint sentence generation (exact word counts, required/forbidden tokens, a lipogram) — at effort: low, and in one control with chain-of-thought suppressed entirely. On the same battery Sonnet at low effort scored 17/20, missing exact-count constraints Haiku satisfied.

Triage before investing in a full distillation:

  1. Output mechanically checkable (schema validates, tests run, counts count, regex matches)? → A minimal prompt + a deterministic verifier + retry-or-escalate-on-fail is cheaper and more reliable than an example-heavy prompt with no verifier. Spend your effort writing the checker, not the examples. Precision constraints ("exactly N words") held reliably when the prompt defined the counting rule explicitly ("a word = any run of letters or apostrophes") — definitional precision, not scaffolding volume, did the work.
  2. Output requires judgment (tone, relevance, quality, ambiguity resolution)? → This is what the rest of this skill is for. Distill fully: procedures, rubrics, n-shot examples.
  3. Mixed? → Split the task. Route the checkable part per (1); distill only the judgment part.

Iteration warning (measured, same calibration): never ask Haiku to blindly re-improve its own output in a loop. On correct answers, re-application returned the identity (0 changes in 96 loop-pairs); on the one output that did change, quality regressed on the second pass and then froze on the degraded text for all remaining passes. If you iterate, score every iteration with an out-of-band checker or judge, keep argmax, and stop at the first regression.

Activation

When triggered, perform these steps:

  1. Extract task context from the conversation: what is the user trying to accomplish? What domain knowledge applies? What quality criteria matter?

  2. Identify the reasoning gaps — what would Opus infer automatically that Haiku needs spelled out? Common gaps:

    • Ambiguity resolution (Opus picks the sensible interpretation; Haiku needs a decision rule)
    • Quality judgment (Opus knows "good enough"; Haiku needs explicit criteria)
    • Edge case handling (Opus reasons through novel situations; Haiku needs enumerated cases)
    • Output calibration (Opus matches tone/length intuitively; Haiku needs explicit constraints)
  3. Generate the distilled prompt following the structure in Prompt Architecture

  4. Add examples. On Haiku 5.5, only at the boundaries a bare run missed (see Start Bare), usually 0–3. On Haiku 4.5, generate 4–7 diverse examples per Example Design; that was the highest-leverage step there.

  5. Audit your example set before delivering. Two checks, both must pass:

    • Source-anchoring: for each example output, every concrete fact (named technology, number, comparison, quoted phrase) is inferrable from that example's input. If you can't reproduce the output knowing only the input, either add the detail to the input or remove it from the output. Invented facts in examples cause Haiku to copy the invention pattern at runtime.
    • Length calibration: example output lengths sit inside the stated output range. If your rule says "60–90 words" but your examples average 35, the rule will not hold — Haiku follows the example central tendency.

    If either check fails, regenerate the offending examples before delivering. Examples beat rules; misaligned examples beat aligned rules.

  6. Deliver the complete Haiku-ready prompt as a copyable artifact or file, including system prompt and user prompt components as appropriate

Prompt Architecture

Structure every distilled prompt with these components in this order. Haiku responds best to this specific sequencing:

<role>
[Single sentence: who Haiku is and what it does]
</role>

<task>
[2-3 sentences: the specific task, its purpose, and the deliverable]
</task>

<rules>
[Numbered list of explicit constraints. Be precise about:]
- Output format (JSON schema, markdown structure, etc.)
- Length bounds (word/token counts, not vague "brief"/"detailed")
- Required elements (must-include fields or sections)
- Prohibited behaviors (specific failure modes to avoid)
- Decision rules for ambiguous cases
</rules>

<process>
[Numbered steps. Maximum 7 steps. Each step is one action.]
[Include validation checkpoints: "Before proceeding, verify X"]
[Include decision points: "If X, do Y. If Z, do W."]
</process>

<examples>
[Haiku 5.5: 0-3 boundary examples, each calibrating a judgment rule]
[Haiku 4.5: 4-7 diverse examples; the LARGEST part of the prompt]
[See Example Design section for distribution requirements]
</examples>

<context>
[Task-specific data, reference material, or domain knowledge]
[Use labels: [Context], [Policy], [Reference]]
</context>

Haiku Optimization Rules

Apply these when generating any Haiku-targeted prompt:

Structure & Syntax

  • Use XML tags to delimit every section — Haiku respects labeled boundaries
  • Keep sentences under 25 words where possible
  • One instruction per sentence; split compound instructions
  • Use numbered steps, not prose paragraphs, for procedures
  • Specify token/word budgets explicitly: "respond in 80-120 words"

Reasoning Support

  • Replace open-ended judgment with decision rubrics: BAD: "Assess whether the code is production-ready" GOOD: "Check: (a) no TODO comments, (b) all functions have error handling, (c) no hardcoded secrets. Score pass/fail per item."
  • Bound reasoning depth: "Think in 3-5 steps, then give your answer" (Haiku 4.5; on 5.5 this lands in reasoning it does not take instructions about, see step 7 above)
  • Provide a fallback for uncertainty: "If you cannot determine X, respond with: 'UNCERTAIN: [brief reason]'"

Context Management

  • Front-load critical instructions (Haiku attends strongly to position)
  • Budget rule of thumb (Haiku 4.5): instructions + rules ≤ 800 tokens, examples get the rest. For a task processing an 8K document, 3-4K tokens of examples is well within budget and pays for itself in reliability
  • Pass only the 1-3 most relevant context snippets, not full documents
  • Use explicit delimiters between context and instructions

Output Control

  • Require structured output (JSON, labeled sections) for extractable results
  • Provide an output template Haiku can fill in
  • Specify what comes first in the response: "Begin your response with..."
  • For classification tasks, enumerate all valid categories

Failure Prevention

  • Anticipate Haiku's common failure modes and add guardrails:
    • Hallucination: "Use ONLY information from the provided context. If the answer is not in the context, say 'Not found in sources.'"
    • Verbosity: "Maximum 150 words. Do not add preamble or caveats."
    • Format drift: Include the output schema in both rules and examples
    • Instruction skipping: Number all constraints; reference them in the process steps: "Apply rules 2-4 from "

Example Design

On Haiku 4.5, examples were the single highest-leverage investment. On Haiku 5.5 their job narrows to calibrating judgment rules and showing what to leave unsaid; see Start Bare. The rest of this section is the 4.5 method. Rules tell Haiku what to do; examples show it what "done right" looks like. When rules and examples conflict, Haiku follows the examples. When rules are ambiguous, Haiku extrapolates from examples. This makes examples the primary steering mechanism — not a supplement to rules, but the dominant signal.

Given the economics (see Economics), you should invest heavily here. A prompt with 800 tokens of rules and 3,000 tokens of examples will outperform one with 2,000 tokens of rules and 500 tokens of examples almost every time.

Minimum Example Count: 4 (Haiku 4.5)

For Haiku 4.5, generate 4-7 diverse examples per distilled prompt. Fewer than 4 is under-investing. The marginal cost of each example is negligible compared to the reliability improvement. Use this distribution:

#RolePurpose
1Typical caseThe most common, straightforward input. Establishes the baseline pattern.
2Second typical variantA different but common input — varies length, domain, or structure from #1. Prevents Haiku from over-fitting to a single pattern.
3Edge caseUnusual but valid input: empty fields, very long text, special characters, boundary conditions, ambiguous phrasing.
4Negative case (tagged BAD/GOOD pair)For tasks where Haiku could output something plausible-but-wrong (rewriting, summarization, NL→command, anything that rewards confident specificity), include an example pair tagged "BAD output:" and "GOOD output:" that demonstrates the specific failure mode you want to prevent for this task — invented architecture details, invented CLI flags, invented numbers, etc. A generic "bad output" example is much less effective than one that names the failure category Haiku is most likely to fall into.
5+Tricky/boundary casesInputs near decision boundaries where Haiku is most likely to fail. The cases you'd use for a test suite.

Why the second typical case matters: With only one typical example, Haiku may latch onto incidental features of that example (its length, word choice, domain). A second typical case from a different angle shows Haiku which features are task-relevant and which are coincidental.

Example Format

<example>
<input>
[Realistic input data — use real-world length and complexity]
</input>
<output>
[Exact format Haiku should produce — not a description, the actual output]
</output>
<reasoning>
[1-2 sentences: WHY this output is correct. Which rules applied.
 What Haiku might have gotten wrong without this example.]
</reasoning>
</example>

The <reasoning> tag is not optional for complex tasks. It acts as a chain-of-thought anchor, showing Haiku the reasoning pattern to follow. For classification and extraction tasks, reasoning should reference the specific rule numbers that drive the decision.

When the input could be abstract: model the silence

Some tasks (rewriting, summarization, voice/register editing, explanation-from-source) take inputs where the source content is itself abstract — marketing copy, vague descriptions, high-level summaries. These tasks are uniquely prone to a failure mode where Haiku invents concrete details that sound right for the domain but aren't in the source.

Symptom: an output that confidently mentions specifics — algorithm names, technology choices, percentile numbers, version comparisons — not present in the input.

Fix: include an example pair where the input is genuinely abstract and the output explicitly acknowledges what is unspecified, rather than filling the gap with plausible-sounding inferences. Pattern:

<example>
<input>
"We're excited to release our new search backend. We think it's a
massive upgrade over the old one."
</input>
<output>
New search backend released. The announcement does not specify the
underlying changes — whether they affect indexing, ranking, query
parsing, or operational characteristics is not stated. Migration
guidance will follow. The old search path remains available during
transition.
</output>
<reasoning>
The source provides only one factual claim ("released") and one
unverifiable claim ("upgrade"). The output stays at the source's
level of abstraction and explicitly names the categories of
information the source omits, rather than inventing plausible
specifics (algorithm changes, performance numbers, technology names).
</reasoning>
</example>

Pair this with the tagged BAD/GOOD negative example (row 4 in the distribution table above) that demonstrates the exact invention you want to prevent. On a voice-rewrite task, switching from un-anchored examples to this pattern dropped Haiku's architectural-hallucination rate from 95% to 0% (n=25 across two probes).

Example Sizing Guidance (Haiku 4.5)

Task processing...Recommended example budget
Short inputs (<500 tokens)1,500-2,500 tokens of examples (4-5 examples)
Medium inputs (500-4K tokens)2,500-4,000 tokens of examples (4-6 examples)
Long inputs (4K-8K tokens)3,000-5,000 tokens of examples (5-7 examples)

Context is no constraint here: Haiku 5.5 has a 1M window, though a prompt over 100K tokens is billed at 5× on both sides ($0.50/$2.50 per MTok). The limit is diminishing returns: after 7 examples the marginal benefit drops sharply unless the task has a very large classification space.

Example Quality Criteria

  • Examples must be realistic, not toy data — match the complexity and messiness of real inputs
  • Output format must be identical across all examples — Haiku treats format inconsistency as a signal that format doesn't matter
  • Include the hardest case you expect Haiku to handle
  • Vary input characteristics: length, complexity, domain, tone
  • Never include an example that contradicts your rules
  • Order examples from simplest to most complex — this progressive difficulty helps Haiku build up its understanding
  • Source-anchor every concrete fact. Each specific noun, number, or claim in an example output must be inferrable from that example's input. If your example output invents a plausible architectural detail, Haiku will copy that invention pattern even when its real input doesn't support it. Audit by covering the input and asking: "could I have produced this output knowing only the input?" If not, fix one or the other.
  • Match example length to the stated output range. Examples outweigh rules on length. If your rules say "output 60–90 words" but your examples average 35 words, Haiku will produce ~35 words. Calibrate example output lengths to sit inside (or slightly above) the stated output range.

Delivery Format

Present the distilled prompt in a code block or artifact with clear section markers. Include:

  1. System prompt (if applicable): role + persistent rules
  2. User prompt template: with {{placeholders}} for variable content
  3. Examples: embedded in the prompt or as a separate few-shot section
  4. Usage notes: any caveats about when this prompt may fail and what to watch for

When generating for API use, include the model parameter and recommended settings:

model: claude-haiku-4-5-20251001
max_tokens: [appropriate for task]
temperature: 0 (for deterministic tasks) or 0.3 (for creative tasks)

Agentic Resource Selection

The skill includes two directories of granular reference files. Do NOT read them all. Scan the index below, then read only the files relevant to the current task.

gaps/

Each file documents one reasoning pattern where Haiku diverges from Opus, with a tested mitigation strategy. Read 2-4 per task.

FileUse when the task involves...
ambiguity-resolution.mdInput that has multiple valid interpretations; vague user requests
code-generation.mdGenerating code, scripts, or queries; style matching to existing code
comparative-analysis.mdComparing options, pros/cons, tradeoff analysis
conditional-logic.mdDecision trees, branching rules, nested if/then logic
context-utilization.mdLong context windows, documents >2K tokens, position-sensitive info
counting-enumeration.md"Generate exactly N items", counting occurrences, list lengths
creative-generation.mdWriting, tone adaptation, persona consistency, style matching
implicit-constraints.mdTasks where tone, audience, or format norms are assumed not stated
instruction-density.mdTasks requiring 8+ simultaneous constraints; complex rule sets
multi-hop-reasoning.md3+ step inference chains; cause-effect-consequence analysis
multi-turn-consistency.mdChatbot behavior, stateful conversations, persona maintenance
negation-handling.mdConstraints phrased as "don't", "never", "avoid"; prohibitions
nuanced-classification.mdBorderline cases, multi-label classification, overlapping categories
output-calibration.mdLength control, format precision, verbosity management (ALWAYS read)
parallel-consistency.mdGenerating multiple similar items; lists where format must be uniform
partial-information.mdMissing fields, incomplete input, optional data, error states
schema-adherence.mdStructured output (JSON, tables) that must survive edge-case inputs
self-correction.mdTasks needing verification; quality checks before output
summarization-fidelity.mdSummarizing documents without distortion, position bias, or fabrication
tool-use-planning.mdMulti-tool workflows, API orchestration, dependency ordering

examples/

Complete before/after distillations. Each shows an Opus-level task → Haiku-optimized prompt with annotated examples. Read 1-2 closest to the current task domain.

FileUse when distilling...
api-orchestration.mdMulti-step tool/API workflows with dependencies and branching
code-review-triage.mdAnalysis tasks with severity classification and structured JSON output
content-moderation.mdSafety-critical classification with "when uncertain" defaults
creative-rewriting.mdTone adaptation, audience-aware rewriting, style transfer
data-extraction.mdSchema-bound extraction from unstructured text to JSON
document-qa.mdRAG / retrieval-grounded QA with citation and "not found" handling
email-summarization.mdInformation extraction from conversations/threads into sections
meeting-notes.mdTranscript processing into decisions, actions, and next steps
resume-screening.mdMulti-criteria evaluation with parallel scoring structure
sql-generation.mdNatural language to code with schema constraints and error handling
step-by-step-analysis.mdMulti-step analytical reasoning with explicit decision rubrics
text-classification.mdMulti-label classification with confidence and ambiguity handling

Self-Check

Before delivering, verify the distilled prompt against these criteria:

  • Every Opus inference is made explicit
  • All constraints are numbered and cross-referenced
  • 4+ diverse examples with consistent output format
  • Examples include: 2 typical, 1+ edge case, 1+ negative/rejection case
  • Example tokens ≥ 2× rule tokens (examples should be the bulk of the prompt)
  • No instruction assumes Haiku will "figure it out"
  • Decision points have explicit branches, not open-ended judgment
  • Output format is demonstrated, not just described
  • <reasoning> tags explain WHY each example output is correct

Evolving with swarms

Each round, scripts/evolve.py runs K swarms in parallel. A swarm is N proposer calls, each reading one parent program and one focus x thinking-style prompt and replying with an idea of at most 45 words, then one writer call that turns the ideas and the parents into a complete program. The program is scored by your evaluator as soon as it exists and joins the archive the next round draws parents from: refine one of the best two, combine three of the best ten, explore two from anywhere, start fresh from the initial program. Every model call goes through headless claude -p, is priced at list rates from its modelUsage, and counts against a dollar cap.

The design is a text-only version of SwarmEvolve (Jiang and Karniadakis, arXiv 2611.08483), whose agents think inside an open model's activations, which an API model does not expose.

When it pays

Measured 2026-11-09 against stock OpenEvolve on Sonnet 5.5, three seeds per arm, three problems (references/evidence-2026-11.md):

swarm (defaults)Sonnet loop, best effort per problem
$ to 99% of reference, circle packing$0.013–0.019$0.32–0.44
$ to 99%, hexagon packing$0.035–0.060$0.53–0.58
$ to 99%, Heilbronn triangle$0.09–0.50$0.77–1.02
best at $0.25above every paired Sonnet seed on all three
final score at $3swarm stops at 60 min, about $1the same finals, in 7–42 minutes

Use the swarm when the budget is small or the goal is "good, cheaply". When the last 0.1% matters, the money is there, and wall-clock time counts, a Sonnet loop at the problem's best effort level gets to the same place faster.

When NOT to use this skill

situationuse instead
hundreds of failing tests to drive to greenswarming-to-green
choosing a model or effort for one subagent or a cascadeagent-routing
many independent tasks with checkable outputs, not one program to improvedispatching-subagents (headless runner)
quality only a person or a judge model can rateno scorer, no search: write the scorer first, or use a judged loop
claude.ai chat, where there is no claude binaryorchestrating-chat-agents

Abandon a run mid-way when:

  • status shows rejected_above_max climbing or a best score above anything the problem allows: the scorer has a hole and the search is now optimising the hole;
  • the best score is flat for ten rounds and rounds.jsonl shows every refine swarm parenting the same program: the archive has converged, and more rounds buy nothing;
  • last_quota.seven_day passes about 0.40: the gateway stops at 0.45 anyway.

Earned exceptions

rulebanned whenearned when
writer at effort medium—effort low on a problem already known to be easy (circle packing: low reached 0.999 cheaper)
N=8 proposers per swarmthe problem is new and N=0 has not been triedan N=0 control on this problem lost to N=8 (on circle packing the writer alone nearly tied)
no sandbox prefixthe evaluator imports model-written code on a machine with secrets or networka throwaway container with no credentials, and the user agreed

Setup

  1. Make a problem directory. Copy assets/example/ and replace its four files:
    • problem.md: the task in plain words, including the reference value if known;
    • initial_program.py: a valid, weak starting program;
    • evaluator.py: evaluate(program_path) -> dict of numbers and strings, with the score under score_key (default combined_score) and an error key on failure;
    • config.toml: entry function name, the interface text the models see, budgets.
  2. Score the initial program by hand and read the dict:
    python3 -c "import importlib.util as u; s=u.spec_from_file_location('e','evaluator.py'); e=u.module_from_spec(s); s.loader.exec_module(e); print(e.evaluate('initial_program.py'))"
    
    A score of 0 with no error, or a missing score_key, means the contract is wrong; fix it before spending anything.
  3. Set score_max from a bound you can prove (a known optimum, an area or volume bound), slightly above it. Scores above it count as 0 and are flagged. Leave it unset only when no bound exists, and then re-check the best program independently.
  4. Choose the sandbox. The evaluator imports and runs model-written code.
    • Claude Code on the web with the claude-workspace hub: the hub jail.
      [sandbox]
      prefix = ["sh", "/home/user/claude-workspace/scripts/jail.sh", "-C", "{run_dir}", "--"]
      chown_uid = 65534
      
      and put --run-dir under /tmp/jail/<name>/ (mkdir and chmod 755 it first). Launch through a script file: the hub's jail_route guard refuses a Bash command that names /tmp/jail/ next to python3.
    • Desktop with Docker: prefix = ["docker", "run", "--rm", "--network=none", "-v", "{run_dir}:{run_dir}", "python:3.12-slim"] (add numpy or what the evaluator needs to the image).
    • No sandbox only under the earned exception above.
  5. Check the CLI: which claude (desktop: the app's bundled CLI may not be on PATH; set EVOLVE_CLAUDE_BIN). Then run the offline selftest:
    python3 scripts/test_evolve.py      # three "ok" lines, about 3 s, no model calls
    

Run

S=/mnt/skills/user/evolving-with-swarms/scripts
python3 $S/evolve.py run problem/config.toml --run-dir RUN --seed 1 --cap 1.0 --wall 60 > RUN.log 2>&1

Launch it with Bash run_in_background: true and end the turn. The completion line is done -> RUN/summary.json; relaunching the same command resumes from RUN/archive.json and keeps the spent dollars. Check in with python3 $S/evolve.py status RUN. In Claude Code on the web a background task is stopped after two hours: keep --wall under 110 minutes per launch, or relaunch on the completion notice.

knob ([swarm], [budget])defaultmeasured
K, N4, 8K=2,N=16 and K=8,N=4 did no better on circle packing; N=0 nearly tied
proposer_effortlowthe floor; Haiku still thinks on about a third of calls
writer_effortmediumlow helped on circle packing, hurt on hexagon and Heilbronn
writer_modelhaikusonnet (one seed): 99% at $0.11 on circle packing, between swarm and loop
cap_usd, wall_minutes1.0, 60the swarm spent $0.9–1.4 an hour on 4 vCPUs

On a new problem, run the default and an N=0 control at the same cap before a long run.

Reading results

RUN/summary.json: programs, best_score, spent, usd_to (dollars when a program first scored 0.99, 0.999, 0.9999), calls_by_role, rejected_above_max, duplicates, no_code, last_quota. RUN/best_program.py is the top program. events.jsonl has one line per evaluation with the spend at that moment, which is the curve to plot; rounds.jsonl has each swarm's parents and ideas.

Common failure modes

  • best_score far above the reference, or rejected_above_max > 0 → the scorer accepts something it should not. Read the offending program in RUN/r*/s*/prog.py, fix the evaluator or tighten score_max, and start a new run: the archive is poisoned.
  • Every refine swarm in rounds.jsonl parents the same broken program → a scorer hole made it the leader; same remedy. Diagnosed 2026-11-09: a hexagon run spent its last 40 minutes refining a program the scorer had wrongly rated 3.9e9.
  • no_code climbing → the writer's reply has no fenced block defining entry. Check that entry in the config matches the interface text and the initial program.
  • duplicates close to the number of writer calls → the writer returns its parent unchanged; usually a writer that cannot see why the score is low. Add the useful metrics to show_metrics, or raise writer_effort.
  • Evaluation errors with rc=125 or rc=2 → the jail could not isolate (125) or nobody cannot enter the run directory (2). Put the run dir under /tmp/jail/<name>/ and set chown_uid = 65534.
  • Spend per program well above a cent → the gateway's ledger shows the cost by role; proposers were 30–39% of the spend in the 2026-11 ledgers, the writer the rest. Each claude -p call carries about 1,370 tokens of the CLI's own, so many tiny calls cost more than their prompts suggest. Fewer proposers (or N=0) and the writer's effort are the levers.
  • pauses.jsonl appears and the run slows → the account's five-hour window passed 90% (non-Haiku calls) or 97% (Haiku). The gateway waits for the reset and leaves the wait out of the run clock; nothing to do unless it is urgent.
  • A run stops with stopped_by starting quota: → seven-day utilisation reached 0.45. Stop launching runs this week, or raise week_stop in gateway.py deliberately.

Verification

python3 $S/test_evolve.py                         # offline: run, resume, status, score bound, sandbox prefix
python3 $S/evolve.py status RUN                   # programs > 1, spent <= cap + one round, best_score <= score_max

Re-score RUN/best_program.py with the evaluator yourself (in the sandbox) and compare it with best_score. Programs that optimise with randomness or a time budget re-score lower; two of three hexagon programs did on 2026-11-09. A bad success is a high score the evaluator should not have given: check the best program against an independent condition (a stricter geometric test, a known bound) before reporting a record.

Diagnosed failures

  • 2026-11-09, hexagon packing. OpenEvolve's stock hexagon evaluator scored an outer side of 1e-9 at 3.9e9: an absolute tolerance on an unnormalised cross product. Signal: a score nine orders above the reference. Fixed by an area bound; score_max exists for this.
  • 2026-11-09, cost accounting. claude -p wrote prompt cache at the 1-hour TTL (2x input) until FORCE_PROMPT_CACHING_5M=1; the gateway sets it. Signal: cacheCreationEphemeral1hInputTokens non-zero in the ledger.
  • 2026-11-09, effort. MAX_THINKING_TOKENS=0 and alwaysThinkingEnabled: false did not stop Haiku 5.5 from thinking, and an output cap below its need failed the call while billing it. Effort low is the floor; do not set max_tokens tight.
  • 2026-11-09, usage window. Ten parallel runs used most of a five-hour window in about 70 minutes and the old gateway aborted runs at 90%. It now pauses and resumes instead.

gating

A gate is a check that blocks. Its only job is to go red when it should.

The characteristic failure is not a wrong check — a wrong check gets noticed. It is a check that cannot fail, which reports PASS forever and is indistinguishable from a working one from the outside. That is what makes this different from ordinary testing: the object under suspicion is the check.

When NOT to use this skill

Scope is ONE check and whether it can be made to fail.

SituationUse
Sequence several steps with branches and retriesflowing
Find correctness bugs with generated inputs against a reference modelbuilding-property-tests (its step 5 calibrates with scripts/mutate.py from here)
Run the repo's existing suiterun it
Decide what to test at allthis skill has no opinion; that is design

A gate is a thing that goes red. If nothing here can go red, there is no gate to audit.

The three obligations

Every gate owes these. A gate missing any of them is not yet a gate.

1. An anchor outside your own code. Something the check compares against that your implementation did not produce: a published constant, a closed-form answer, a conservation law, a degenerate case with a known result, an independent implementation. A check that compares this run to the last run only ever tells you the code still does what it did. See references/anchors.md.

2. A known-bad it demonstrably rejects. Break the subject the way it would plausibly break, run the gate, confirm red. Until you have done this you have not shown the gate works — you have shown it runs. This is the obligation people skip, because a passing gate feels like evidence.

Two things about known-bads that are easy to get wrong:

  • Validate it at the configuration it will run in. A case tuned on a small or fast setting can stop being bad at full size. An "untrained" grid built from one Lloyd iteration was genuinely zero-gain at m=2/K=16 and earned a real +0.10 dB at m=8/K=65536, where one iteration relocates ~63,000 empty cells toward the mode. It passed the fast gate and certified nothing about the real one. This matters more than it sounds, because mutate.py needs a fast gate variant and it is tempting to validate everything there.
  • Measure its reach. One known-bad is the floor, not the goal. Name which checks it exercises (known_bad(..., covers=(...))); the harness prints the checks no known-bad reaches. An audited gate had a single known-bad covering 1 of 8 checks — and the check its whole result rested on accepted the same bad case.

3. A written statement of what it cannot catch. Coverage holes are invisible from inside a green run: the gate is silent about the thing it does not look at, in exactly the same tone it uses for the thing it looked at and approved. The author has to assert the hole; nothing else will.

scripts/gate.py enforces obligations 2 and 3 mechanically — it returns exit code 2 (INCONCLUSIVE, not PASS) when a gate registers no known-bad or no coverage limit.

Building a gate

Work in this order. The first step is the one that determines whether the rest is worth anything.

Name the wrong conclusion, not the component. Not "check the quantizer is correct" but "prevent shipping scalar wins at high bit rates when that would really be an optimizer artifact." A gate aimed at a conclusion knows what counts as a near-miss; a gate aimed at a component just exercises the code.

Find an anchor. references/anchors.md lists the kinds, in rough order of strength, with the questions that find each one.

Prefer brackets to point checks. Assert a value lies strictly between two things it cannot legitimately pass: better than a baseline, worse than a theoretical bound. A one-sided check passes for a result that collapsed as readily as for one that is right — which is how an implementation that silently does nothing gets certified.

Derive the tolerance from measured noise. Run the thing several times, see how much it moves, put the threshold outside that. A tolerance picked for comfort tends to land wider than the defect you are trying to catch, and then it swallows it.

Then check it is not too tight to mean anything. The opposite failure is real and less obvious: a margin can be statistically impeccable and practically empty. A paired estimator — scoring both arms on one shared sample so the common fluctuation cancels — is the right way to measure a difference, and its standard error shrinks as the two arms converge. So "beats the baseline by 3 se" degenerates: a codebook perturbed by N(0, 1e-3) gained +0.0001 dB against a 3-se margin of 1.2e-06 and was accepted, while real ones gained 0.35–1.41 dB. The check certified the effect is real, not the effect is worth having. Those are different assertions and need different thresholds — and the second one has to come from an anchor (there, a published lattice codebook), never from the estimator, which knows nothing about what magnitude would matter.

Build the known-bad and confirm red. Then run scripts/mutate.py for the failures you did not think of.

Wire it to a non-zero exit and run it before the thing it gates, not after. A gate that runs after the results are written is a report.

Auditing an existing check suite

Given tests, a linter config, a CI job, or a gate someone already wrote, the question is not "do these pass" but "can these fail". Full procedure in references/auditing.md; the fast version:

  • For each assertion, name a concrete input that makes it fail. If you cannot, it is decoration — delete it or fix it.
  • Check each oracle's range against the range you actually operate in. A published table that stops short of your regime is a hole with a green light on it.
  • Run scripts/mutate.py against the code the suite covers. Every survivor is a behaviour nothing checks.
  • Look for assertions whose truth does not depend on the subject at all.
  • Look for tolerances wider than the effect being measured.

Scripts

# harness: refuses to report PASS without a known-bad and a coverage limit
python3 scripts/gate.py           # importable; see the module docstring

# mutation pass: which single-token changes does the gate NOT notice?
python3 scripts/mutate.py --target src/codec.py -- python3 calibrate.py
python3 scripts/mutate.py --target grids.py --max 40 -- pytest -q

mutate.py requires the gate to pass on unmutated code first and refuses to run otherwise, because survivor counts against an already-red gate mean nothing. It restores the file even on interrupt, and uses tokenize so string literals and comments are never corrupted. It is the zero-dependency pass that works against any gate command; once it stops finding survivors, mutmut or cosmic-ray go deeper on Python test suites specifically.

Anti-patterns

Each of these has shipped a wrong result somewhere. They are ordered by how convincingly they impersonate a working gate.

Anti-patternWhy it survives review
Slack wider than the defectA tolerance chosen for comfort. The gate passes the real thing and the broken thing, and reports PASS for both. Derive the threshold from noise, then confirm the known-bad falls outside it.
An oracle with a coverage holePublished anchors end somewhere. If the defect is past the end of the table, the check is structurally incapable of catching it and looks fine. State the range the anchor covers.
An assertion whose truth doesn't depend on the subject"The output has at least N distinct colours" is equally true of an unchanged frame. Prefer differential checks: the state must change when it should, and a toggle applied twice must return to the byte-identical original.
Confirming the check ran, not that it can fail"Invoke it and confirm the step appears in the output" catches a check that was never wired up. It says nothing about a check that is wired up and toothless.
Comparing against your own previous outputRegenerated goldens ratify drift. If the golden came from the code under test, it is a changelog, not an oracle.
A cache keyed on the problem rather than the methodcache[(m, K)] cannot notice that the code producing the value changed. Version-stamp the artifact and delete on mismatch instead of trusting.
A self-matching predicateuntil ! pgrep -f trainer never exits, because the watching shell's own argv contains trainer. Worse, a malformed variant exits immediately and reports the job finished while it runs. Wait on a PID.
A margin that is significant but not meaningfulA threshold derived purely from estimator noise certifies that an effect is real, not that it is worth having — and a paired estimator's noise shrinks as the arms converge, so the margin can approach zero. Pair every noise-derived floor with a magnitude an anchor says would matter.
A strict bracket at an attainable optimumA theoretical bound is often reachable, and reaching it is the best possible outcome. A strict edge then goes red on a perfect result and blocks real work. Ask of each edge whether the subject can legitimately sit exactly there.
A gate written by whatever produced the artifactShared assumptions produce shared blind spots, and the convention both inherited is the one neither questions. Anchors are the defence, because an anchor is the one input the producer did not choose.

Division of labour

This skill is for results and pipelines where the failure mode is a plausible wrong number that would survive a careful read.

UseWhen the risk is
gating (this)A number or empirical result is about to be published or acted on, and a wrong-but-reasonable value would pass unnoticed. Output: a gate that blocks.
challengingAn artifact would draw a specific objection from a skeptical reader — prose, analysis, a recommendation, a diff. LLM judgement against a persona. Output: findings and a SHIP/REVISE/RETHINK verdict.
verifying-claimsDocumentation says something about code that is no longer true. Output: prose-vs-code disagreements.
A test suite / TDDCode you wrote does not behave as specified. Output: red tests.

challenging asks would a careful reader object? gating asks can this check go red? They are complements and they miss different things: an adversarial reviewer will not recompute your constants, and a gate will not notice that your framing is wrong.

A gate checks correctness, and cannot check comparability. If two arms of a comparison are each individually correct but not comparably implemented, no anchor and no mutant will see it: nothing is broken, so nothing goes red. A published ablation reported one transform 11–24× slower than another and carried a caveat saying the number was implementation-bound; both arms passed every correctness check, and the real finding — one arm was a tuned BLAS call and the other an interpreted loop — was found by a human reviewer months of gate-work later. Related: performance claims have no anchor in this framework at all. There is no published constant for how fast your code should be. Wall-clock belongs to benchmarking discipline (matched implementation effort, min-of-trials, stated hardware), not to gating.

One caution about pairing them. A same-model reviewer is an independent context, not an independent reviewer — it shares your priors, so the convention you did not question is the one it will not question either. Where that matters, an anchor beats a reviewer, because an anchor is not negotiable.

References

  • references/anchors.md — kinds of oracle, strongest first, and how to find one when nothing published exists.
  • references/auditing.md — the full "can this fail?" pass over an existing suite, including how to read a mutation report.

Generative Thinking — One Move, Committed

Fixation is the default state. When a generator (human or LLM) has been working on a problem, attention concentrates on the current framing and subsequent ideas tend to be local variations on it. This skill is the interrupt: spend one move stepping sideways, then resume.

The discipline matters more than the move. Pick ONE technique per invocation, run it without second-guessing, and produce explicit reframings or candidate entry points — not a brainstorm list in the same frame.

When to reach for this

Claude activates this skill when:

  • The last 2+ iterations are variations of one idea (fixation signal)
  • The user says "widen this", "think bigger", "I'm stuck", "what am I missing", "too narrow", "break out of"
  • A problem is framed as a binary ("X or Y") and neither option is good
  • An agent loop is producing convergent variations — suspect the frame, not the search depth
  • The user explicitly wants a pre-mortem on the framing (not the plan) before committing

Do NOT activate this skill as a default before every consequential task — fixation is the trigger, not stakes. If the current frame is working, let it work.

Core discipline

Three rules that apply across every move below. Violations make the output ideation-flavored but structurally identical to what came before.

  1. Generation before evaluation. De Bono's core distinction: lateral thinking cares about movement value, not truth value. A provocation is not a proposal. Evaluate only after the move has produced 3+ candidate framings.
  2. One move, committed. Do not rotate through five techniques as a menu. Pick the move that matches the diagnosis, execute it fully, surface what shook loose, then stop. Menus produce noise; commitment produces distance.
  3. Output framings, not ideas. The deliverable of a generative move is usually a new statement of the problem or a new entry angle, not another solution candidate inside the old frame. If the output could have been produced without the move, the move didn't fire.

Stop condition. Stop when the move has produced 3+ non-trivial framings, or a single framing that reorganizes the problem (one sharp surprise beats five adjacent). Do not keep generating because the list looks short — volume is not the product.

The fire test. After every move, ask: could this output have been produced without the move? If yes, the move did not fire. Either commit harder (push the provocation further, make the reframe more aggressive, re-roll the random word, invert on a different axis) or the move was mismatched to the stuck-pattern — re-diagnose and pick the better-matched move. Re-diagnosis after a miss is not menu-rotation; menu-rotation is cycling through techniques without commitment. One move at a time, each one fully, and if it misses, diagnose why before the next.

Tail sift. The fire test with a number attached. Post-trained models carry a typicality bias: asked for one candidate they return the modal one, and asked for a list they return the top-k modes — a bestseller list, not a sample (Zhang et al. 2025). The 3 framings a move produces are subject to this too. So before evaluating, over-generate — 5 to 8 candidates rather than 3 — and write next to each one the probability, 0 to 1, that a generator still inside the old frame would have produced it. Drop everything above the threshold; the remainder is the move's actual output. Start the threshold at 0.3 and lower it if the survivors still read as adjacent. Writing the number is the mechanism, not decoration: prompts that request a distribution with verbalized probabilities recover diversity that prompts for instances or lists do not, quality holds when the candidates are reasoned rather than listed, and lowering the stated threshold moves output further into the tail. The gain scales with model capability; on a small model the sift adds burden without adding distance.

Diagnostic → Move

Match the stuck-pattern to the move. When unsure, default to Reframe.

If the stuck-pattern is…Reach for…
Framing feels forced ("must be X or Y")Reframe — change the verb, subject, scope, or level
Generator keeps returning near-duplicatesRandom stimulus — force an unrelated concept into the frame
Obvious answer is wrong but you can't see past itProvocation (Po) — state something impossible, extract movement
Iterating on an existing artifactSCAMPER — seven structured transforms
Stuck on "how do we make X succeed?"Inversion — ask "how do we guarantee X fails?" then negate
Problem is defined entirely in one domain's vocabularyStructured analogy — map objects and relations by function into a distant domain, search there
Every solution is blocked by a constraintConstraint play — remove it ("assume magic"), or add an absurd one ("must fit in a tweet")
Two known examples, no theory of the space between/beyond themFamily traversal — name the shared family, walk it to its limits

Reframe

The problem-as-stated is rarely the problem-to-solve. Mutate the sentence:

  • Change the verb: "reduce X" → "redistribute X", "time-shift X", "prevent X from mattering"
  • Change the subject: "our team can't ship faster" → "reviewers are the bottleneck" → "the artifact is too reviewable"
  • Change the scope: zoom out (whose problem is this upstream?) or zoom in (which one user, which one minute?)
  • Change the frame: from "problem to fix" to "signal to interpret"

Produce 3 reframings. Fired if: at least one makes the original statement sound naive, or shifts who owns the problem.

Provocation (Po)

Edward de Bono's method. Prefix a deliberately wrong, impossible, or absurd statement with Po: to signal it is not a proposal — it is a stimulus. Then extract movement: what principle, consequence, or adjacent idea does this surface?

Four recipes (de Bono's formal provocations):

  • Escape: remove an essential feature. Po: the database has no writes.
  • Reversal: flip the causal direction. Po: users pay us to NOT use the product.
  • Exaggeration: push a quantity to the absurd limit. Po: onboarding takes six months.
  • Wishful thinking: assume impossible capability. Po: we know what the user will click next week.

The canonical example: a factory pollutes a river. Po: the factory is downstream of itself. Impossible, but it generates: move intake downstream of discharge. Internal incentive to not pollute. Closed-loop water. The provocation is discarded; the movement stays.

Fired if: the provocation is genuinely impossible or absurd (not merely edgy), AND extracting movement yields a principle that survives translation back to the real constraints. If the "provocation" is a thing you could actually do, it's a proposal, not a Po — push it further.

Random stimulus

Pick a word, object, or domain with no connection to the problem. Force a connection. The forced-feel is the point — it routes around the habituated pathway.

Template: "How is [problem] like [random]?" then "What does that suggest?"

Sourcing for humans: a random Wikipedia article, a nearby physical object, an Oblique Strategies card, a concept from an unrelated field on the current desk. Commit to the first thing you land on; re-rolling defeats the method.

Sourcing for an LLM agent: an LLM asked directly for a random word does not produce one — the same attention that locked the frame picks a word adjacent to it, and post-training biases the pick toward whatever is typical. Two sources work:

  • External: ask the user for a word, any word; fetch a random Wikipedia article; draw an Oblique Strategy by tool.
  • Internal, hashed — String Seed of Thought (Misaki & Akiba 2025). Write out a long random string (40+ characters, mixed letters, digits and symbols, no visible pattern). Then write out the reduction — sum the character codes, or compute a rolling hash — modulo the size of a pre-listed vocabulary, and take the word at that index. references/stimulus-vocabulary.md holds 128 nouns from distant domains for exactly this; the Oblique Strategies deck works the same way. Measured: the recipe reaches near-PRNG faithfulness on long-reasoning models, and on open-ended generation it beat both an injected PRNG seed and a random-number tool call, because the string can be re-hashed for several local choices and the derivation is on the page. Two failure modes, both measured: a lazy extraction that reads only the first character (LLM-generated strings have strong positional bias — 947 of 1000 QwQ-32B strings opened with "7"), and skipping the written arithmetic, after which reasoning models hallucinate the result. Use the whole string; write the sum. Models under ~8B cannot execute the reduction reliably — hand them an external source.

Fired if: the connection is genuinely forced (the first 10 seconds feel wrong), and working through the force produces an angle that was not in your prior search space. If the random word feels "relevant" immediately, you re-rolled, picked from attention, or read one character of the seed — get a new one.

SCAMPER

For iterating on an existing artifact. Walk the seven prompts once; do not pick favorites in advance.

  • Substitute — swap a component, material, person, step
  • Combine — merge with something else, including something it competes with
  • Adapt — borrow a mechanism from elsewhere that solves a related problem
  • Modify / magnify / minify — change a dimension, frequency, or weight drastically
  • Put to another use — who else could use this, for what?
  • Eliminate — what happens if this part simply is not there?
  • Reverse / rearrange — swap order, roles, or polarity

Fired if: at least one prompt produced a candidate you would not have reached by asking "what's a better version of this?". If all seven outputs are adjacent polish, the artifact is not the unit of analysis — zoom out and try Reframe.

Inversion

Solve the inverse problem, then negate the solution. Works because failure modes are often more concrete than success paths.

  • "How would I guarantee this fails?" → each failure mode is a protective requirement
  • "What would the worst possible version look like?" → the inverse shape of the good version
  • "If an adversary wanted us to choose X, why?" → surfaces the hidden trap in X

Fired if: inverting surfaced a concrete risk, mechanism, or incentive the forward framing was hiding. If negating the inverted answer gives you the same thing you already had, the inversion was too symmetric — invert on a different axis (goals → incentives, success → unobservable, user → operator).

Structured analogy (perspective shift)

Move the problem into a distant domain by its relational structure, not by asking the distant domain a question. "How would biology solve this?" is the cross-domain baseline in Shen, Druckmann & Zou (2026), and it collapses almost as hard as no domain prompt at all: across 150 generations per problem, about 5% of proposed domains were unique, and solution diversity (Vendi score) was 8.3 against 5.8 for the unconstrained baseline. Explicit structure-mapping scored 90–173% higher on solution diversity, produced solutions judged novel 50–69% of the time against 1.6–38% for the baselines, and reached domains further from the problem. The written mapping is what changes the outcome.

Four steps, after Gentner's structure-mapping theory:

  1. Extract. List the problem's objects with their functional roles, and the relations between them — predicates over two or more objects ("a stationary source releases a substance into a heterogeneous medium"). Attributes of single objects are not mapped.
  2. Map by function. Pick a distant domain and write object ↔ object pairs with the reason each pair holds. "Delivers a payload" is a mapping basis; "is a liquid" is not. Partial coverage is allowed — the analogy needs a subset of the relations, not all of them.
  3. Search the target side. Find an existing, named method in the distant domain that operates on the mapped objects and preserves the shared relations. Search there, not for solutions to the original problem. (Drug delivery ↔ groundwater contamination gave reactive-transport plume models; EEG source localization ↔ seismology gave the double-difference earthquake location algorithm.)
  4. Transfer back. Re-substitute the source objects and state what the method becomes in the original domain.

Domain menu, when step 2 needs a starting point: natural (biology, ecology, geology), trade (kitchen, ER triage, shipping dock), role (CFO, child, historian, adversary), scale (100x, 1/100x).

Fired if: the mapped domain is one the problem's own literature would not cite, AND step 3 returned a named, existing method — not a metaphor. If the object pairs match by resemblance rather than role, or the "method" is the original problem restated in new nouns, the mapping was surface-level; redo step 2 with a further domain.

Constraint play

Constraints define the solution space. Move them deliberately.

  • Remove: "assume infinite budget / time / compute / permission" — what opens up?
  • Add absurd: "must fit in a tweet", "under $5", "no code", "one meeting only", "physical-only"
  • Invert: the constraint becomes the feature. Limited time → event-driven. Small budget → tiny team as the pitch.

Fired if: a relaxed-constraint solution reveals what you actually value (not just what you'll accept), or an added-constraint solution is sharper than the unconstrained one. If both feel like the same answer with a different budget, the binding constraint is elsewhere — find it.

Family traversal

For when you hold two (or more) related instances and generation keeps orbiting them. The pair is not the object — the parametrized family containing both is. Sub-moves in descending observed yield:

  1. Name the face. State the family or frontier both instances lie on, and its parameter. (Two embedding quantizers → the rate–distortion frontier, parametrized by bits per dimension.) If you cannot name the parameter, you have not found the family yet.
  2. Walk to the limits. Continue past both anchors to the family's extreme points — limits are where members change character. (The 0-bit limit of vector search is ranking by document prior; the zero-distortion limit is exact symbolic addressing — which revealed two "unrelated" projects as endpoints of one frontier.)
  3. Swap the bound object. Name the supporting constraint that walls the family — the bound that cannot be crossed. Since crossing is impossible within the model, change what the constraint binds. (Cannot beat rate–distortion for embeddings → quantize the weights, or the knowledge, instead.)
  4. Probe the chord. Mixtures and blends of the anchors (dithering, ensembles, fusion weights). Lowest yield — blends average rather than extrapolate — but cheap. Run last, not first.

Then sharpen and verify: the traversal's real product is questions precise enough to have derivable answers; the discovery happens in the derivation and a cheap measurement, not in the geometry. (A frontier framing of an addressing scheme implied a capacity law, j² ≤ 2^(significand bits); sixty seconds of numpy confirmed a cliff at exactly N = 4096, closing a four-month-old empirical mystery.) Without this step the move outputs taxonomy, not generation.

Fired if: a limit point or constraint-swap landed outside the prior search space, AND at least one output is a checkable claim. If the output is only a tidy classification of the anchors, the move stalled — push further along the edge or swap the constraint.

Caution: do not equate the interior of the space with novelty. In high dimensions essentially all operation is already extrapolation outside the training hull (Balestriero & LeCun 2021), and mixtures of known points are averages. The generative directions are the limits and the constraint-swaps; the chord is a probe, not a doctrine.

Applying the skill to an agent's own reasoning

LLM agents exhibit a context-bound analog of functional fixedness: attention concentrates on current framing and generates variations of it. A second mechanism compounds it and does not depend on context: preference data favors familiar text, so post-training sharpens the policy toward the modal continuation for any prompt (Zhang et al. 2025, Theorem D.1) — which is why "give me five alternatives" returns five near-neighbors even in a fresh context. Signals this is happening:

  • The nth iteration has the same structure as the first
  • The agent has rejected the same class of option three times with similar reasoning
  • The plan has a step labeled "brainstorm" that is producing adjacent bullets

When detected, the fix is the same: pick one move from the diagnostic table, execute it on the agent's own current framing, and explicitly write out the new framing(s) before resuming work. The write-out is essential — a framing that stays implicit in attention gets re-absorbed into the previous frame.

What this skill does NOT do

  • Does not evaluate. Pair with challenging after generation if the artifact is high-stakes.
  • Does not do MECE coverage. Use tiling-tree for exhaustive partitioning.
  • Does not synthesize multiple viewpoints. Use convening-experts for collaborative multi-role panels.
  • Does not produce long idea lists. A three-reframe output with one surprising move beats a fifty-item brainstorm in the original frame.

Canonical references (progressive disclosure)

Load these only when the user wants depth on a specific technique.

  • Lateral thinking, provocation, Po, random stimulus — Edward de Bono. Primary: Wikipedia: Lateral thinking, Wikipedia: Po. Essay: de Bono, "Serious Creativity". Books: Lateral Thinking (1970), Serious Creativity (1992).
  • SCAMPER — Bob Eberle, SCAMPER: Games for Imagination Development (1971), built on Alex Osborn's Applied Imagination (1953) checklist. Summary: Wikipedia: SCAMPER.
  • Oblique Strategies — Brian Eno & Peter Schmidt (1974–2001, five editions). Summary: Wikipedia: Oblique Strategies. Full deck: mattrickard.com/list-of-all-oblique-strategies. Draw one at random when reaching for random stimulus.
  • Functional fixedness (the cognitive bias this skill counters) — Karl Duncker, originally Zur Psychologie des produktiven Denkens (1935); English translation On Problem-Solving (1945), doi:10.1037/h0093599. The candle problem is the canonical demonstration.
  • Extrapolation vs interpolation in high dimensions (why "interior = novelty" is wrong) — Balestriero, Pesenti & LeCun, "Learning in High Dimension Always Amounts to Extrapolation," arXiv:2110.09485.
  • Typicality bias and mode collapse; verbalized-probability prompting (the tail sift) — Zhang, Yu, Chong, Sicilia, Tomz, Manning & Shi, "Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity," arXiv:2510.01171. Distribution-level prompts with probabilities raised creative-writing diversity 1.6–2.1× over direct prompting; a threshold in the prompt tunes how far into the tail; gains grow with model scale.
  • String Seed of Thought (the hashed internal random stimulus) — Misaki & Akiba, Sakana AI, "String Seed of Thought: Prompting LLMs for Distribution-Faithful and Diverse Generation," arXiv:2510.21150. Total-variation distance to the target distribution shrinks with string length even under autoregressive correlation (Thm 4.2); §D.5 for the first-character failure, §D.6 for the comparison against injected seeds and tool calls.
  • Structure-mapping theory of analogy — Gentner, "Structure-Mapping: A Theoretical Framework for Analogy," Cognitive Science 7(2), 1983, doi:10.1207/s15516709cog0702_3. Relations map, attributes do not.
  • Analogical reasoning as a diversity engine (the structured-analogy procedure) — Shen, Druckmann & Zou, Stanford, "Unlocking LLM Creativity in Science through Analogical Reasoning," arXiv:2605.11258. Extraction and search prompts in §G.1; four AR-generated methods implemented and benchmarked in biomedicine.
  • Design fixation and generative-AI specific failure modes — Wadinambiarachchi, Kelly, Pareek, Zhou & Velloso, "The Effects of Generative AI on Design Fixation and Divergent Thinking," CHI 2024, paper 380, doi:10.1145/3613904.3642919 (arXiv:2403.11164). N=60 visual ideation experiment: participants with AI image-generator support showed higher fixation on the initial example, and lower fluency, variety, and originality than the no-support baseline. Directly relevant when the fixation is coming from an LLM's own prior outputs.

Quick-reference card

DIAGNOSE: What kind of stuck?
  → framing forced        : REFRAME
  → near-duplicates       : RANDOM STIMULUS
  → can't see past obvious: PROVOCATION (Po)
  → iterating an artifact : SCAMPER
  → chasing success       : INVERSION
  → one domain vocabulary : STRUCTURED ANALOGY
  → blocked by constraint : CONSTRAINT PLAY
  → two examples, no theory: FAMILY TRAVERSAL

DISCIPLINE:
  1. Generation before evaluation
  2. One move, committed
  3. Output framings, not ideas
  TAIL SIFT: over-generate 5-8, tag P(old frame reaches it), drop > 0.3

STOP when: 3+ non-trivial framings produced, or one surprising framing that reorganizes the problem.

Invoking Antigravity

Run Google's Antigravity CLI (agy) headless, then call agy -p "<task>" as an orchestrated sub-agent on Google's harness (Gemini 3.5 Flash).

Install, authenticate once, orchestrate. For the why behind any step — the 30 s timeout, the keyring fallback, the token format, network hosts, troubleshooting — read references/auth-internals.md.

When to use

  • Orchestrating agy as a sub-agent: shell out, capture stdout, fold back.
  • Running Antigravity agents from a script, CI, or sandboxed container.
  • Wanting a second opinion from Google's harness alongside Claude.

For interactive local use, skip this — run agy directly and it opens a browser.

1. Install

curl -fsSL https://antigravity.google/cli/install.sh | bash

Idempotent, ~3 s, installs ~/.local/bin/agy.

2. Authenticate — once, human-in-the-loop

Export these in every shell that runs agy, so it stores the token in a file rather than an OS keyring:

export SSH_CONNECTION="203.0.113.1 50000 203.0.113.2 22"
export SSH_CLIENT="203.0.113.1 50000 22"
export SSH_TTY="/dev/pts/0"

Run the broker, then relay the login to a human:

python3 scripts/agy_auth_broker.py &      # spawns agy, captures the OAuth URL
sleep 15 && cat /tmp/agybroker/url         # a human opens this and consents
printf '<code>' > /tmp/agybroker/code      # paste the authorization code back

agy writes the token to ~/.gemini/antigravity-cli/antigravity-oauth-token; later agy -p calls run silently. Complete the login within a few minutes — an idle container pause kills the broker's agy process.

3. Orchestrate

agy --dangerously-skip-permissions -p "<task>"

Place every flag before -p — it treats the next argument as the prompt. Add --add-dir <path>, --print-timeout 10m, or --conversation <id> as the task needs.

Reuse auth on a fresh container

Save ~/.gemini/antigravity-cli/antigravity-oauth-token and write it back (with the SSH vars set) — its refresh_token is durable, so no repeat OAuth. Keep the file out of git and logs; it is a personal Google credential.

Invoking Gemini

Delegate tasks to Google's Gemini models when they offer advantages over Claude.

When to Use Gemini

Image generation:

  • Blog header images, illustrations, diagrams
  • Style-guided image creation (risograph, editorial, etc.)
  • Text rendering in images

Speech (TTS):

  • Narration, voice-over, read-aloud with style direction per line
  • A custom voice designed from a written description
  • Two-speaker dialogue

Structured outputs:

  • JSON Schema validation with property ordering guarantees
  • Pydantic model compliance
  • Strict schema adherence (enum values, required fields)

Cost optimization:

  • Parallel batch processing (Gemini 3 Flash is lightweight)
  • High-volume simple tasks

Multi-modal tasks:

  • Image analysis with JSON output
  • Video processing
  • Audio transcription with structure

Setup

uv pip install requests pydantic

Credentials — Option A (recommended): Cloudflare AI Gateway

Source /mnt/project/proxy.env with CF_ACCOUNT_ID, CF_GATEWAY_ID, CF_API_TOKEN. Requests route through Cloudflare AI Gateway, bypassing IP blocks. Google API key stored in gateway via BYOK.

Credentials — Option B: Direct Google API

If no proxy.env, falls back to direct: GOOGLE_API_KEY.txt or API_CREDENTIALS.json.

Image Generation

Generate images using Gemini's native image models. This is the primary way to create illustrations, blog headers, diagrams, and visual content.

Quick Start

import sys
sys.path.append('/mnt/skills/user/invoking-gemini/scripts')
from gemini_client import generate_image

# One call — returns {"path": "...", "caption": "..."} or None
result = generate_image("A watercolor painting of a mountain lake at sunset")
print(result["path"])  # /mnt/user-data/outputs/gemini_image_1740000000.png

Function Signature

generate_image(
    prompt: str,                    # The image description
    output_path: str = None,        # Auto-generates if omitted
    model: str = "nano-banana-2.1",  # Default: fast. Use "image-pro" for quality
    temperature: float = 0.7,       # 0.5-0.7 for diagrams, 0.7-0.8 for illustrations
) -> dict | None
# Returns: {"path": "/mnt/user-data/outputs/gemini_image_*.png", "caption": str|None}
# Returns None on failure

Model Selection

AliasModelBest ForCost/image
"nano-banana-2.1" or "image"gemini-nano-banana-2.1Fast iteration, drafts, text rendering (default)$0.034 (1K) – $0.113 (4K)
"nano-banana-2"gemini-3.1-flash-imagePrevious version of the default; use only to reproduce earlier results$0.067 (1K)
"nano-banana-2-lite"gemini-3.1-flash-lite-imageLowest latency, and 6x cheaper than the default on text and reference-image input; same 1K image price$0.034 (1K)
"image-pro" or "nano-banana-pro"gemini-3-pro-imageHighest fidelity, published content$0.134 (1K/2K)

Complete Blog Header Example

import sys
sys.path.append('/mnt/skills/user/invoking-gemini/scripts')
from gemini_client import generate_image

# 1. Compose prompt with style prefix + subject
style_prefix = (
    "Style: Risograph-inspired editorial illustration. "
    "Visible halftone dot texture and slight color misregistration between layers. "
    "Limited ink palette: deep indigo, warm coral, and sage green on off-white paper. "
    "Layered transparency where colors overlap creates rich secondary tones. "
    "Modern and professional — the aesthetic of an indie design studio, not a fantasy novel. "
    "Generous whitespace. No photorealism, no glow effects, no cyberpunk. No text or labels."
)
subject = "A raven perched on a stack of books, observing a network graph"
prompt = f"{style_prefix}\n\nSubject: {subject}. Wide landscape format, suitable as a blog header."

# 2. Generate (use image-pro for published content)
result = generate_image(prompt, model="image-pro", temperature=0.75)

if result:
    print(f"Saved: {result['path']}")
    # 3. Present to user
    # present_files([result["path"]])

Prompt Patterns

  • Style prefix + subject: Prepend a style description, then describe the subject
  • Be specific about style: "Risograph-inspired editorial illustration" not "a nice picture"
  • Include composition: "Wide landscape format" / "centered, high contrast"
  • Text rendering: "A poster with the text 'SALE' in bold red letters" (works well with image-pro)
  • Negative constraints: "No photorealism, no glow effects" to avoid defaults

Custom Output Path

result = generate_image(
    "A logo for a coffee shop called 'Bean There'",
    output_path="/mnt/user-data/outputs/coffee_logo.png"
)

Speech Generation (TTS)

Gemini 3.8 Flash TTS and Flash-Lite TTS went GA on 2026-09-23. Output is WAV, 24 kHz mono 16-bit, SynthID-watermarked.

from gemini_client import generate_speech, design_voice, list_voices

r = generate_speech("Odin kept two ravens. <short pause> Huginn was thought.",
                    output_path="/tmp/line.wav", voice="Algenib",
                    style="quiet and dry, unhurried")
# {'path': '/tmp/line.wav', 'seconds': 4.2, 'audio_tokens': 135} or None

v = design_voice("A low, dry, quietly amused male voice with a faint rasp. "
                 "Soft southern British accent.", "narrator", gender="male",
                 language_code="en-GB")      # {'id': 'voice_...', 'sample_path': ...}
generate_speech("...", voice=v["id"])

lows = list_voices(gender="male", pitch="low")   # library of 2,089 prebuilt voices
  • Voices: 30 studio voices (Charon, Kore, Algenib "gravelly", Enceladus "breathy", ...) plus 2,059 persona voices with ids like en-gb-storyteller-4. list_voices() returns accent, pitch, gender and a description for each; the accent filter needs the exact string ("Winchester English"), so filter accents on the returned field.
  • Style: pass style= (a speech_metadata annotation). Do not prefix the text with "Style: ..." — the 3.8 models read the prefix aloud, and systemInstruction is rejected. Inline events go in the text: <laugh>, <sigh>, <breath>, <short pause>; CAPITALS stress a word.
  • Designed voices are stored (1-year expiry, 200 per project). The description sets baseline delivery too: "thoughtful pauses" in it produced 1–1.8 s mid-line pauses that no per-line style removed.
  • The model can change words. Seen in a 29-line narration: "Hmm, I get things wrong", "tell them" for "tell him". Anything with subtitles or a fixed script needs an ASR check (faster-whisper medium.en) and a retake.
  • Cost: about 32 audio tokens per second of speech, $9/M through 2026-12-31 on 3.8 Flash TTS ($6/M Lite), so a minute is about $0.02.
  • Voice replication (cloning from a 30 s sample plus a recorded consent clip) is not wired in, and is unavailable in the EEA, UK, Switzerland, India, Illinois and Texas.

Basic Text Usage

import sys
sys.path.append('/mnt/skills/user/invoking-gemini/scripts')
from gemini_client import invoke_gemini

response = invoke_gemini(
    prompt="Explain quantum computing in 3 bullet points",
    model="flash",  # gemini-4-hydrogen (default)
)
print(response)

Structured Output

Use Pydantic models for guaranteed JSON Schema compliance:

from gemini_client import invoke_with_structured_output
from pydantic import BaseModel, Field

class BookAnalysis(BaseModel):
    title: str
    genre: str = Field(description="Primary genre")
    key_themes: list[str] = Field(max_length=5)
    rating: int = Field(ge=1, le=5)

result = invoke_with_structured_output(
    prompt="Analyze the book '1984' by George Orwell",
    pydantic_model=BookAnalysis
)
print(result.title)  # "1984"

Nested models are supported. Gemini's responseSchema rejects $ref/$defs, which pydantic emits for every nested model, so the client inlines them before sending:

class Finding(BaseModel):
    claim: str
    confidence: Literal["high", "medium", "low"]
    note: str | None = None

class Analysis(BaseModel):
    findings: list[Finding]     # nested — inlined for you
    gaps: list[str]

Budget output generously. Thinking tokens count against max_output_tokens (default 32768). Too low and the JSON truncates mid-object, which surfaces as a pydantic parse error rather than a length error — the client now detects finishReason=MAX_TOKENS and says so explicitly.

Parallel Invocation

from gemini_client import invoke_parallel

results = invoke_parallel(
    prompts=["Summarize Hamlet", "Summarize Macbeth", "Summarize Othello"],
    model="lite",  # gemini-3.5-flash-lite — cheap/fast tier for batch
)

Available Models

The current generation is Gemini 4 (Argon 2026-10-13, Carbon 2026-11-10, Hydrogen 2026-11-12, Neon 2026-11-18). gemini-4-hydrogen is the default and the flash alias, following Google's routing guide (ai.google.dev/gemini-api/docs/choosing-a-model): Hydrogen for everyday work, Neon for harder high-frequency work, Carbon for the hardest problems, 3.5 Flash-Lite for bulk text.

No Gemini 4 model is called "Pro", and this skill does not use the old gemini-3.1-pro-preview: it costs several times as much as Hydrogen for less benefit on coding and agentic tasks. The pro alias therefore resolves to Hydrogen, and "maximum reasoning" means thinking_level='xhigh' on Hydrogen. Neon and Carbon are not the default because they cost more (output price: Neon 2x Hydrogen; Carbon 1.3x, rising to 2.7x on 2026-12-01). Ask for them by name when a task needs the extra reasoning.

Older model IDs and aliases (listed in references/models.md) are rewritten to a current model, with a note on stderr.

Text / Reasoning Models

ModelAliasInput/1MOutput/1MThinking levels (default)Notes
gemini-4-hydrogenflash, pro, hydrogen$1.50$7.50low, medium, high (default), xhighDefault. Released 2026-11-12. Fast, has a free tier. Google reports >5x fewer fabricated claims than 3.8 Flash.
gemini-4-neonneon$3.00$15.00low, medium, high (default), xhighReleased 2026-11-18. Between Hydrogen and Carbon: deeper reasoning than Hydrogen, faster than Carbon. Paid tier only.
gemini-4-carboncarbon$2.00 → $4.00$10.00 → $20.00low, medium (default), highReleased 2026-11-10. Strongest reasoning; for hard debugging, architecture, proofs, security. Price doubles on 2026-12-01. Rejects xhigh.
gemini-4-argonargon$2.00 → $4.00$10.00 → $20.00low, medium, highReleased 2026-10-13. Reasoning close to Carbon with fewer tokens per turn, same price. Rejects xhigh.
gemini-3.8-flashflash-3.8$0.75 → $1.50$3.75 → $7.50low, medium (default), highPrevious-generation Flash, for callers that need to stay on it. Nothing falls back to it automatically. Intro price runs through 2026-12-31.
gemini-3.5-flash-litelite$0.30$2.50minimal (default), low, medium, highBulk tier. Lowest-cost general model; only text model here that accepts minimal.

All Gemini 4 models: 1M unified context, 64K default output (up to 1M with long decode), implicit caching only. → marks Carbon/Argon's introductory price through 2026-11-30 and 3.8 Flash's through 2026-12-31. Output prices include thinking tokens. Google's Batch API is half of standard price and long decode extends output to 1M tokens; this client exposes neither.

Image Models

ModelAliasInput/1MPer Image
gemini-nano-banana-2.1image, nano-banana-2.1$1.50 (text/image input)$0.034 (1K) / $0.050 (2K) / $0.113 (4K)
gemini-3.1-flash-imagenano-banana-2$0.50$0.067 (1K)
gemini-3.1-flash-lite-imagenano-banana-2-lite$0.25$0.034 (1K)
gemini-3-pro-imageimage-pro, nano-banana-pro$2.00$0.134 (1K/2K)

"Input/1M" is the price of the prompt and any reference images. The per-image price is set by image output tokens ($30/M on the 2.1 and Lite models), so a model with a higher input price can still cost less per image. Both tables describe the same models; the first is for choosing, this one for pricing.

Speech Models

ModelAliasInput/1MOutput/1M (audio)Notes
gemini-3.8-flash-ttstts$0.50 → $1.00$9.00 → $18.00GA 2026-09-23. Expressive, 130 languages, 2 speakers. Use generate_speech(), not invoke_gemini().
gemini-3.8-flash-lite-ttstts-lite$0.50 → $1.00$6.00 → $12.00GA 2026-09-23. Bulk / read-aloud, 101 languages. Replaces gemini-3.1-flash-tts-preview ($1 / $20).

Speech aliases live in SPEECH_ALIASES, not MODEL_ALIASES, so a text call can never resolve to an audio model.

See references/models.md for full details.

Thinking Budget (Gemini 3.x and 4)

Gemini 3.x and 4 models reason before responding. Set it with the string thinking_level ∈ {minimal, low, medium, high, xhigh} (integer thinking_budget is not supported). Defaults: high on Hydrogen and Neon, medium on Carbon, Argon and 3.8 Flash, minimal on 3.5 Flash-Lite. A model that thinks by default will silently spend output tokens on reasoning (symptom: empty response with finishReason=MAX_TOKENS), so budget max_output_tokens generously or lower the level.

Only lite accepts minimal. Every Gemini 4 model and 3.8 Flash return HTTP 400 (Thinking level MINIMAL is not supported for this model), so low is their floor. Carbon and Argon also reject xhigh. The client downgrades minimal to low and xhigh to high on those models and prints a note to stderr, so a request never fails on this (verified live 2026-11-20). For a true no-thinking pass, use lite.

response = invoke_gemini(
    prompt="Transcribe this image.",
    model="lite",              # only lite takes 'minimal'
    image_path="/tmp/screenshot.png",
    max_output_tokens=4000,
    thinking_level="minimal",  # don't burn output budget on reasoning
)

Error Handling

response = invoke_gemini(prompt="...", model="flash")
if response is None:
    print("API call failed — check credentials")

result = generate_image("...")
if result is None:
    print("Image generation failed — check credentials or try again")

Common issues: Missing API key → see Setup. Rate limit → auto-retries with backoff. Network error → returns None.

Advanced Features

Custom Generation Config

response = invoke_gemini(
    prompt="Write a haiku",
    model="flash",                  # gemini-4-hydrogen
    temperature=0.9,
    max_output_tokens=200,
    top_p=0.95,
    thinking_level="low",           # Hydrogen thinks at "high" by default; "low" keeps the 200-token budget for the answer
)

Multi-modal Input

from pydantic import BaseModel
from gemini_client import invoke_with_structured_output

class ImageDescription(BaseModel):
    objects: list[str]
    scene: str
    colors: list[str]

result = invoke_with_structured_output(
    prompt="Describe this image",
    pydantic_model=ImageDescription,
    image_path="/mnt/user-data/uploads/photo.jpg"
)

See references/advanced.md for more patterns.

Troubleshooting

"No credentials configured": Create /mnt/project/proxy.env with CF credentials, or add GOOGLE_API_KEY.txt.

CF Gateway 401/403: Verify CF_API_TOKEN has AI Gateway permissions. If not using BYOK, add GOOGLE_API_KEY to proxy.env.

Import errors: uv pip install requests pydantic

Image generation returns None: Check credentials. If persistent, try model="nano-banana-2" (more reliable than image-pro). Check for content policy blocks in error output.

LLM-as-Computer: Compiled Transformer Stack Machine

A working computer built from transformer primitives. Every instruction fetch and stack read is a parabolic attention head (dot-product → argmax → value extraction). The transformer's weights ARE the interpreter — compiled analytically, not trained.

What This Proves

Attention is lookup; feed-forward is routing. A vanilla transformer with compiled weights can execute arbitrary programs: loops, recursion, arithmetic, memory access. 55 opcodes covering WASM i32 semantics. 21M+ steps/second via the Mojo executor.

Setup (once per session)

cd /mnt/skills/user/llm-as-computer/src && bash setup.sh

This installs Mojo (~20s) and compiles the executor binary (~6s). If Mojo is unavailable, the skill falls back to a pure-Python executor (slower but functional).

Usage

import sys
sys.path.insert(0, '/mnt/skills/user/llm-as-computer/src')

from programs import make_fibonacci, make_factorial, make_gcd, make_multiply
from runner import run, setup

# Ensure Mojo is compiled (idempotent)
setup()

# Run a program — shows instructions, trace, result
prog, expected = make_fibonacci(10)
print(run(prog))

# Benchmark mode — measures throughput
print(run(prog, benchmark=True, repeat=200))

Available Programs

From programs.py — all return (program, expected_result):

GeneratorDescriptionExample
make_fibonacci(n)Iterative fib via SWAP+OVER+ADD+ROTfib(10)=55, 111 steps
make_multiply(a, b)Repeated additionmul(7,8)=56
make_factorial(n)Loop with MULfact(8)=40320
make_gcd(a, b)Euclidean algorithmgcd(48,18)=6
make_power_of_2(n)Repeated doubling2^7=128
make_sum_1_to_n(n)Accumulation loopsum(15)=120
make_is_even(n)Parity checkis_even(7)=0
make_native_multiply(a,b)Single MUL opcode
make_native_divmod(a,b)DIV_S + REM_S
make_compare_binary(op,a,b)eq/ne/lt_s/gt_s/le_s/ge_s
make_bitwise_binary(op,a,b)and/or/xor/shl/shr_u/rotl/rotr
make_select(a,b,c)Conditional select

Writing Custom Programs

from isa_lite import program

# Assembly tuples
prog = program(
    ('PUSH', 10),
    ('PUSH', 20),
    ('ADD',),
    ('DUP',),
    ('ADD',),  # (10+20)*2 = 60
    ('HALT',),
)
print(run(prog))

# Loops: countdown from 5
prog = program(
    ('PUSH', 5),    # 0: counter
    ('PUSH', 1),    # 1: decrement
    ('SUB',),       # 2: counter - 1
    ('DUP',),       # 3: copy for JNZ test
    ('JNZ', 1),     # 4: loop if non-zero
    ('HALT',),      # 5: done, top = 0
)
print(run(prog))

ISA Reference (55 opcodes)

Stack: PUSH n, POP, DUP, SWAP, OVER, ROT Arithmetic: ADD, SUB, MUL, DIV_S, DIV_U, REM_S, REM_U Comparison: EQZ, EQ, NE, LT_S/U, GT_S/U, LE_S/U, GE_S/U Bitwise: AND, OR, XOR, SHL, SHR_S/U, ROTL, ROTR Unary: CLZ, CTZ, POPCNT, ABS, NEG, SELECT Control: JZ addr, JNZ addr, CALL addr, RETURN, HALT, NOP Locals: LOCAL.GET idx, LOCAL.SET idx, LOCAL.TEE idx Memory: I32.LOAD, I32.STORE, I32.LOAD8_U/S, I32.LOAD16_U/S, I32.STORE8, I32.STORE16

All arithmetic is 32-bit signed with WASM i32 semantics (wrap on overflow).

Architecture

The key insight: parabolic encoding k = (2j, -j²) makes dot-product attention peak sharply at a target position. Same encoding addresses program memory, stack, locals, and heap without interference. Each attention head is a compiled W_Q @ state → query, W_K @ memory → keys, scores = K @ q, output = V[argmax(scores)].

Updating from Repo

To pull latest source from the repository:

cd /mnt/skills/user/llm-as-computer/src
GH_TOKEN=$(grep GH_TOKEN /mnt/project/GitHub.env 2>/dev/null | cut -d= -f2)
for f in executor.mojo; do
  curl -sL -H "Authorization: token $GH_TOKEN" -H "Accept: application/vnd.github.v3.raw" \
    "https://api.github.com/repos/oaustegard/llm-as-computer/contents/src/$f?ref=main" > $f
done
for f in isa_lite.py programs.py runner.py; do
  curl -sL -H "Authorization: token $GH_TOKEN" -H "Accept: application/vnd.github.v3.raw" \
    "https://api.github.com/repos/oaustegard/llm-as-computer/contents/skill/src/$f?ref=main" > $f
done
rm -f percepta_exec  # force recompile
bash setup.sh

Orchestrating Agents (deprecated)

Renamed to orchestrating-chat-agents on 2026-10-11.

Why

The old name sent requests to the wrong skill. Shown only skill names, Haiku, Sonnet and Opus 5.5 routed 10 of 45 parallel-dispatch requests here, work that belongs to dispatching-subagents. Of seven candidate names, orchestrating-chat-agents caught the most of this skill's own requests (19/21 against 15/21) without drawing more of the others' (oaustegard/experiments subagent-skill-choice). The content is the same: API fan-out from surfaces with no native subagents, and Gemini delegation.

Where things went

you wantgo to
parallel Claude calls over the API from claude.ai, ConversationThread, AgentPoolorchestrating-chat-agents
fan-out in Claude Code: Agent tool vs Workflow vs headless claude -pdispatching-subagents
SendMessage / ListAgents between native subagentsdispatching-subagents/references/native-messaging.md
which model or effort a subagent getsagent-routing

scripts/ here holds shims, so sys.path.insert(0, '/mnt/skills/user/orchestrating-agents/scripts') followed by from claude_client import invoke_claude still works. Point new code at /mnt/skills/user/orchestrating-chat-agents/scripts.

SURFACE ROUTING — read first

Fan-out has three possible engines. Which exist depends on where you are running. Pick the engine before writing any orchestration code.

Engineclaude.aiCoworkClaude Code / CCotw
Native subagents (Agent / Task / Workflow)✗✓✓
Gemini via CF AI Gateway (invoking-gemini)✓✓✓
This skill's httpx fan-out (raw Anthropic API)✓last resortlast resort

Primary discriminator — check the tool list, not the filesystem. If an Agent, Task, or Workflow tool is callable, native subagents exist. That single fact decides the row. Everything below is elaboration.

If native subagents exist (Cowork, Claude Code, CCotw)

Use them. Do not hand-roll from this skill. The managed runtime gives 16-concurrent / 1000-agent ceilings, an approval gate, adversarial cross-review, and in-session resume — all of which this skill would reimplement worse. Route model and effort per agent-routing (calibrated on 300 measured Haiku calls); do not re-derive that here.

Which native mechanism to use (Agent tool, Workflow, or headless claude -p children launched from one background command) is dispatching-subagents. In Claude Code the claude binary authenticates through the host, so a headless fan-out needs no API key either.

Cowork adds one option Claude Code doesn't: subagents can be declared rather than spawned ad hoc, as agents/*.md in a plugin — frontmatter name, description, model, effort, maxTurns, tools, disallowedTools, skills, memory, background, isolation: worktree. They appear as plugin-name:agent-name. Note hooks, mcpServers, and permissionMode are refused in plugin agents for security, so a declared agent inherits the session's MCP connections and cannot bring its own.

Reach back into this skill on those surfaces only for what the runtime lacks: stall detection, or a long-lived ConversationThread. Inter-agent messaging is NOT on that list — the runtime ships SendMessage and ListAgents, and AgentPool reimplements them worse.

Inter-agent messaging between native subagents (SendMessage, ListAgents) and its measured traps are in dispatching-subagents/references/native-messaging.md.

If native subagents do NOT exist (claude.ai chat and project sessions)

Two engines, and Gemini is the default — see subagent-delegation-protocol in ops. Use this skill's httpx fan-out when you specifically want Claude-family output, multi-turn threads with cached history, or inter-agent messaging.

Gemini via Cloudflare — available on every surface

Even where native subagents exist, Gemini is the right call for mechanical-but-large work (extractions, ports, boilerplate, schema transforms) and for a genuinely independent second opinion in a judge panel — a different model family fails differently, which is the whole point of a panel.

Call mechanics live in invoking-gemini; do not duplicate them here. Three things to know:

  • invoking-gemini's default model is gemini-4-hydrogen, Google's current default; its flash alias points there too. Use gemini-3.5-flash-lite (lite) for bulk extraction and classification.
  • thinking_level is a string in {minimal, low, medium, high, xhigh}. Hydrogen defaults to high, so set low for mechanical generation or the model silently spends its output budget reasoning — symptom is an empty or truncated response. Only lite accepts minimal; on other models invoking-gemini sends low instead (a raw API call gets HTTP 400). xhigh works on Hydrogen and Neon.
  • Credentials come from the CF AI Gateway config, BYOK. Requests route through the gateway rather than Google directly.

Non-negotiable on all surfaces

Review is not delegable. Diff security- and protocol-critical paths line-by-line against source, run syntax/lint checks, live-test whatever is network-testable. Delegated output ships only after your own review, regardless of which model produced it or which engine ran it.

Cross-model review tools (challenge, verify_patch) keep their own model config, often deliberately a Claude. This routing does not silently repoint them.

Orchestrating Chat Agents

Formerly orchestrating-agents (renamed 2026-10-11, because the old name drew parallel-dispatch requests that belong to dispatching-subagents).

This skill enables programmatic API invocations for advanced workflows including parallel processing, task delegation, and multi-agent analysis using the Anthropic API.

Quick Start

Single Invocation

import sys
sys.path.append('/mnt/skills/user/orchestrating-chat-agents/scripts')
from claude_client import invoke_claude

response = invoke_claude(
    prompt="Analyze this code for security vulnerabilities: ...",
    model="claude-sonnet-5-5"
)
print(response)

Default model is claude-sonnet-5-5 ($2/$10 per MTok); claude-opus-5-5 ($4/$20) for open-ended work and claude-haiku-5-5 ($0.10/$0.50, prompts up to 100K tokens, 1M context) for cheap first-pass calls. These models reject non-default sampling parameters, so temperature / top_p / top_k are accepted by the wrappers but only sent to legacy (4.x / 3.x) model ids. They also think adaptively; the wrappers return the joined text blocks, never content[0].

Parallel Multi-Perspective Analysis

from claude_client import invoke_parallel

prompts = [
    {
        "prompt": "Analyze from security perspective: ...",
        "system": "You are a security expert"
    },
    {
        "prompt": "Analyze from performance perspective: ...",
        "system": "You are a performance optimization expert"
    },
    {
        "prompt": "Analyze from maintainability perspective: ...",
        "system": "You are a software architecture expert"
    }
]

results = invoke_parallel(prompts, model="claude-sonnet-5-5")

for i, result in enumerate(results):
    print(f"\n=== Perspective {i+1} ===")
    print(result)

Parallel with Shared Cached Context (Recommended)

For parallel operations with shared base context, use caching to reduce costs by up to 90%:

from claude_client import invoke_parallel

# Large context shared across all sub-agents (e.g., codebase, documentation)
base_context = """
<codebase>
...large codebase or documentation (1000+ tokens)...
</codebase>
"""

prompts = [
    {"prompt": "Find security vulnerabilities in the authentication module"},
    {"prompt": "Identify performance bottlenecks in the API layer"},
    {"prompt": "Suggest refactoring opportunities in the database layer"}
]

# First sub-agent creates cache, subsequent ones reuse it
results = invoke_parallel(
    prompts,
    shared_system=base_context,
    cache_shared_system=True  # 90% cost reduction for cached content
)

Multi-Turn Conversation with Auto-Caching

For sub-agents that need multiple rounds of conversation:

from claude_client import ConversationThread

# Create a conversation thread (auto-caches history)
agent = ConversationThread(
    system="You are a code refactoring expert with access to the codebase",
    cache_system=True
)

# Turn 1: Initial analysis
response1 = agent.send("Analyze the UserAuth class for issues")
print(response1)

# Turn 2: Follow-up (reuses cached system + turn 1)
response2 = agent.send("How would you refactor the login method?")
print(response2)

# Turn 3: Implementation (reuses all previous context)
response3 = agent.send("Show me the refactored code")
print(response3)

Streaming Responses

For real-time feedback from sub-agents:

from claude_client import invoke_claude_streaming

def show_progress(chunk):
    print(chunk, end='', flush=True)

response = invoke_claude_streaming(
    "Write a comprehensive security analysis...",
    callback=show_progress
)

Parallel Streaming

Monitor multiple sub-agents simultaneously:

from claude_client import invoke_parallel_streaming

def agent1_callback(chunk):
    print(f"[Security] {chunk}", end='', flush=True)

def agent2_callback(chunk):
    print(f"[Performance] {chunk}", end='', flush=True)

results = invoke_parallel_streaming(
    [
        {"prompt": "Security review: ..."},
        {"prompt": "Performance review: ..."}
    ],
    callbacks=[agent1_callback, agent2_callback]
)

Interruptible Operations

Cancel long-running parallel operations:

from claude_client import invoke_parallel_interruptible, InterruptToken
import threading
import time

token = InterruptToken()

# Run in background
def run_analysis():
    results = invoke_parallel_interruptible(
        prompts=[...],
        interrupt_token=token
    )
    return results

thread = threading.Thread(target=run_analysis)
thread.start()

# Interrupt after 5 seconds
time.sleep(5)
token.interrupt()

Core Functions

FunctionModulePurpose
invoke_claude()coreSingle synchronous invocation, full parameter control
invoke_parallel()coreConcurrent invocations, results in input order
invoke_claude_streaming()coreSingle invocation, token-by-token callback
invoke_parallel_streaming()coreConcurrent invocations with per-agent stream callbacks
invoke_parallel_interruptible()coreConcurrent invocations cancellable mid-flight
ConversationThreadcoreStateful multi-turn thread with cached history
StallDetectorcoreFlags agents idle beyond a timeout
TaskTrackertask_stateTracks task status across an orchestration run
invoke_with_retry()orchestrationSingle invocation with backoff on transient errors
invoke_parallel_managed()orchestrationConcurrency-limited parallel run with retry, stall hooks, reconciliation

Full signatures, parameters, and worked examples for each: references/function-reference.md.

Example Workflows

See references/workflows.md for detailed examples including:

  • Multi-expert code review
  • Parallel document analysis
  • Recursive task delegation
  • Advanced Agent SDK delegation patterns
  • Prompt caching workflows

Execute Mode (Default Sub-Agent Prompt)

For autonomous sub-agents that should execute without asking questions:

from claude_client import invoke_claude, EXECUTE_MODE

response = invoke_claude(
    prompt="Review auth.py for SQL injection vulnerabilities",
    system=f"You are a security expert.\n\n{EXECUTE_MODE}"
)

EXECUTE_MODE encodes these principles (adapted from OpenAI Codex):

  • Make assumptions instead of asking questions; state them briefly
  • Think ahead: what else might be needed?
  • Report failures with what you tried and what you'll do next
  • Summarize deliverables and how to validate them

Agent Pool (Named Agents with Messaging)

For workflows where multiple agents need to communicate:

from agent_pool import AgentPool

pool = AgentPool(
    shared_system="You are reviewing the auth module of a web app.",
    max_depth=3,    # prevent recursive spawn explosion
    max_agents=10,
)

# Spawn named agents with roles
pool.spawn("security", system=f"Focus on vulnerabilities.\n\n{pool.EXECUTE_MODE}")
pool.spawn("perf", system=f"Focus on performance.\n\n{pool.EXECUTE_MODE}")

# Run turns (pending inter-agent messages auto-injected)
sec_result = pool.run("security", "Review the login flow")

# Agent-to-agent messaging
pool.send("security", to="perf",
          content="Auth does N+1 queries in the session check loop",
          trigger_turn=True)  # auto-runs perf with this context

# Broadcast to all agents
pool.broadcast("security", "Auth uses bcrypt cost=12, 200ms per hash")

# Query pool state
pool.agents()           # ["security", "perf"]
pool.agent_info("perf") # {name, depth, children, pending_messages, turns}

Spawn Reservation (Atomic Agent Creation)

For complex workflows where agent creation might fail:

from agent_pool import AgentPool

pool = AgentPool(shared_system="Code review team")

# Reservation pattern: name is reserved, rolled back on exception
with pool.reserve("analyst", parent="lead") as res:
    res.configure(system="You analyze code complexity.", model="claude-opus-5-5")
    # If configure or any other work raises, the name is released
# Agent "analyst" is now live

# Depth limits prevent unbounded recursion
pool.spawn("sub-analyst", parent="analyst")  # depth=2, OK
pool.spawn("sub-sub", parent="sub-analyst")  # depth=3, raises ValueError

When to Use AgentPool vs invoke_parallel

PatternUse When
invoke_parallel()Independent tasks, no inter-agent communication needed
AgentPoolAgents need to share findings, build on each other's work, or have parent/child relationships
invoke_parallel_managed()Independent tasks with retry, stall detection, concurrency limits

Setup

Prerequisites:

  1. Install anthropic library:

    uv pip install anthropic
    
  2. Configure the API key as a file the shell reads directly — never as something a tool call returns.

    On claude.ai the project's files are mounted at /mnt/project, so the key can be sourced without ever entering context:

    set -a; . /mnt/project/ANTHROPIC.env 2>/dev/null; set +a
    

    ⚠️ Do not use project_read to fetch a credential, on any surface. Small docs are returned inline, so the key lands in the transcript — verified 2026-07-30: the documented "large text is written to a local file" branch does not fire even at 64 KB. In Cowork there is no /mnt/project mount at all and no safe read path, so the key must arrive by a route the shell can read (synced skill directory, or fetched by a script from the CF config store). Writing is safe in both directions — project_write with local_path keeps contents out of context — but reading is not.

    Get your API key: https://console.anthropic.com/settings/keys

Installation check:

python3 -c "import anthropic; print(f'✓ anthropic {anthropic.__version__}')"

Error Handling

The module provides comprehensive error handling:

from claude_client import invoke_claude, ClaudeInvocationError

try:
    response = invoke_claude("Your prompt here")
except ClaudeInvocationError as e:
    print(f"API Error: {e}")
    print(f"Status: {e.status_code}")
    print(f"Details: {e.details}")
except ValueError as e:
    print(f"Configuration Error: {e}")

Common errors:

  • API key missing: Add ANTHROPIC_API_KEY.txt to project knowledge (see Setup above)
  • Rate limits: Reduce max_workers or add delays
  • Token limits: Reduce prompt size or max_tokens
  • Network errors: Automatic retry with exponential backoff

Prompt Caching

For detailed caching workflows and best practices, see references/workflows.md.

Performance Considerations

Token efficiency:

  • Parallel calls use more tokens but save wall-clock time
  • Use prompt caching for shared context (90% cost reduction)
  • Use concise system prompts to reduce overhead
  • Consider token budgets when setting max_tokens

Rate limits:

  • Anthropic API has per-minute rate limits
  • Default max_workers=5 is safe for most tiers
  • Adjust based on your API tier and rate limits

Cost management:

  • Each invocation consumes API credits
  • Monitor usage in Anthropic Console
  • Use smaller models (claude-haiku-5-5) for simple tasks
  • Use prompt caching for repeated context (90% savings)
  • Cache lifetime: 5 minutes, refreshed on each use

Best Practices

  1. Use parallel invocations for independent tasks only

    • Don't parallelize sequential dependencies
    • Each parallel task should be self-contained
  2. Test with small batches first

    • Verify prompts work before scaling
    • Check token usage and costs

Token Efficiency

Loading this skill costs roughly 2k tokens. On surfaces with native subagents the routing table at the top is usually all you need — read it, spawn natively, and skip the rest of the file.

See Also

Routing companions — read these before choosing an engine:

  • agent-routing skill — model + effort selection for native subagents (Haiku/Sonnet/Opus, cascades, verifier gates). Calibrated on measured data. Applies to Cowork and Claude Code; explicitly not to claude.ai.
  • invoking-gemini skill — call mechanics for the CF AI Gateway path, model table, and thinking_level semantics.
  • subagent-delegation-protocol (ops config) — why Gemini is the claude.ai default, the Sonnet fallback config, and the non-delegable-review rule.

This skill's own internals:

Reviewing AI Papers

When users request analysis of AI/ML technical content (papers, articles, blog posts), extract actionable insights filtered through an enterprise AI engineering lens and store valuable discoveries to memory for cross-session recall.

When NOT to use this skill

The subject matter has to be machine learning. Adjacent asks that are not:

SituationUse
"Does this text read as AI-written?"declauding
Register or voice pass on a draftdeclauding
Review a pull request or a diffcode-review
A paper outside MLread it directly; this skill's lens will not fit

"AI" appearing in the request is not the trigger — AI being the topic of the document is.

Contextual Priorities

Technical Architecture:

  • RAG systems (semantic/lexical search, hybrid retrieval)
  • Vector database optimization and embedding strategies
  • Model fine-tuning for specialized scientific domains
  • Knowledge distillation for secure on-premise deployment

Implementation & Operations:

  • Prompt engineering and in-context learning techniques
  • Security and IP protection in AI systems
  • Scientific accuracy and hallucination mitigation
  • AWS integration (Bedrock/SageMaker)

Enterprise & Adoption:

  • Enterprise deployment in regulated environments
  • Building trust with scientific/legal stakeholders
  • Internal customer success strategies
  • Build vs. buy decision frameworks

Analytical Standards

  • Maintain objectivity: Extract factual insights without amplifying source hype
  • Challenge novelty claims: Identify what practitioners already use as baselines. Distinguish "applies existing techniques" from "genuinely new methods". The procedure for this is "The ablation the paper did not run" below. Run it; do not improvise a judgement
  • Separate rigor from novelty: Well-executed study of standard techniques ≠ methodological breakthrough
  • Confidence transparency: Distinguish established facts, emerging trends, speculative claims
  • Contextual filtering: Prioritize insights mapping to current challenges

The ablation the paper did not run

Run this before writing any part of the assessment. It reads off the paper's own tables and needs no code, no reimplementation and no access to the data.

A paper's ablation table names what its authors thought was contestable. The axis every row holds fixed is the one nobody argued, and it is where an unearned mechanism survives review.

Four steps:

  1. List every ablation the paper reports and name the axis each one varies.
  2. Name the axis that no row varies.
  3. On that axis, name the cheapest mechanism producing the same output shape, meaning what a practitioner would reach for having never read the paper.
  4. Determine whether the paper ran it.

Emit this block in every review. A review without it is incomplete:

VARIED:               <axes the paper ablates>
HELD FIXED:           <the axis no row varies>
CHEAPEST ALTERNATIVE: <what a practitioner would use on that axis>
RAN IT:               yes | no | partially, against <what>

Step 3 is where this fails. The alternative a paper argues against is the one its mechanism was built to beat, and adopting it as the comparator inherits the paper's framing. Write the comparator from what a practitioner would use, not from the alternatives the paper chose to name.

RAN IT: no describes the paper's coverage. It does not settle whether the method works. State what the missing comparison would decide and what running it would cost. A mechanism can beat the cheap alternative; the paper simply does not say so yet. When the answer is yes, say so in the credibility assessment: a paper that ran the comparison is stronger for having run it.

DIAGNOSED, twice, both times found after the fact:

  • SPD/hLLM (arXiv:2609.01807, 2026-09-08). Tables 3 and 4 ablate the scoring head, the training signal and the backbone. No row varies the decoder, and the Hungarian solve is the paper's contribution. Measured afterwards: sorting one column of the score matrix ties the assignment solve, +0.0008 [−0.0028, +0.0044] NDCG@10 over 1,785 slates. The paper's own comparator, row-argmax with repair, does lose to the Hungarian, so its ablation is correct as far as it goes and the finding sits one step past it. (oaustegard/experiments PR #92.)
  • TTT-Embed (arXiv:2608.12569, 2026-08-14). Ablates reward scope and reward budget. No row varies how the residual query vector is obtained, which is the contribution. Two label-free constructions of the same object cost zero reward budget and were not run: Rocchio over the top-k retrieved documents, and the query-document modality-gap direction. (Memory 7461f178.)

Analysis Structure

For Substantive Content

Article Assessment (2-3 sentences)

  • Core topic and primary claims
  • Credibility: author expertise, evidence quality, methodology rigor

Prioritized Insights

  • High Priority: Direct applications to active projects
  • Medium Priority: Adjacent technologies worth monitoring
  • Low Priority: Interesting but not immediately actionable

Technical Evaluation

  • Distinguish novel methods from standard practice presented as innovation
  • Flag implementation challenges, risks, resource requirements
  • Note contradictions with established best practices

Actionable Recommendations

  • Research deeper: Specific areas requiring investigation
  • Evaluate for implementation: Techniques worth prototyping
  • Share with teams: Which teams benefit from this content
  • Monitor trends: Emerging areas to track

Immediate Applications Map insights to current projects. Identify quick wins or POC opportunities.

For Thin Content

  • State limitations upfront
  • Extract marginal insights if any
  • Recommend alternatives if topic matters
  • Keep brief

Memory Integration

Automatic storage triggers:

  • High-priority insights (directly applicable)
  • Novel techniques worth prototyping
  • Pattern recognitions across papers
  • Contradictions to established practice

Storage format:

remember(
    "[Source: {title or url}] {condensed insight}",
    "world",
    tags=["paper-insight", "{domain}", "{technique}"],
    conf=0.85  # higher for strong evidence
)

Compression rule:

  • Full analysis → conversation (what user sees)
  • Condensed insight → memory (searchable nugget with attribution)
  • Store the actionable kernel, not the whole analysis

Example:

Analysis says: "Hybrid retrieval (BM25 + dense) shows 23% improvement over pure semantic search for scientific queries. Two-stage approach..."

Store as: "[Source: arxiv.org/abs/2401.xxxxx] Hybrid BM25+dense retrieval: 23% lift over semantic-only for scientific corpora. Requires 10K+ domain examples for fine-tuning benefit."

Tags: ["paper-insight", "rag", "hybrid-retrieval", "scientific-domain"]

Output Standards

  • Conciseness: Actionable insights, not content restatement
  • Precision: Distinguish demonstrates/suggests/claims/speculates
  • Relevance: Connect to focus areas or state no connection
  • Adaptive depth: Match length to content value

Constraints

  • No hype amplification
  • No timelines unless requested
  • No speculation beyond article
  • Note contradictions explicitly
  • State limitations on thin content

Swarming to green

A loop that turns a test suite's failures into merged fixes, unattended, for hours:

known.json --cluster by signature--> work items --one headless worker each, own worktree-->
  result JSON --serial gate: apply on main, compat check, run smoke + targets + sample,
  zero regressions, >=1 target flips--> fast-forward main --> re-check the whole cluster --> repeat

Every mechanism in it fixed a failure that cost hours on 2026-10-12, when Haiku 5.5 workers built hcc (a C compiler in Python) against ~42K tests: 1,051 hub workers, 464 merges, $24.88 of worker spend by 18:30Z. The catalogue of those failures, each with its signal and where it is handled, is references/run-2026-10-12.md.

SK=/mnt/skills/user/swarming-to-green       # or <claude-skills clone>/swarming-to-green
SWARM="python3 $SK/scripts/swarm.py"             # every command below; --root defaults to the git toplevel

When to use it, and when not

Use it when all hold: a test command can report every test's status as JSON; there are hundreds or more failing tests; a fix can be checked by re-running tests; and the work will run for hours, longer than anyone watches.

situationuse instead
fewer than ten tasks, e.g. setup, a suite builder, a refactorthe Agent tool directly (on 2026-10-12, 3 setup agents were run through claude -p for no gain)
a one-off fan-out of independent tasks (judges, extractors, per-file reviews) with no mergingdispatching-subagents
which model or effort a worker gets, and how to escalateagent-routing (this skill's measured answer for test-fix work is in "Which model" below)
a fixed multi-step pipeline with no model in the loopflowing
parallel Claude calls from claude.ai chatorchestrating-chat-agents (orchestrating-agents is its deprecated name)
a handful of failing testsfix them yourself, or one Agent-tool worker per cluster
one program to improve against a numeric scorer, not tests to turn greenevolving-with-swarms
no machine-readable test output yetwrite the runner first ("Project setup"), then come back

Abandon the run when the signals say the harness, not the workers, is the problem: SAME FAILURE, DENIALS or GATE REJECTS ALL twice in a row after a fix. Stop the loop ($SWARM stop), fix the cause, unpark if the fix invalidates earlier attempts, relaunch. Spending through a harness bug is how the abandoned tail was made.

Preflight checklist

Do every item before the first real launch. Each one cost hours when missed.

  1. Failing-test supply first. Haiku drained haiku-cc's conformance suites in 70 minutes and the queue sat empty. Have thousands of failing tests, a tier of real programs (one build failure blocks hundreds of tests), and a suite builder that works in its own worktree and commits to main. The loop baselines new suites within two minutes. Details: references/project-setup.md.
  2. Plan containers now if workers do real compute. When each worker compiles and runs tests, CPU caps the swarm whatever the launch mechanism: 4 vCPU held haiku-cc at load 10-22 for eight hours before siblings were added. Decide the partition and the number of sibling sessions at preflight: references/scaling-out.md.
  3. Disk. df -h /tmp <root>/.. needs 10 GB or more free. Worker scratch reached 14 GB and stale worktrees 5.6 GB on 2026-10-12. The loop deletes scratch on reap, sweeps stale worktrees at start, and stops launching below run.min_free_gb.
  4. The 2-hour background cap. A background Bash job dies at 2 hours, and at 30 minutes if you omit timeout. Launch with timeout: 7200000 and --max-minutes 110; the loop exits cleanly, workers keep running, and the next launch adopts them.
  5. Coordinator models. Every session you create inherits your model unless you pass model. Idle Opus siblings cost about $1.40 per check-in. Give siblings and experiment sessions a Sonnet or Haiku model.
  6. Push namespace. Federation pushes <prefix>main/state/cand/*. A push-policy guard (claude-workspace scripts/guards.py _ALLOWED_PREFIXES) must allow the prefix.
  7. Gitignore build artefacts and pin the language version the fast interpreter supports; set gate.compat_command to enforce the pin.

Setup

  1. Config. Copy assets/swarm.example.toml to <root>/swarm.toml and fill tests.command, paths.code/allowed/forbidden, queue.ladder, gate.always_suites/cheap_suites, gate.compat_command, worker.system_prompt, worker.repro_command, and worker.deny for whatever the workers must never run (network fetches when the answer is on the internet). Every key: references/config.md. A complete working config for haiku-cc is assets/haiku-cc.toml.
  2. Worker prompt. Start from assets/worker_prompt.template.md. Keep the result-JSON contract, "no git commits", "scratch outside the repo", "add a file, don't edit a shared list", and "revert and report failed with notes".
  3. Check the runner's output: $SWARM doctor --run-suite <a suite>. It exits 1 on a contract violation and prints WARN lines for a stage label on over 90% of failures, truncated located reasons, or signatures that do not cluster. Fix the runner before launch: on 2026-10-12 a broken stage regex labelled every failure parse. A stage that holds nearly every failure by design (decompilation: all missing at the start, nearly all match later) goes in tests.one_stage_ok.
  4. Selftest the harness (no model calls, about 3 minutes):
    $SWARM selftest; echo "selftest exit=$?"
    
    Exit 0 means every check passed. Read the exit code, never a tail of the output.
  5. Dry look at the queue: $SWARM refresh (first baseline), then $SWARM queue --top 20. The top items should be the clusters you would hand out first. If they are not, fix queue.ladder now.

Launch

One Bash call, run_in_background: true, timeout: 7200000. Run the loop itself, not a waiter, not nohup, and not inside a subagent (a subagent's background job dies with the subagent):

cd <root> && $SWARM run --concurrency 24 --max-usd 50 --max-minutes 110 \
    --batch 6 --sample 80 >> runs/swarm.log 2>&1

In the same turn: ps aux | grep '[s]warm.py run' shows it, and tail -5 runs/swarm.log shows LAUNCH lines or a first full refresh. Preflight failures print preflight: lines and exit 2; fix them rather than passing --force. Then end the turn and arm the first check-in: send_later(delay_minutes=50, initiation="own_followup", message="swarm check-in: ...").

Useful flags: --only 'suiteA/*,suiteB/*' (dispatch only these), --suites (what this node's gate runs), --model, --effort, --max-workers N, --no-launch.

Check-in protocol

Every ~50 minutes, by a self-re-arming send_later. Never loop on sleep.

tail -40 runs/swarm.log; $SWARM status; df -h /tmp <root>/..

Read the SIGNALS: block at the end of status and act:

signalmeaningact
none, loop alivehealthyre-arm the check-in
loop not running / DEAD (stale pid file)--max-minutes exit, the 2-hour cap, or a crashread the log tail for a traceback; relaunch the same command
REFRESHINGa refresh owns the gate tree; merging is pausedwait; a full refresh past an hour means raise gate.full_every (or set it to 0 and rely on the sweep), a quick one past ~10 min means trim gate.quick_suites, a sweep step past ~20 min means that suite is too slow to rotate: split it
SWEEP BEHINDa suite has gone over twice gate.sweep_lag merges without running on mainrefreshes are starving the rotation or merges outpace it: lower quick_every or raise sweep_lag, never turn the sweep off mid-run
SWEEP REGRESSIONSa sweep step found regressions it could not revertread the log's bisect/revert lines; revert the culprit by hand while it is at most sweep_lag merges back
SWEEP ERRORthe test command produced nothing for a suite on its sweepdoctor --run-suite <suite>; until fixed, that suite's pass count is unchecked
QUICK REGRESSIONSthe last quick refresh found regressions it could not revertread the log's bisect/revert lines; revert the culprit by hand now, while it is at most gate.quick_every merges back
GATE BACKLOGfinished workers outrun the gateraise --batch, lower --sample, or lower --concurrency so the gate gets CPU (haiku-cc: ~15 to ~41 merges/h)
CPU-BOUND (log: THROTTLE)load above the throttle with work waitingscale out: references/scaling-out.md
SUPPLY LOWunder 10 dispatchable itemsstart or speed up a suite builder now
PARTITION EXHAUSTED (log: IDLE)nothing dispatchable on this nodeadd supply, re-partition, unpark after a harness fix, or stand the node down
SAME FAILUREmost recent workers fail with one reasona prompt, auth or permission bug: stop, fix, relaunch
DENIALSworkers hit permission denialscheck worker.allow against what the prompt tells them to do
GATE REJECTS ALLnothing merged in the last 15 gatedread .swarm/rejected/; one test named in all = stale known state or a cross-partition regression: refresh --only-suites <suite>
DISK LOW (log: DISK)free space under 2x min_free_gbfind the hog (du -sh /tmp/* <wt_root>/*); the loop already stopped launching
UNBASELINED SUITESsuite dirs the baseline never rannormally clears within 2 minutes; else refresh --only-suites
ONE STAGEone stage label on over 90% of failuresthe runner's stage detection is broken: doctor, fix the runner

Also read the plain lines: quick refresh: (cadence, merges since the last, and its last result: regressions found, reverted, left), sweep: (the bound, the worst lag and the last step's outcome), base check: (items not launched because their tests already passed at HEAD), conflict hot spots (a shared file workers keep colliding on: make it self-registering, see project-setup), top rejection/failure reasons, last merge ... ago, and known now pass counts.

Scaling out

When one container's CPU is the cap, add sibling sessions, each owning a partition of the suites, joined through branches of one git repo with a hub that owns main and re-gates every candidate. The full procedure (when, partitioning, transport and its guard prerequisites, the sibling brief, create_session parameters, the cross-partition gate rule, stand-down and ledger collection, isolated experiment sessions) is references/scaling-out.md. Brief template: assets/sibling-brief.md. Bootstrap script: scripts/sibling_setup.sh.

Which model

Default: Haiku plus the harness's retry ladder. Treat Sonnet as an optional rung for a cluster that has failed the whole Haiku ladder, and never escalate a tail the harness created.

Measured 2026-10-12 on haiku-cc over three escalation runs (40, 4 and 10 clusters after base checks; same prompt, budget and base; each patch graded in isolation):

armsolve rate$ per solve
Haiku 5.5, default effort76/80, 8/8, 19/20 runs$0.011-0.012
Haiku 5.5, high effort8/8, 20/20 (runs 2-3)$0.018-0.023
Sonnet 5.5, medium80/80, 8/8, 20/20$0.075-0.087 (about 7x Haiku)

Over 54 clusters, cold Sonnet solved 2 that cold Haiku did not (run 1, c06 and c16). Haiku feature-mode workers in the live swarm fixed both later for $0.054 and $0.022: the retry path (notes from failed attempts, feature budget, effort ladder) closed both gaps. Run 3 was the tail the swarm had abandoned; Haiku solved it cold once the harness bugs that had parked it were fixed. The harness made that tail, not a Haiku capability limit. Scope: one compiler workload, small n. Re-measure on a new workload with $SWARM compare select|run|report, in its own container (compare selects from the abandoned tail at HEAD and base-checks the selection, because random failing clusters saturate and known state goes stale).

A second workload had a real Haiku ceiling. On matching decompilation (2026-10-13, gcc -O2, 1,768 functions) the tail Haiku left was large, loop-heavy functions, and each effort step past high bought fewer of them at a steeper price per function flipped: default $0.039, high $0.20, high feature $0.58, xhigh $1.30, max $3.90. Climb past high to learn where the ceiling is, not to clear a tail cheaply. Details: references/run-2026-10-13.md.

The ladder the harness runs per signature: attempt 1 at the configured effort; after a real failure (a lost merge race is not one) at the next rung of worker.effort_ladder (default ["", "high"]; Haiku 5.5 also takes xhigh and max); after feature_after failures a single feature-mode worker with a bigger budget at worker.feature_effort or above; a conflicted fix comes back as a rebase task. A rebase waits while a live worker modifies a file its patch modifies (queue.near_hold_s, default 1200 s after the conflict; HOLD in the log), and other work takes the slot: on the haiku-cc ledgers 31 of 43 rebases launched beside such a worker conflicted again, against 1 of 11 without one. First attempts never wait, because nothing known at dispatch predicted their files. The rung counts a test's failures over all time, so an unpark retries the tail at the effort it had reached. Rates and prices change: check agent-routing before reusing these numbers.

Partial credit

When a test can get measurably closer without passing (a byte-match fraction, a count of matching outputs) and failures depend on each other, a strict flip gate throws work away. On 2026-10-13 (swarm-decomp, matching decompilation) 125 partial workers, 39% of spend, were discarded; 103 of their notes named a callee another cluster had not written yet. Have the runner emit the measure per test (frac), set gate.partial_credit = "frac", and refresh. The gate then merges a candidate that raises the summed measure of its targets by gate.partial_min_gain with zero regressions, and the next worker starts from that code. Leave it off for pass/fail-only suites: there it only merges noise.

Earned exceptions

rulebanned whenearned when
the gate needs a target to flip—ingested sibling candidates: the sibling's gate already verified the flip
one worker per clusterthe signature is concrete (one root cause): shards write the same fix and conflicta masked signature with 10+ tests: up to 6 shards
full refresh (full_every, default off)the sweep is on: it runs every suite within sweep_lag merges anyway; a suite was just added: baseline only ita change to the runner outside the suite dirs
unpark after a fixthe fix does not touch why tests were parkeda harness bug made attempts fail (stale state, skip-as-regression, conflicts counted)
--force past preflightdirty code/tests, missing runner, low diska known-harmless warning you can name

Failure modes

The 2026-10-13 decompilation run broke assumptions that only held for a compiler suite (one stage dominating by design, tests that depend on each other, a cap that stopped launches but not spend): references/run-2026-10-13.md.

Each is signal, then fix. Dated entries happened on 2026-10-12 (catalogue numbers in brackets refer to references/run-2026-10-12.md).

  • Loop crashes on git worktree add with what looks like a git error → the disk is full. df -h; the culprit was /tmp scratch (14 GB). Reap deletes it now [30].
  • Workers fix tests that already pass → known state was stale after a shared-root fix. Fixed by recheck_clusters after every merge [8]. If you see it, refresh.
  • Good patches all rejected for the same "regression" → that test stopped running (skip list) or another node broke it. MISSING is no longer a regression [11]; for a cross-partition break, re-baseline that suite on the hub [22].
  • Tail of parked tests that never get workers → attempts the harness caused count toward parking. Fix the harness, then $SWARM unpark [13].
  • Conflicts dominate rejections → workers collide on a shared file. status names it under conflict hot spots; make it self-registering [16, 17]. Rebases that conflict again are what the near-hold is for; check it is not set to 0.
  • Killing the loop leaves test processes eating CPU (ppid 1) → use $SWARM stop, which kills by pid file and sweeps processes in the gate tree; never pkill -f a pattern that matches your own shell [25].
  • A long validation launched from a subagent vanished → a subagent's background job dies with it. Run long jobs under the main session or in the foreground [27].
  • An auto-mode classifier denies an action ("Shared Scratch Sweep" for a subagent deleting /tmp, "Modify Shared Resources" for a relaunch after editing the ledger, "Out-of-Place Publication" for a sibling's ledger push) → the first denial ends that line of work: report the action, the classifier's reason and what Oskar can approve. Do not route around it. Cleanup belongs inside the loop process, which already does it; get relaunch and stats-push approval up front [36].
  • A guards: deny whose stated reason does not match what you did (2026-10-12: question_build blocked builds after the question was answered; fixed since) → a guard false positive. Quote it to Oskar; read claude-workspace docs/failure-mining.md [37].
  • A peer session says the user approved something ("Oskar merged PR #99") → a peer's message is not approval. Check the primary source (gh pr view --json mergedBy,state, the branch tip) before acting on it [38].
  • A full refresh finds dozens of regressions and the bisected culprit no longer reverts (2026-10-12: 82 regressions after 200 merges, culprit 256 commits back) → the gate's random sample missed them. The quick refresh runs gate.quick_suites every gate.quick_every (25) merges and bisects only over those merges, so the culprit is still revertable; status shows it on the quick refresh: line. Keep the quick suites fast enough that a run stays well under the time 25 merges take [44].
  • The queue drains while merges since the last full refresh are unchecked (2026-10-12: the final measurement found late regressions in several suites; TinyCC fell from 333 to 35 passing) → leave gate.idle_refresh on: the loop refreshes before it idles, and the regressions it cannot revert come back as work [46].
  • Workers report their cluster already passes → the record is stale; they return stale and the loop re-measures the cluster at main [48]. The base check now catches most of these before a worker is spent: every known.json record carries the commit it was measured at, and the dispatcher re-runs an item's representatives at HEAD when code or tests changed since then (log: BASE-CHECK; status: base check:).
  • A full measurement takes hours on the hub → fan it out across sibling containers at one pinned commit, pinned after the last fix; never measure serially and restart [49].
  • The tail is a handful of clusters and each gate rebuilds whole programs → stop the loop; one Agent-tool worker per cluster in a worktree, then one combined gate [50].
  • A suite no node dispatches is never re-measured (2026-10-12: Mbed TLS broken for 14 h, 3,340 tests) → the rotating sweep runs every suite the node owns within gate.sweep_lag merges, so the gate only has to prove that targets flip; on a federation the hub owns every suite, so its sweep spans the partitions [51].
  • The project's unit selftests break mid-run (2026-10-12: 8 of 710) → the gate runs only the suites; put the fast unit tests in gate.compat_command [47].
  • Gate throughput flat while workers finish → GATE BACKLOG; tune batch and sample, keep CPU headroom for the gate [18].
  • A refactor landed mid-run conflicts with every in-flight fix → land structural changes before launch, or stop, land, relaunch [20].

Verification

  • Harness: $SWARM selftest; echo "selftest exit=$?" must print 0 failures and exit=0. It runs every gate path on a fake project: merge, forbidden edit, compat rejection, no-op, regression, conflict pair, doc conflict, non-UTF-8 diff, concrete signature not sharded, cluster recheck, skipped-not-regressed, effort ladder and feature mode, park and unpark, new-suite baseline, stale-worktree sweep, stop, the federation round trip (publish, seed, ingest, rebase, conflict-as-rebase, ledger export), a quick refresh and a sweep step that each revert a regression the gate sample missed, a base check that launches no worker on a test main already passes, and the near-hold. On 2026-10-12, selftest | tail && git commit committed with 3 failures because the pipe hid the exit code. Gate commits on the real exit code.
  • A run: $SWARM status pass counts in known now rise between check-ins, and git log --format=%s -5 on main shows swarm commits with Swarm-Task: trailers.
  • A bad success: merges climbing while known now stays flat (fixes that flip only their own targets via special-casing), or no target test flipped dominating rejections (stale state). Sample three merged diffs before reporting progress.
  • Reports: state an outcome only from a check run after the last edit or relaunch: the status output, the selftest exit code, the merge count from the ledger. Not from memory of an earlier check.
  • Stats for a write-up: $SWARM stats --out runs/stats (metrics.json, report.md, and charts when matplotlib is importable). Annotate harness changes as they happen with $SWARM event "conflicts become rebase tasks"; the merge-rate curve cannot be read without them.

Tiling Tree

Implements the MIT Synthetic Neurobiology tiling tree method: recursively partition a problem space into non-overlapping, collectively exhaustive subsets until reaching actionable leaf ideas, then evaluate those leaves.

Core Concept

The method's power comes from MECE splits forcing exploration of unfamiliar territory. A split is only valid when you can state precisely what each branch excludes — if you can't, the criterion is too vague and branches will overlap.

Key insight from the source method: always look for the "third option" that falls outside an obvious binary split. The bloodstream-secretion approach to neural recording only emerged because "wired vs. wireless" was defined precisely enough to reveal it covered neither case.

When to Use

  • "What are all the ways we could solve X?"
  • "Apply the tiling tree method to Y"
  • "Exhaustively map the solution space for Z"
  • Any request for MECE decomposition of a problem domain

Setup

Requires orchestrating-chat-agents skill to be installed. Load it first:

import sys
sys.path.insert(0, '/mnt/skills/user/orchestrating-chat-agents/scripts')

from claude_client import invoke_claude, invoke_parallel, parse_json_response

Running the Tiling Tree

# Basic usage
python3 /mnt/skills/user/tiling-tree/scripts/tiling_tree.py "Your problem here"

# With options
python3 /mnt/skills/user/tiling-tree/scripts/tiling_tree.py \
  "How can we record neural activity?" \
  --depth 3 \
  --criteria "impact,novelty,feasibility" \
  --output /mnt/user-data/outputs/neural_recording_tree.md

Parameters

ParameterDefaultNotes
problemrequiredNatural language problem statement
--depth2Max recursion depth. Depth 2 ≈ 16 leaves, depth 3 ≈ 64 leaves
--criteriaimpact,novelty,feasibilityComma-separated evaluation dimensions
--outputtiling_tree.mdOutput markdown path

Depth guidance: Start with depth 2 to validate the problem framing. Increase to 3 only when the domain genuinely warrants it — depth 3 generates ~64 leaves and ~40 API calls.

Architecture

  • Orchestrator (this script): builds tree skeleton, dispatches parallel split jobs per level, merges results, detects gaps
  • Branch agents (invoke_parallel): each receives one node to split, returns MECE branches with explicit exclusion statements
  • Evaluator (invoke_claude): single agent scores all leaves for cross-leaf consistency

Parallel splitting happens level-by-level (not node-by-node), so a depth-2 tree makes only 2 API round-trips for the splitting phase regardless of branching factor.

Output

A markdown file containing:

  1. Full tree diagram with split criteria and evaluation scores at leaves
  2. Ranked leaf table sorted by overall score

Interpreting Results

Good trees have:

  • Split criteria that are definitions, not questions ("energy source type" not "is it renewable?")
  • Leaf exclusions that confirm non-overlap
  • A "surprising" branch — something you wouldn't have thought of without the tree

If all leaves feel obvious, the split criteria were too coarse. Redo the tree with more precise definitions at the branch level where it went flat.

verifying-claims

Check that what a document says about code is true, by reading the document, the code, and the tests together and reporting where they disagree.

What changed (v0.1 → v0.2)

v0.1 was a comment-DSL: you hand-wrote <!-- claim: ... --> next to prose and a script checked the comment against the code. That had a fatal gap — the comment and the prose were two artifacts stapled together, and only the comment was checked, while humans read the prose. The prose could lie with a green run.

v0.2 drops the DSL. The reviewer is the agent: it reads the prose's meaning directly and compares it to what the code does and what the tests assert. No shadow copy, because the thing being checked is the thing the human reads. (Existing tools already own the alternatives — Gherkin binds executable scenarios, Lean's Verso transcludes facts into prose, TDD couples code to tests. This fills the remaining slot: free-prose documentation, judged.)

Division of labor — read this first

This skill does NOT gate merges and is NOT a test framework.

  • The test suite (TDD/CI) owns the behavioral contract: deterministic, cheap, auditable, gated. A green check is something you can hold CI to.
  • This skill owns the prose layer: does the documentation match reality? That needs semantic judgment across artifacts, which is non-deterministic and fallible — so it runs as a triggered review (before docs ship, on request, as a sweep), not as a per-commit gate. "The agent said the docs match" is not a guarantee you gate a merge on; it's a review you act on.

Tests are the anchor. The docs are correct when they agree with what the tests assert about the code. So write/keep good tests first; this skill keeps the prose pinned to them.

Procedure

  1. Identify the document(s) to check and the code + tests they describe.
  2. Gather consistent input: run scripts/gather_context.py --doc DOC --src SRC --tests TESTS. It ast-parses source (no imports, no execution) and bundles the document text, the public API surface, and the test inventory.
  3. Extract the claims the prose makes — every checkable assertion about the code (signatures, behavior, return shapes, defaults, guarantees, examples). Do this by reading; there are no claim markers.
  4. Judge each claim against the API surface and the tests:
    • Does the code actually do what the prose says?
    • Is the claim backed by a test, or merely asserted?
    • Does it reference something that no longer exists?
  5. Report drift, ranked by severity, each finding citing the prose claim and the contradicting reality (file/function). Use the verdicts below.
  6. Optionally fix: rewrite the prose to match reality, and/or flag claims that need a test (an UNSUPPORTED claim is a missing test, not just a doc bug).

Verdicts

  • PASS — the prose claim matches the code and is exercised by a test.
  • FAIL — the code contradicts the claim (the doc is wrong, or the code regressed and the doc caught it).
  • UNSUPPORTED — the claim matches the current code but no test backs it, so nothing protects it from future drift. Surface as a missing test.
  • STALE — the claim refers to something removed or renamed.

Invoking

  • "Check the README against the code before I publish it."
  • "Does docs/api.md still match pkg/?"
  • "Sweep the docs for drift after this refactor."

Run it at moments that matter — pre-publish, post-refactor, on a docs PR — not on every commit. The deterministic gate is the test suite; this is the layer tests can't reach.

Honest limits

  • Non-deterministic and fallible: a review can miss drift or misjudge. Treat output as a careful review, not a proof.
  • Cost/latency: reading three artifacts and reasoning is expensive next to a test run. Don't wire it where a cheap deterministic check belongs.
  • It checks prose against code+tests; it does not verify the tests themselves are correct. Garbage tests → confident-but-wrong PASS. TDD discipline upstream still matters.

When NOT to use

  • As a CI merge gate (use the test suite).
  • To verify behavior (write a test).
  • On prose with no factual claims about code (nothing to check).

Files

  • scripts/gather_context.py — deterministic input bundler (doc + API surface + test inventory), ast-only, no imports.
  • references/drift-report-example.md — what a review report looks like.

Verifying review findings

A model that reviews code for defects also invents them: it misreads a guard ten lines up, a loop bound, or a lock the caller holds. This skill splits the review into two roles, as the open-source reviewer bluesky-social/roast does. A finder (you, the code-review skill, any reviewer) writes each finding with a verbatim quote of the code it is about. A verifier from a different model family then reads that code and confirms or refutes the claim.

The verifier is Jev, a decision model from TypeSafe. It answers typed questions (yes/no, multiple choice, ordered scale) with a probability for every answer in about 0.3 s. Jev is not a Claude model, so when Claude finds and Jev verifies, the verifier is less likely to share the finder's blind spots. Below, P(x) means the probability Jev gives to answer x.

scripts/verify.py does the following for each finding:

  1. Grounding. The quote must occur in the file, compared with runs of whitespace collapsed. A finding with no quote, or a quote that is not in the file, is dropped before Jev sees it.
  2. The window. The verifier sees a window of the file: the top-level block (function, class, statement) that contains the quote, with at least 12 lines on each side, capped at 160 lines. Lines are numbered and the quoted ones are marked >. With --diff, the file's diff hunks go along too.
  3. Three questions to Jev, one call per finding, in parallel:
    • verdict: confirmed, refuted or insufficient_context
    • severity if the defect is real, on the scale info, low, medium, high, critical
    • when the finding proposes a fix, whether the fix works
  4. Thresholds. P(refuted) of at least 0.7 drops the finding (--refute-at). P(confirmed) of at least 0.7 confirms it (--confirm-at). Anything else is uncertain. Only a refutation removes a finding; a verifier that is unsure keeps it.

When NOT to use this skill

SituationUse instead
Nobody has looked for defects yetcode-review (or any reviewer) first; this skill only judges findings that exist
Checking that a README or docstring still matches the codeverifying-claims
An adversarial read of a whole deliverable (prose, analysis, a PR's design)challenging
A routing, triage or yes/no question that is not about a code findingdeciding-with-confidence, or the Muninn decide tool directly
Building a deterministic check that must be able to go redgating
The claim is about runtime behaviour the code cannot show (performance, a flaky test, production data)run it; a verifier reading code can say insufficient_context at best

Stop the run when most findings come back uncertain with reason needs_more_context: the evidence for those claims lives outside their windows (in a caller, another file, a config). Verify those findings by reading the code yourself.

Procedure

Throughout, V=/mnt/skills/user/verifying-review-findings/scripts/verify.py.

  1. Write the findings as a JSON list. file (relative to the repository root) and code_quote are required. Copy the quote character for character from the file, one to six lines; a paraphrase is dropped.

    [{"id": "1", "file": "src/util.py", "line": 6,
      "description": "Off-by-one: the loop skips the last element.",
      "code_quote": "for i in range(len(items) - 1):",
      "fix": "for i in range(len(items)):", "severity": "high"}]
    

    The other fields are optional. id defaults to the item's position. line picks between several matches of the same quote. description is the claim Jev judges. fix adds the fix question. severity is the finder's own estimate and is passed to Jev as a claim. Findings in the ReportFindings shape (summary and failure_scenario instead of description) are accepted too. When you are the finder, emit this shape directly.

  2. Pick the path. --transport auto, the default, uses the first of jev and gateway whose credentials are set.

    PathNeedsHow
    Jev direct (jev)TYPESAFE_API_KEY, a TypeSafe API keyrun (step 3a)
    Jev via Cloudflare Workers AI (gateway)CF_ACCOUNT_ID and CF_API_TOKEN, optionally CF_GATEWAY_IDrun; pass --transport gateway when both paths have credentials
    Muninn decide tool, which calls Jevthe Muninn MCP connector in the sessionemit, then apply (step 3b)
    Claude Haiku via deciding-with-confidencethat skill installed next to this onerun --transport decider; the verifier is then also Claude
  3. Run it.

    a. With credentials in the environment:

    python3 $V run findings.json --repo . --format md      # default output is JSON
    git diff main... > /tmp/d.diff && python3 $V run findings.json --diff /tmp/d.diff
    python3 $V run findings.json --ref HEAD~1               # read files at a git ref
    

    b. Through the Muninn decide tool:

    python3 $V emit findings.json --out batch.json
    

    emit saves the grounded findings to batch.json and prints the Jev requests, one per surviving finding, each as {"id", "state", "questions"}. Call decide once per request, passing its state and questions unchanged; the calls are independent, so send them together. Save the replies in answers.json as an object that maps each finding's id to the full text decide returned, then:

    python3 $V apply batch.json answers.json --format md
    
  4. Read the report. The top level has complete (false when any finding could not be verified), counts per status, and gate. Each finding keeps its input fields and adds:

    • status: confirmed, uncertain, dropped, or unverified (Jev was not reached or replied in an unexpected shape)
    • reason: why it is not confirmed. One of missing_code_quote, ungrounded_code_quote, file_not_found, refuted_by_verifier, split_verdict, needs_more_context (insufficient_context was the largest verdict), no_answer, or verifier_error: ...
    • verdict: the three verdict probabilities
    • severity: probabilities per level, and expected, the level nearest the probability-weighted average
    • fix_ok: the probability that the fix works
    • claimed_severity: the finder's own severity from the input
    • fingerprint: a hash of the file and the whitespace-collapsed quote. Use it to match a finding across runs, never to suppress one: two different claims about the same lines share a fingerprint.
  5. Act on it.

    • Report confirmed findings as findings.
    • Report uncertain ones separately and read their code yourself; the verifier's mistakes cluster there.
    • Leave dropped findings out, but give their count and reasons so a reader can recover one that was refuted wrongly.
    • When a confirmed finding has fix_ok under 0.5, rewrite the fix before proposing it.
  6. Gate a commit or CI step with --fail-on LEVEL (any severity level). Exit codes:

    CodeMeaning
    0every finding verified, and no gate tripped
    1a confirmed finding has P(severity at least LEVEL) of at least --severity-at (default 0.5)
    2bad input, or no transport available
    3some finding is unverified: the run is incomplete, not clean

    A pre-commit hook that should not block on an outage can let the commit through on 3, as long as the hook prints the report so the gap is visible:

    python3 $V run staged.json --fail-on high --format md; rc=$?
    [ $rc -eq 3 ] && echo "verify: incomplete run, commit allowed" >&2 && rc=0
    exit $rc
    

Earned exceptions

DefaultChange it whenKeep it when
--refute-at 0.7a missed defect costs more than reading a false one (security review, release gate): raise it toward 0.9 so fewer findings are droppedthe review is advisory, and noise is what makes people stop reading
The gate counts only confirmed findingsthe gate guards something irreversible: lower --confirm-at so more findings count as confirmeduncertain findings get a human look before merge
One window per findingthe claim depends on another file (a caller, a type, config)the evidence is in the same top-level block, which covers most of what reviewers flag

Common failure modes

  • verify: no Jev transport, exit 2 → neither TYPESAFE_API_KEY nor the Cloudflare pair is set. Use the emit/apply path, or --transport decider.
  • Every finding dropped with ungrounded_code_quote → the finder quoted from memory or reformatted the code, or --repo points at the wrong checkout. Run grep -F with one quote against the file to tell which.
  • file_not_found for files that exist → file is resolved against --repo (default: the current directory). A path that leaves the repository is treated as missing on purpose.
  • unverified with verifier_error: HTTP 4xx → check the error text. When it names the input length, the window was too long for Jev's default context: lower MAX_LINES (the 160-line cap, a constant at the top of verify.py). The report says complete: false until those findings are re-run.
  • unverified with no_answer after apply → answers.json has no entry for that finding's id. Ids are strings: "1", not 1.
  • A high severity.expected on a dropped finding → severity answers "how bad if real". Read it only on findings that survive.

Verification

python3 -m pytest /mnt/skills/user/verifying-review-findings/tests -q   # offline, no network
python3 $V eval /mnt/skills/user/verifying-review-findings/assets/eval.jsonl

eval runs 36 labelled findings (16 real defects, 16 false findings, 4 quotes not in the file) and prints the counts. Grounding is deterministic, so ungrounded_dropped must be 4/4. Jev's part gave real_kept 16/16 and false_dropped 11/16 with jev-1.14.0 on 2026-11-07. A lower count means the Jev version or the window settings changed; compare per item with --out results.jsonl.

A bad success is a report with complete: true in which every finding is confirmed near 1.0, including ones you know are false: Jev is judging the description without the code. Check that state.code in the requests emit prints holds the numbered window.

The eval's per-item results, the threshold sweep and what it does not cover are in references/method.md.

Diagnosed failures

  • 2026-11-07, eval item ts-04: a finding claimed a race condition in synchronous single-threaded JavaScript. Jev confirmed it at 0.89, the only false finding confirmed. No threshold catches a confident error like this. Read the code behind any concurrency claim about single-threaded code yourself.
  • 2026-11-07, eval items py-05 and sh-03: the evidence that refutes each claim was in the same file but in a different function, outside the window. Neither was dropped. sh-03 came back uncertain with needs_more_context (P(insufficient_context) 0.82). py-05 came back uncertain with split_verdict (P(confirmed) 0.55, P(insufficient_context) 0.40). Both outcomes are correct: the finding stays in the report, flagged for a human to read.

Sponsors

Details

Marketplace
oaustegard/claude-skills
Source
plugins/ai-and-reasoning
Category
Ai And Reasoning
License
MIT
Keywords
orchestrating-chat-agents, agent-routing, challenging, convening-experts, deciding-with-confidence, delegating-with-context, dispatching-subagents, down-skilling, evolving-with-swarms, gating, generative-thinking, invoking-antigravity, invoking-gemini, llm-as-computer, orchestrating-agents, reviewing-ai-papers, swarming-to-green, tiling-tree, verifying-claims, verifying-review-findings

More from oaustegard/claude-skills

All 14