
cost-cutter
Skills that cut your AI bill with proof instead of vibes: routing, call budgets, receipts audits, tested fallbacks, batch sizing, guarded swaps, effort throttles, bake-offs, context diets, and free-model triage. Each skill is backed by a CI-verified recipe.
- GitHub stars
- 19
Install
/plugin install cost-cutter@ai-cost-cutter-skillsRun it inside Claude Code after adding the marketplace with /plugin marketplace add Neeeophytee/ai-cost-cutter-skills.
Contents
Discovered from sourceSkills 10
Cap the advisor, catch the drift, compute the real discount
The advisor pattern (a cheap model executes; an expensive model is consulted rarely) is only as cheap as the word "rarely". The user is about to adopt it on the strength of a benchmark number. Your job is to make the savings enforceable instead of assumed: a hard cap on advisor calls, a drift-check so the executor can't run unchecked, and an effective-cost number computed from what actually happened.
Steps
- Write down the pair with real unit prices: the executor model and the advisor model, each with
usd_per_mtok_outfrom the current pricing page (never from memory). The reference result this pattern leans on: Anthropic's internal eval had Sonnet 5 executing with Fable 5 advising about once per task, landing ~92% of Fable's SWE-bench Pro score at ~63% of the price. That is their task and their call rate, not the user's. - Set
max_advisor_callsfor the task, a hard cap, not a hope. An uncapped advisor erodes the discount one "just checking" call at a time. - Set
drift_check_every_n_steps: past this many executor steps, the executor must get a fresh advisor check-in before continuing. Between check-ins the executor can silently drift from the advice it was given; this bounds how far. - After (or while) running, record
advisor_calls_actualand compute the effective cost from the actual call mix. Compare it against the benchmark's assumed rate. If the task pulled the advisor in more often than "about once", the real discount is smaller than the quoted one, say by how much. - Run the proof below. It fails on a missing cap, a missing drift-check, or an actual call count that blew the budget.
Prove it
cat > advisor-task.json <<'JSON'
{
"executor": { "model": "sonnet-5", "usd_per_mtok_out": 15 },
"advisor": { "model": "fable-5", "usd_per_mtok_out": 50 },
"max_advisor_calls": 3,
"drift_check_every_n_steps": 5,
"task": { "steps": 20, "advisor_calls_actual": 2 }
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("advisor-task.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if (!c.max_advisor_calls || c.max_advisor_calls < 1) bad("set a hard max_advisor_calls, an uncapped advisor erodes the discount");
if (!c.drift_check_every_n_steps || c.drift_check_every_n_steps < 1) bad("set drift_check_every_n_steps so the executor cannot run unchecked forever");
const t = c.task || {};
if (t.advisor_calls_actual == null) bad("record advisor_calls_actual so the real rate can be compared to the assumed one");
if (t.advisor_calls_actual > c.max_advisor_calls) bad("advisor_calls_actual exceeds max_advisor_calls, the budget was not enforced");
const mix = t.advisor_calls_actual / (t.steps || 1);
const eff = ((1 - mix) * c.executor.usd_per_mtok_out + mix * c.advisor.usd_per_mtok_out) / c.advisor.usd_per_mtok_out;
console.log("advisor OK: cap " + c.max_advisor_calls + ", actual " + t.advisor_calls_actual + " call(s) over " + t.steps + " steps; effective cost ~" + Math.round(eff * 100) + "% of pure-advisor rate");
'
Guardrails
- Never present 63% (or any benchmark discount) as the user's number. It was measured on two specific benchmarks with a specific call rate; their workload has neither.
- The pattern assumes hard reasoning concentrates at a few forks. If difficulty is spread evenly across the task, one check-in per task misses it and the executor drifts uncaught, that workload should not use this pattern.
- If the job is easy, skip the pattern entirely and run the cheap model alone. Orchestration only pays for itself on hard, multi-step work.
- Don't let the planner grade its own plan. If advisor and executor share a wrong assumption, this setup amplifies it with confidence; put an objective check (test, schema, known answer) wherever correctness matters.
Backed by a machine-verified recipe, re-checked by CI: Advisor pattern: cap how often the expensive model gets called
Swap cheap, but guard the cases the cheap model loses
Every "the cheap model is 95% as good" claim hides a distribution: on most inputs the gap is invisible, and on a few specific cases the premium model isn't a few percent better, it's a different class. The canonical example (image generation, as of 2026-06): a cheap model like Wan runs ~7-8x cheaper per image than GPT-Image-2 and a normal viewer barely notices, except on text inside the frame, charts, dense layouts, and typography, where GPT-Image-2 leads its leaderboard by the largest margin recorded. A blind swap saves money and quietly ships garbled slides. Your job is to make the swap conditional and write the conditions down.
Steps
- Name the pair: the cheap model the user wants and the premium model it replaces. This works for any modality: text, image, video, speech.
- List the dominated cases: the input types where the premium model is known to win by a class, not a margin. Get these from a current leaderboard's category breakdown or the user's own failure examples, not from the headline average. If the user claims there are none, make them say it explicitly; that claim is the riskiest line in the config.
- Build the route table: every dominated case routes to
premium; everything else routes tocheap. Then check the saving is real: if nothing routes to cheap after honest accounting, the swap was a fantasy and the user just learned that for free. - Run the proof below. It fails if any declared dominated case leaks to the cheap model, or if nothing routes cheap at all.
- At request time, tag each input by case before dispatch (for images: does the prompt ask for words in the frame?). The same pattern covers video: cheap for iterations, premium for the physics-heavy hero shot.
Prove it
cat > decision.json <<'JSON'
{
"modality": "image",
"cheap": "wan-2.5",
"premium": "gpt-image-2",
"dominated_cases": ["text_in_frame", "charts_or_layout"],
"route": [
{ "when": "text_in_frame", "use": "premium" },
{ "when": "charts_or_layout", "use": "premium" },
{ "when": "general_scene", "use": "cheap" }
]
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("decision.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if (!c.cheap || !c.premium) bad("need both a cheap and a premium model");
const ruleFor = {};
for (const r of c.route || []) if (r.when && r.use) ruleFor[r.when] = r.use;
if (!(c.dominated_cases || []).length) bad("declare the cases the premium model dominates (or explicitly claim there are none)");
for (const d of c.dominated_cases) if (ruleFor[d] !== "premium") bad("dominated case " + d + " is not routed to the premium model");
if (!(c.route || []).some(r => r.use === "cheap")) bad("nothing routes to the cheap model, so there is no saving");
console.log("swap OK: " + c.cheap + " default, " + c.dominated_cases.length + " dominated case(s) guarded to " + c.premium);
'
Guardrails
- Never let a single percentage summarize the gap. Averages hide exactly the cases this skill exists to guard.
- Verify open-weights claims per version before anyone plans to self-host the cheap model; a family being "open" in one release says nothing about the next (Wan 2.1/2.2 shipped weights; 2.5 is API-only).
- Where one wrong output is expensive (customer-facing, legal, medical), keep a human check on the cheap lane regardless of what the table says.
Backed by a machine-verified recipe, re-checked by CI: Swap to a cheap image model, but guard the cases it loses
Feed the agent less, not a smarter model
When an agent re-reads the same files into context on every question, the bill scales with repetition, not with difficulty. The structural fix is a queryable index: the agent asks for the two facts it needs instead of swallowing whole files. One published reference point (arXiv:2603.27277, evaluated across 31 real repos, using a knowledge-graph MCP server): ~10x fewer tokens and 2.1x fewer tool calls at 83% answer quality versus file-by-file exploration. Treat those numbers as a ceiling from someone else's benchmark, not a promise; the whole skill is: measure your own drop.
Steps
- Baseline first. Pick one representative task (a real question about the codebase or corpus) and record tokens used and tool calls made, straight from the session's usage stats. Without this number, step 5 is theater.
- Wire an index the agent can query instead of reading raw files. For codebases over MCP,
codebase-memory-mcpis the worked example below; a repo map, a knowledge bundle, or a RAG index over docs serve the same role. What matters is the shape: query in, small answer out, instead of file in, everything in. - Validate the MCP config parses and points at a real binary (proof below), then run the initial index (a few minutes on a large repo, sub-millisecond queries after).
- Re-run the exact same task from step 1. Record tokens and tool calls again.
- Keep the index only if the drop is real and answer quality held. Report both numbers side by side. If quality dropped more than the bill, the diet was too aggressive: the agent needs some raw context, and hybrid is a fine outcome.
Prove it
cat > mcp-config.json <<'JSON'
{
"mcpServers": {
"codebase-memory": {
"command": "/usr/local/bin/codebase-memory-mcp",
"args": []
}
}
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("mcp-config.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
const servers = c.mcpServers || c.mcp_servers || {};
const keys = Object.keys(servers);
if (keys.length === 0) bad("mcpServers must have at least one entry");
const srv = servers[keys[0]];
if (!srv.command || typeof srv.command !== "string") bad("server entry must have a command (path to the binary)");
if (!Array.isArray(srv.args)) bad("args must be an array");
console.log("context-diet OK: MCP server " + JSON.stringify(keys[0]) + " wired at " + srv.command + " (now measure tokens before/after on the same task)");
'
Guardrails
- The index is a structural backend, not a smarter brain. It reduces what the agent reads; the model and the prompts still drive answer quality. Never promise quality gains from a context diet.
- Small repos don't need this. Below a few thousand lines, file-by-file context is cheap and the indexing overhead isn't worth it.
- Headline numbers (10x, 83%) came from someone else's 31 repos. The user's number is the one from step 5, and it's the only one worth repeating.
Backed by a machine-verified recipe, re-checked by CI: Wire the knowledge graph, stop re-reading files
Triage is a free-model job
Deciding what to read is a cheaper problem than reading: a one-sentence summary plus a needs-reply flag per item. Skimming twenty long items by hand runs about an hour; reading twenty one-liners and opening only what earned it is about fifteen minutes. On a top-tier paid model that triage volume costs real money every month; on a gateway's free tier it costs nothing (OpenRouter free models, as of 2026-06: 20 requests/minute and 50/day, rising to 1,000/day once $10 of credit has ever been added). Your job is to wire the free lane with a schema strict enough that the output is usable, and to keep anything sensitive out of it.
Steps
- Confirm the pile fits the caps: daily item count vs. 50/day (or 1,000/day post-credit). Over the cap means batching across days or the cheapest paid model; still pennies.
- Point any OpenAI-compatible client at
https://openrouter.ai/api/v1with a model whose id ends in:free. That suffix is the contract; without it the "free" triage quietly bills. - Pin the output schema to exactly two fields: a string
summaryand a booleanneeds_reply. The boolean is what makes triage actionable: a summary alone still makes the user read everything to find what needs them. - Give it one standing instruction ("Summarize in one sentence and flag whether it needs a reply") and run the proof below to validate the config shape.
- Sort the results: needs-reply items first, everything else skimmable. The user opens what earned it.
Prove it
cat > triage.json <<'JSON'
{
"base_url": "https://openrouter.ai/api/v1",
"model": "deepseek/deepseek-r1:free",
"instruction": "Summarize this in one sentence and flag whether it needs a reply.",
"response_schema": {
"type": "object",
"properties": {
"summary": { "type": "string" },
"needs_reply": { "type": "boolean" }
},
"required": ["summary", "needs_reply"]
}
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("triage.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if (!String(c.base_url || "").includes("openrouter.ai")) bad("base_url does not point at OpenRouter");
if (!String(c.model || "").endsWith(":free")) bad("model id must end with :free, otherwise this quietly bills");
const props = (c.response_schema && c.response_schema.properties) || {};
if (!props.summary || props.summary.type !== "string") bad("schema missing a string summary");
if (!props.needs_reply || props.needs_reply.type !== "boolean") bad("schema missing a boolean needs_reply");
console.log("triage OK: free model " + c.model + " returns a one-line summary + needs_reply flag");
'
Guardrails
- Free models are the most likely to retain or train on what they're sent. Anything private (client email, unreleased work, health, legal) goes to a cheap paid model instead; the difference is pennies and the policy is better.
- Triage decides what to read; it does not replace reading. A one-line squeeze sometimes flattens the one detail that mattered, so the flagged items still get opened.
- Respect the caps in the design, not in the error handler: 20/minute and 50/day sized up front, per
free-tier-batch-plan, beats discovering the limit mid-pile.
Backed by a machine-verified recipe, re-checked by CI: Let a free model triage your reading
Prove the batch fits before you start it
Some free tiers have a strange, exploitable shape: a painfully low rate ceiling paired with a huge monthly token budget (Mistral's free Experiment tier, as of 2026-06: about 2 requests/minute but around a billion tokens/month). Useless for anything live; ideal for a patient overnight grind. The failure mode is starting the job unsized and finding it stalled, or 12% done, in the morning. Your job is the arithmetic, up front.
Steps
- Count the backlog: how many items, and a realistic
est_tokens_per_item(input + output; when unsure, run 3 samples and measure rather than guessing). - Look up the tier's actual limits (the rate ceiling in requests/minute and any monthly token budget) from the provider's current docs, not from memory or this file.
- Compute two numbers and show both:
- Budget fit: items x tokens/item vs. the monthly budget. Over budget means shrink the job or split it across months.
- ETA: items / (rpm x 60) hours at the ceiling. This tells the user whether "overnight" is 8 hours or 3 days; both are fine, but only if known in advance.
- Run the proof below; it fails on a job that doesn't fit its budget.
- Only then queue it, with a runner that actually paces to the rate limit (bursting past it gets throttled or banned, which destroys the ETA you just computed).
Prove it
cat > batch.json <<'JSON'
{
"provider": "mistral",
"items": 3000,
"rate_limit_rpm": 2,
"est_tokens_per_item": 1200,
"monthly_token_budget": 1000000000
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("batch.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if (!c.items || c.items < 1) bad("no items to process");
if (!c.rate_limit_rpm || c.rate_limit_rpm < 1) bad("rate_limit_rpm must be at least 1");
if (!c.est_tokens_per_item || c.est_tokens_per_item < 1) bad("est_tokens_per_item must be set (measure 3 samples if unsure)");
const used = c.items * c.est_tokens_per_item;
if (c.monthly_token_budget && used > c.monthly_token_budget) bad("job needs " + used + " tokens but the monthly budget is " + c.monthly_token_budget);
const etaH = (c.items / (c.rate_limit_rpm * 60)).toFixed(1);
console.log("batch OK: " + c.items + " item(s) at " + c.rate_limit_rpm + " req/min = ~" + etaH + "h, using " + used + " of " + c.monthly_token_budget + " monthly tokens");
'
Guardrails
- Nothing live or interactive belongs on a 2-requests/minute tier. This pattern is for a one-time backlog with nobody waiting on it.
- Free tiers often use prompts to improve models. Sensitive or confidential material goes on a paid tier; still pennies, and worth it.
- This pays off for a single big grind, not recurring work. If the same batch runs weekly, price a paid lane and compare honestly.
Backed by a machine-verified recipe, re-checked by CI: Grind a huge one-time job overnight on a free tier
Pick a model with evidence, not vibes
A "GPT-killer at a tenth of the price" launches roughly every other week, and the user's realistic options are (a) ignore it and maybe overpay forever, or (b) burn a day benchmarking. There's a third option: a one-afternoon bake-off on a free tier, sized so it provably fits the cap before it starts. GitHub Models (as of 2026-06) exposes 45+ models, frontier ones included, behind an existing GitHub login, capped around 50 requests/day on top models: too tight for production, exactly enough for a careful comparison.
Steps
- Collect the prompts that actually matter: 3-7 real examples of the user's task, with a known-good answer or a concrete scoring rule each. Not synthetic puzzles; the actual tickets, extractions, or drafts this model will face.
- Pick 2-4 candidate models. Include the incumbent (the model currently doing the job) so the bake-off can conclude "keep what you have"; that is a valid and cheap outcome.
- State
select_bybefore running: exact-match rate, win rate on pairwise comparison, latency-under-quality-bar; anything, as long as it's written down first. A criterion chosen after seeing outputs is a rationalization. - Check the arithmetic: prompts x models must fit the free daily cap, or the bake-off dies at the rate limit halfway through with a biased partial result. Run the proof below.
- Run, score, keep the winner; then move the workload to whichever provider hosts the winner properly. The free tier was for deciding, not serving.
Prove it
cat > bakeoff.json <<'JSON'
{
"prompts": ["summarize ticket", "extract fields", "classify intent", "draft reply", "rate sentiment"],
"models": ["gpt-5.5", "deepseek-r1", "llama-4-maverick"],
"select_by": "win_rate",
"free_daily_cap": 50
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("bakeoff.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
const P = (c.prompts || []).length, M = (c.models || []).length;
if (P < 2) bad("need at least 2 prompts to compare meaningfully");
if (M < 2) bad("need at least 2 models for a bake-off");
if (!c.select_by) bad("no selection criterion stated before running (that is how rationalization starts)");
const calls = P * M;
if (c.free_daily_cap && calls > c.free_daily_cap) bad("bake-off needs " + calls + " calls but the free cap is " + c.free_daily_cap + "/day");
console.log("bakeoff OK: " + P + " prompt(s) x " + M + " model(s) = " + calls + " call(s), within " + c.free_daily_cap + "/day; pick by " + c.select_by);
'
Guardrails
- The free tier is for deciding, not serving: roughly 10 requests/minute and 50/day on top models, with contexts capped near 8k in / 4k out; keep bake-off prompts compact and never point production at it.
- Scoring with an LLM judge inherits the judge's blind spots, especially when judge and contestant share a family. Prefer a concrete metric (exact match, schema-valid, tests pass) wherever one exists.
- One bake-off is a snapshot, not a marriage. Hosted models drift and reprice; for a workload where the choice keeps mattering, re-run the same bake-off on a schedule and keep the receipts.
Backed by a machine-verified recipe, re-checked by CI: A GitHub Models bake-off that fits the free cap
Don't pay for deep thinking on easy turns
Reasoning effort is mostly output tokens, and output tokens are usually the expensive direction, so the effort knob is often the single biggest lever on an agent's bill, bigger than switching models. A cheap model at maximum effort can out-spend a mid model at low effort. Your job is to set a sane default, define when escalation is earned, and validate the config actually says what the user thinks it says.
Steps
- Confirm the user's harness supports per-task effort control. The worked example is Hermes (levels:
none,minimal,low,medium,high,xhigh, switchable at runtime with/reasoning xhigh, no restart); Claude Code, the Claude API's thinking budgets, and most agent runtimes have an equivalent. If theirs doesn't, effort throttling means routing between the same model's endpoints at different configured efforts. - Set the default one notch below the maximum the user has been running, not at the bottom. The reference setup (as of 2026-06): DeepSeek V4 Flash at $0.098/M in, $0.196/M out on OpenRouter; default
high, reservexhighfor turns that earned it. Check current prices before quoting. - Define "earned" concretely so escalation is a rule, not a mood: a failed first attempt, a task the user tagged hard, a step with irreversible consequences. Everything else runs at the default.
- Run the proof below; it validates the config parses and the effort level is one the harness actually accepts (a typo'd level silently falls back to a default you didn't choose). The example uses ruby's stdlib YAML parser; if ruby isn't on the machine, any YAML parser serves, since the point is proving the file parses and the level is a real one.
- After a week, check what fraction of the bill is output tokens at each effort level. If
xhighturns are more than ~10-20% of traffic, either the work is genuinely hard (fine, seeadvisor-call-budgetfor the next lever) or the escalation rule is too loose.
Prove it
cat > config.yaml <<'YAML'
model:
provider: openrouter
model: deepseek/deepseek-v4-flash
agent:
reasoning_effort: high
YAML
ruby -ryaml -e '
allowed = ["none","minimal","low","medium","high","xhigh"]
c = YAML.load_file("config.yaml")
m = c["model"] || {}; a = c["agent"] || {}
abort "BAD: no model set" unless m["model"].to_s.length > 0
re = a["reasoning_effort"].to_s
abort "BAD: reasoning_effort #{re.inspect} is not a valid level" unless allowed.include?(re)
puts "throttle OK: " + m["model"] + " with default reasoning_effort=" + re + " (escalate per task, not globally)"
'
Guardrails
- Leaderboard entries named "Max" and "High" are frequently the same model at different effort settings, not different models. Check before concluding a model got beaten.
- The savings are workload-dependent. Effort throttling pays when traffic is mostly easy turns with occasional hard ones; a workload that is uniformly hard just needs the effort, and pretending otherwise trades money for silent quality loss.
- CI-style validation proves the config, never the spend. The only honest savings number comes from comparing the bill before and after, same workload.
Backed by a machine-verified recipe, re-checked by CI: A one-line reasoning-effort throttle
Route the bulk cheap, escalate the hard turns
The user is paying premium-model prices for work that is mostly not premium-hard. Your job is to split their workload into a cheap default and a gated escalation, write that split down as a config, and prove the config is sane, without pretending the split is free quality.
Steps
- Ask (or infer from their traffic) what "hard" means for this workload. Escalation needs a gate you can state: a task type, a difficulty signal, or a failed first attempt. If they can't name one yet, "retry-on-failure" (cheap model first, premium on a failed check) is the honest default gate.
- Pick the pair. The cheap default should sit behind an OpenAI-compatible gateway so swapping later is a one-line change. Worked example as of 2026-06: GLM-5.2 (
z-ai/glm-5.2via OpenRouter,$3/M output tokens) as default, Opus 4.8 ($25/M output) as escalation, roughly 5-6x apart on output price, with GLM tying on long-horizon coding benchmarks while Opus still wins short, sharp general coding. Check current prices before quoting them. - Write
routing.json:base_url(the gateway),default_model(cheap),escalate_model(premium), andescalate_when(the gate). The gate must be conditional; anescalate_whenthat matches everything is just the premium model with extra steps. This file is the decision contract: translate it into whatever your dispatch layer speaks (a LiteLLM router config, an OpenRouter preset, or a few lines in your own client). - Run the proof below. It fails if the default isn't the cheap model, the escalation isn't set, or the gate is missing.
- Tell the user to measure cost per finished task, not per token. If the cheap model needs extra reasoning loops or retries to land the same task, the per-token advantage shrinks or inverts. The savings claim is only real after that measurement.
Prove it
cat > routing.json <<'JSON'
{
"base_url": "https://openrouter.ai/api/v1",
"default_model": "z-ai/glm-5.2",
"escalate_model": "anthropic/claude-opus-4.8",
"escalate_when": "task_difficulty == hard || category == general_coding"
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("routing.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if (!c.base_url) bad("route through a gateway base_url, not a hard-wired endpoint");
if (!c.default_model) bad("no cheap default_model set");
if (!c.escalate_model) bad("no escalate_model set");
if (c.default_model === c.escalate_model) bad("default and escalation are the same model, nothing is being saved");
if (String(c.escalate_when || "").length < 3) bad("escalate_when must gate escalation, not match everything");
console.log("routing OK: bulk -> " + c.default_model + ", escalate to " + c.escalate_model + " when " + c.escalate_when);
'
Guardrails
- A leaderboard win is not a promise about the user's task. Third-party and vendor benchmark numbers are directional; the bake-off on their own prompts is the evidence.
- Cheaper per token is not cheaper per task. Say this out loud in your summary; it is the most common way this pattern quietly fails.
- Flag data residency. Many cheap models are served by hosts in jurisdictions the user may care about; the weights being open says nothing about where their prompts land.
- The very cheapest routes on a gateway often serve a quantized variant, very good, not identical. If output quality matters, pin the provider, not just the model.
Backed by a machine-verified recipe, re-checked by CI: Run GLM-5.2 for the bulk, escalate the hard turns to Opus 4.8
A backup you never ran is a hope
Models vanish for reasons outside anyone's control: export controls pulled Fable 5 days after launch, and ordinary deprecation calendars retire models constantly. The user's exposure isn't "will it happen" but "how big is the blast radius when it does". Your job is to shrink that radius to a one-line config change, and to refuse the most common self-deception in this area: a fallback that's listed but has never actually been run.
Steps
- Route through an OpenAI-compatible gateway
base_urlrather than a hard-wired provider endpoint, so the swap is a config edit, not a code change. - Name the primary model. Then pin a fallback that satisfies all four conditions:
- different from the primary (obvious, and still worth checking),
- open-weights: a hosted proprietary fallback can be recalled by the same forces that took the primary; open weights cannot be un-published,
- a
tested_ondate: when the user last actually ran it, - the smoke prompts they ran: the 2-5 prompts that represent the work that matters.
- If any of the four is missing, the config is not done. In particular: if they have never run the fallback, run the smoke prompts now (or schedule it), then record the date. Do not write a
tested_ondate for a test that didn't happen. - Run the proof below; it refuses a fallback that is untested, hosted-only, or identical to the primary.
- Put the rehearsal on a schedule (monthly is fine): re-run the smoke prompts through the fallback, update
tested_on. Keep prompts and context in a portable form so the switch is copy-paste, not a rebuild.
Prove it
cat > failover.json <<'JSON'
{
"base_url": "https://openrouter.ai/api/v1",
"primary": "anthropic/claude-fable-5",
"fallback": {
"model": "z-ai/glm-5.2",
"open_weights": true,
"tested_on": "2026-07-01",
"smoke_prompts": ["summarize this ticket", "extract the invoice fields", "draft a reply in my voice"]
}
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("failover.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if (!c.base_url) bad("no gateway base_url (route through a gateway, do not hard-wire a model)");
if (!c.primary) bad("no primary model set");
const f = c.fallback || {};
if (!f.model) bad("no fallback model set");
if (f.model === c.primary) bad("fallback must differ from the primary");
if (f.open_weights !== true) bad("fallback must be open-weights (a hosted fallback can be recalled too)");
if (String(f.tested_on || "").split("-").length !== 3) bad("fallback needs a tested_on date (a backup you never ran is a hope)");
if (!Array.isArray(f.smoke_prompts) || f.smoke_prompts.length < 1) bad("fallback needs at least one smoke prompt you actually ran");
console.log("failover OK: primary " + c.primary + ", tested open-weights fallback " + f.model + " (tested " + f.tested_on + ", " + f.smoke_prompts.length + " smoke prompt(s))");
'
Guardrails
- Open-weights does not mean frontier. The biggest open models need serious hardware; the realistic fallback is a smaller open model or a cheap cloud host, not a home rig matching a flagship. Set that expectation when you pin it.
- The gateway is itself a dependency: one more company whose terms can change. It shrinks switching time; it does not remove risk. Say so.
- Never build anything important on a preview model; previews get the shortest-notice removals. Prefer generally-available models with a published deprecation policy.
Backed by a machine-verified recipe, re-checked by CI: Route through a gateway with a tested open-weights fallback
Read the token receipts right: volume and cost are different leaderboards
Token rankings lie by omission: cheap open models win volume (they sit in busy agent loops), premium models win spend (they take the fewer, harder calls). The public version of this flip, from OpenRouter's June 2026 rankings: Anthropic models were ~12% of tokens but roughly 46% of revenue. The user's own receipts almost certainly hide the same flip. Your job is to compute it, show it, and stop them optimizing the wrong line.
Steps
- Get the usage export: tokens per model over a real period (a month, not a week; weekly snapshots are volatile and free-preview models spike then vanish). A gateway dashboard export works; so does a hand-assembled list.
- Tier every model
cheap_openorpremium, and attach each model's unit price (usd_per_mtok) from the current pricing page, never from memory. - Compute, per tier: share of total tokens and share of total dollars. The signature of a healthy split is the flip: cheap models dominate volume, premium dominates cost.
- Report it as one table plus one sentence: "X% of your tokens cost Y% of your money." Then the actionable read: if premium's cost-share is high and its calls look like bulk work, that's the routing bug to fix (see
route-cheap-escalate-hard); if premium's cost-share is high on genuinely hard calls, the spend is earning its keep. - Run the proof below on their real numbers (the example uses the June 2026 public shape).
Prove it
cat > receipts.json <<'JSON'
{
"period": "2026-06",
"models": [
{ "name": "DeepSeek V4 Flash", "tier": "cheap_open", "tokens": 167000000000, "usd_per_mtok": 0.28 },
{ "name": "GLM 5.2", "tier": "cheap_open", "tokens": 50200000000, "usd_per_mtok": 3.00 },
{ "name": "Claude Opus 4.8", "tier": "premium", "tokens": 14800000000, "usd_per_mtok": 25.00 }
]
}
JSON
node -e '
const c = JSON.parse(require("fs").readFileSync("receipts.json", "utf8"));
function bad(m) { console.error("BAD: " + m); process.exit(1); }
if ((c.models || []).length < 2) bad("need at least two models to compare");
let totTok = 0, totCost = 0; const tierTok = {}, tierCost = {};
for (const m of c.models) {
if (!m.tier || m.tokens == null || m.usd_per_mtok == null) bad("each model needs tier, tokens, usd_per_mtok");
const cost = (m.tokens / 1e6) * m.usd_per_mtok;
totTok += m.tokens; totCost += cost;
tierTok[m.tier] = (tierTok[m.tier] || 0) + m.tokens;
tierCost[m.tier] = (tierCost[m.tier] || 0) + cost;
}
const pct = (x, t) => (x / t * 100).toFixed(1);
console.log("receipts OK: cheap_open is " + pct(tierTok.cheap_open || 0, totTok) + "% of volume but " + pct(tierCost.cheap_open || 0, totCost) + "% of cost; premium is " + pct(tierTok.premium || 0, totTok) + "% of volume but " + pct(tierCost.premium || 0, totCost) + "% of cost (volume != value)");
'
Guardrails
- Never read a high token ranking as "best model" or "biggest cost". It usually means "cheap model in a busy loop", that's the whole point of this audit.
- Gateway receipts are incomplete by construction: usage on direct provider keys or private deployments never appears there. State what the export can and cannot see.
- Attribute over a real period. One anomalous week (a migration, a runaway loop, a free preview) will produce a confident wrong conclusion.
Backed by a machine-verified recipe, re-checked by CI: Read your token receipts right
Sponsors
Details
- Marketplace
- Neeeophytee/ai-cost-cutter-skills
- Category
- Community