
skill-diet
Agent skills that keep other skills lean, correct, and working on every model you run.
- GitHub stars
- 260
Install
/plugin install skill-diet@skill-dietRun it inside Claude Code after adding the marketplace with /plugin marketplace add nicobailon/skill-diet.
Contents
Discovered from sourceSkills 2
Skill Diet
Levels
Put each fact on the lowest level that still reaches the agent before it needs it.
| Level | Holds | Loaded |
|---|---|---|
description | one trigger per case | every session |
SKILL.md | what every task needs | when the skill is used |
references/ | what only some tasks need, behind a link that says when to read it | when that case comes up |
A skill that teaches a tool has two more levels: the tool's help, then its replies and errors. Exact names, fields and values live there, not in the skill. A check on the skill's output (linter, tests, renderer) can replace a rule, unless one miss costs a slow cycle.
Editing a skill
Stop at the first step that handles the change.
- Fix it below the skill. A mechanical mistake gets a clearer error, a check, or a test in the tool. Prose is for judgment calls.
- Check the model needs it. If a fresh session does the right thing without the line, add nothing.
- Edit, don't append. Change the line that already covers the idea. Each meaning lives in one place across
SKILL.md, references, and the tool's docs. - Pick its level.
- Write the rule, not the story. One positive rule under a short name; the incident goes in the commit message.
When code changes a name or behavior the skill relies on, update the skill in the same change. Index new public operations by name only, unless agents will call them often. Run the skill's coverage check, or add one if drift could recur.
Budget: put SKILL.md's size before and after in the commit message, and offset growth with a cut. When it is over 10% larger than at its last diet commit, or one idea appears in two places, do a full diet. node scripts/eval.mjs check <skill-dir> prints the size and checks links.
Verify: rerun the case that prompted the change in a fresh session, and any tests that assert the skill's text.
Look up the rest
- Full diet of a large, bloated or drifted skill → references/diet.md
Skill model gap
A skill tested on one model (the reference) often passes fewer tasks on another (the target). Find where and why, fix it with text every model reads, and prove the target gains while no model drops.
This runs skill-diet's eval: tasks and scoring per eval.md, the adapter per adapter.md, commands as node ../skill-diet/scripts/eval.mjs …. Edits follow skill-diet's Editing a skill.
Set up
- One adapter, one host for every model, so only the model differs. Pass
--modelto everyrunandscore. - Tasks per eval.md, each labeled with the case it covers. Split them blind and at random: about a third, at least one per case, go to a held-back folder whose
adapterandsk/link to the tuning folder's. Held-back tasks get no runs until the end. - Versions:
base(the skill as it is),none, and one folder per candidate fix. - Models: the reference has the best
basepass rate (on a tie, the one that reads the skill most); every other model with a gap is a target. Add at least one more model as a harm check. - Paid quota: before each step, give the user sessions × cost per run and stay within the budget they set. Two tasks first measure the cost.
1. Measure the gap
base and none on every model, tuning tasks, 3 runs. score <dir> --models <all> --versions none,base puts the models side by side; --model <m> instead gives one model's warnings. A gap is a task the target fails more often than the reference with a cause in its logs; 2/3 against 3/3 alone is noise. If the target does better with none (MISS or WORSE here), look for lines to cut first. No gap on any target → stop and report.
2. Find the cause
Loading first. Compare how many runs read SKILL.md on each model (the --models table's last row; score's under-half warning is this finding). If the target opens the skill much less often, fix loading before any rule: a description that says when to read the skill and what it adds to the tool's own description. If reads rise but passes don't, move the rules the target breaks into the description.
Otherwise a fresh reviewer reads each failing target run beside a passing reference run of the same task, and labels one cause with quoted evidence:
- skipped, misread or found no rule for the case in the skill it loaded → reword or widen the rule, or move it up a level;
- stopped early or asked the user → one line on what done means, unless the task really needs a question;
- tool, fixture or adapter (wrong parameter, tool error, missing file, dropped output) → fix it there, not in the skill;
- can't do it (fails even with the reference's steps written into the prompt) → drop it from the target's goal and report it;
- valid but scored wrong → eval.md's scorer check; fix the task and rerun it in every version on every model.
Each cause needs a held-back task that hits it (the same command or call). Write any that are missing now, before any fix, so step 4 can show a gain.
3. Fix one change at a time
Try a shared fix first: plainer, positive, more direct wording helps every model. Each candidate is a version L<n>: base plus one change.
- Helps the target:
basevsL<n>on the target, the gap tasks it's for, 6 runs. Keep it if the gainscore <dir> --model <target> --versions base,L<n> --tasks <those tasks>prints is larger than its 2 SE, or a task flips from failing every run to passing every run. - Hurts no one: all kept changes as
tuned, vsbaseon every other model, all tuning tasks, 3 runs. On WORSE or MISS, leave one change out at a time to find it, then drop or reword it. Compare reads too: a model that stops opening the skill loses the rules left in its body, so report the drop. - Still helps together:
tunedvsbaseon the target, all tuning tasks.
Every model reads every line; a host can't reliably hide a line by model.
4. Held-back, once
base vs tuned on every model, held-back folder, 9 runs. Ship when the target gains more than 2 SE, no model is WORSE or MISSes, and SKILL.md stays within skill-diet's budget. Report tokens/run per model as a cost: a skill that models now open costs its size on every task. If base already passes the target's held-back tasks, they can't show a gain: add tasks for the gap's causes and run once more. If base fails them and tuned doesn't gain, the fixes are overfit; write new held-back tasks before tuning again.
Afterward
On ship, record every model as eval.md's Tested with says. The commit message names which change came from which target, with the measurements. When a target in tested-with is replaced, rerun base with and without its changes on the new model, and delete what no longer helps.
Sponsors
Details
- Marketplace
- nicobailon/skill-diet
- Category
- Community