
Finetuning
Choosing between SFT, LoRA, DPO, GRPO, and other post-training methods by reward shape, then diagnosing failed training runs.
- GitHub stars
- 1.5K
- last updated
- Jun 11, 2026
What it does
- Covers a decision tree that picks the technique by reward shape: verifiable rewards (RL), preference pairs (DPO/KTO/ORPO), demonstrations (SFT), or scorer-filtered sampling (RFT)
- Includes a smoke-run gate on about 10 examples, periodic mid-training evals with early stopping, and single-attempt retry caps for training-scale runs
- Enforces a held-out eval that is never trained on, with evo's recorded numbers trusted over the run's self-report
- Diagnoses failures such as zero scores after SFT on verifiable benchmarks, sub-random base scores, and repeated plateaus caused by train-verifier format mismatch
Install
$ npx -y skills add evo-hq/evo --skill finetuning -yInstalls into .agents/skills for every agent it finds here, and .claude/skills for Claude Code.
Files
Finetuning
Priors, not rules. Only firm guardrails: held-out eval you never train on, no leakage, trust evo's recorded numbers over the run's self-report. Override anything else against the gate.
Pick the technique by reward shape
Decide on the reward first, technique second. Choosing the comfortable technique over the matching one is the most common failure.
| Reward shape | Technique |
|---|---|
| Verifiable (exact match, unit tests, parser-decidable) | RL (GRPO / RLOO / PPO) — reward includes format, so the model learns to emit verifier-acceptable shape |
| Preference pairs (chosen vs rejected) | DPO / KTO / ORPO — cheaper than full RL, no rollouts |
| Demonstrations only (curated traces, chat data) | SFT — install format/tone/capability the base lacks |
| Have a scorer + want SFT stability | RFT — sample, filter by reward, SFT on survivors |
"SFT-then-RL" is not a law. For a competent base model on a verifiable benchmark, RL-from-base often beats SFT-then-RL end-to-end.
Research the literature before the first commit
Details
- Repository
- github.com/evo-hq/evo
- Category
- Debugging
- License
- Apache-2.0
- First seen
- Jun 11, 2026
More from evo-hq/evo
All 6Similar to Finetuning
More Debugging
pester-migrationgithub/awesome-copilotPester migration skill for upgrading PowerShell Pester test suites across major versions — v3→v4, v4→v5, and v5→v6. Covers the Discovery/Run two-phase model, mo
update-docshoangnguyen0403/agent-skills-standardAutomated workflow to detect and fix missing documentation
verify-workhoangnguyen0403/agent-skills-standardVerify feature, bug, UI, API, mobile, security, or deployment work against acceptance criteria.
mcp-maintainerhoangsonww/claude-code-agent-monitorOperate and maintain the local MCP server for this repository. Use for MCP tool updates, policy-guard changes, host configuration, and MCP runtime troubleshooti
mcp-operationshoangsonww/claude-code-agent-monitorOperate and maintain the local MCP server for this project. Use when creating MCP host config, troubleshooting tool connectivity, modifying tool domains, or adj
meta-adshoodini/ai-agents-skillsPull, analyze, manage, and CREATE Meta ads (Facebook, Instagram, Messenger, Threads, Click-to-WhatsApp) via the Marketing API. Use when the user mentions Meta/F