New advertisers: 50% off your first month with code FIRSTMONTHSee slots
evo-hq avatar

Finetuning

Choosing between SFT, LoRA, DPO, GRPO, and other post-training methods by reward shape, then diagnosing failed training runs.

GitHub stars
1.5K
last updated
Jun 11, 2026
View on GitHub

What it does

  • Covers a decision tree that picks the technique by reward shape: verifiable rewards (RL), preference pairs (DPO/KTO/ORPO), demonstrations (SFT), or scorer-filtered sampling (RFT)
  • Includes a smoke-run gate on about 10 examples, periodic mid-training evals with early stopping, and single-attempt retry caps for training-scale runs
  • Enforces a held-out eval that is never trained on, with evo's recorded numbers trusted over the run's self-report
  • Diagnoses failures such as zero scores after SFT on verifiable benchmarks, sub-random base scores, and repeated plateaus caused by train-verifier format mismatch

Install

$ npx -y skills add evo-hq/evo --skill finetuning -y

Installs into .agents/skills for every agent it finds here, and .claude/skills for Claude Code.

Files

Finetuning

Priors, not rules. Only firm guardrails: held-out eval you never train on, no leakage, trust evo's recorded numbers over the run's self-report. Override anything else against the gate.

Pick the technique by reward shape

Decide on the reward first, technique second. Choosing the comfortable technique over the matching one is the most common failure.

Reward shapeTechnique
Verifiable (exact match, unit tests, parser-decidable)RL (GRPO / RLOO / PPO) — reward includes format, so the model learns to emit verifier-acceptable shape
Preference pairs (chosen vs rejected)DPO / KTO / ORPO — cheaper than full RL, no rollouts
Demonstrations only (curated traces, chat data)SFT — install format/tone/capability the base lacks
Have a scorer + want SFT stabilityRFT — sample, filter by reward, SFT on survivors

"SFT-then-RL" is not a law. For a competent base model on a verifiable benchmark, RL-from-base often beats SFT-then-RL end-to-end.

Research the literature before the first commit

Details

Repository
github.com/evo-hq/evo
Category
Debugging
License
Apache-2.0
First seen
Jun 11, 2026

More from evo-hq/evo

All 6

Similar to Finetuning

More Debugging