Eval Harness

This is a structured framework for writing and running evals before you code, treating them like unit tests for AI development. You define pass/fail criteria upfront, implement against those specs, then measure reliability with pass@k metrics (like pass@3 > 90% for new features). It supports code-based graders for deterministic checks, model graders for subjective evaluation, and human review flags for security-critical changes. The workflow is simple: define evals in .claude/evals/, run them during development, track regressions. Honestly, this feels most valuable if you're already bought into test-driven development and want that discipline for prompt engineering. If you're just prototyping or doing one-off tasks, the overhead might not pay off.

Install to Claude Code

npx -y skills add affaan-m/ecc --skill eval-harness --agent claude-code