Agent Eval

A systematic way to compare coding agents on your actual codebase instead of vibes. You define tasks in YAML with a prompt and judge criteria (tests, grep patterns, or LLM eval), then run multiple agents in isolated git worktrees and collect pass rate, cost, time, and consistency metrics. The honest take: this is what you'd build yourself after the third time someone asks "should we use Claude Code or Aider?" and you realize anecdotes don't cut it. Run three trials per agent on five real tasks from your workflow, pin the commit for reproducibility, and you'll have data to make the call. No Docker required, just git worktrees and a willingness to treat agent selection like any other engineering decision.

Install to Claude Code

npx -y skills add affaan-m/ecc --skill agent-eval --agent claude-code