Agent Eval

Runs controlled benchmarks to measure whether CodeGraph actually helps Claude answer questions about a codebase compared to just using grep and file reads. You pick a repo from the corpus, choose between the local dev build or a published npm version, then kick off either headless runs for exact token counts or tmux sessions that mirror real TUI behavior. Takes several minutes since it clones repos, wipes indexes, and runs the agent twice per test. The output shows tool call counts, file reads, and total cost for each arm. Useful when you're validating changes to CodeGraph itself or need numbers to prove retrieval improvements on real codebases rather than synthetic tests.

Install to Claude Code

npx -y skills add colbymchenry/codegraph --skill agent-eval --agent claude-code