Prompt Regression Tester

This is essentially A/B testing infrastructure for your prompts. You feed it old and new prompt versions along with test cases, and it runs both, diffs the outputs, scores them against constraints like must_include keywords and format requirements, then flags regressions with specific reasons (missing required content, length deviation, format violations). The breakage analyzer is genuinely useful because it groups failures by pattern and suggests concrete fixes like "add explicit instruction to mention cost implications." The stability metrics (output consistency, constraint adherence) give you confidence scores before deploying prompt changes. If you're iterating on production prompts and tired of manually checking if your tweaks broke edge cases, this systematizes the whole process.

Install to Claude Code

npx -y skills add patricio0312rev/skills --skill prompt-regression-tester --agent claude-code