Caio Review

/c-level-agents:caio-review — CAIO Forcing Questions

Command: /c-level-agents:caio-review <plan>

The eval-demanding CAIO pressure-tests any plan that involves AI. Six questions before any AI feature ships, any multi-year vendor commitment, or any AI team expansion.

When to Run

• Before shipping any new AI-powered feature
• Before signing a multi-year AI vendor contract (API or self-hosted infra)
• Before EU launch of any AI feature
• Before a major AI team hire (especially ML engineer or research scientist)
• Before a fine-tuning project commitment
• Before adopting AI in a regulated domain (employment, credit, healthcare, education, etc.)
• When the founder uses the word "AI" near "competitive advantage" or "moat"

The Six CAIO Questions

1. What does this AI need to be good at, and how would you measure it?

No eval set = no ship. Before any AI feature deploys, define the eval criteria.

• 50-100 representative inputs minimum
• Expected outputs OR rubric for grading
• Edge cases: ambiguous, adversarial, format-edge
• If you can't write down what "good" looks like, you don't have a feature; you have a vibe.