Arize Prompt Optimization

This pulls prompts from your production traces, correlates them with eval scores and human annotations, and runs an optimization loop using Arize's ax CLI. You point it at LLM spans in your project, it extracts structured chat messages or prompt templates, then pairs them with feedback signals like correctness labels or LLM-as-judge explanations. The workflow is thorough: export spans, reconstruct the prompt as a messages array, merge performance data from datasets and experiments, then iterate. Most useful when you're already logging OpenInference traces to Arize and have eval or annotation data sitting there. It won't guess at credentials or search your filesystem, it uses ax profiles and ai-integrations commands, which keeps things clean if you're working in a locked-down environment.

Install to Claude Code

npx -y skills add github/awesome-copilot --skill arize-prompt-optimization --agent claude-code