Summary
This pulls together the full monitoring stack for production LLM apps: Prometheus metrics with the RED method, Grafana dashboards, and Langfuse v4 tracing with semantic span types and inline scoring. It covers 12 rules across infrastructure monitoring, LLM observability, drift detection, and silent failure detection. The drift detection is smart, using PSI for production scale and dynamic thresholds to cut alert fatigue. What stands out is the opinionated Langfuse-over-LangSmith stance and the focus on catching silent failures like tool skipping and quality degradation. Reach for this when you need to instrument an LLM system properly, not just log requests.
Install to Claude Code
npx -y skills add yonatangross/orchestkit --skill monitoring-observability --agent claude-codeInstalls into .claude/skills of the current project.
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →
Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →Files
SKILL.md
Select a file.
Featured
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
