CCM
/Skills
SkillsMCPMarketplacesDigestToolsAdvertise

This week in Claude

Every Monday: Claude Code, Agent SDK, MCP, and the Anthropic platform moves worth your time.

Skills by Category
Frontend DevelopmentBackend & APIsTesting & QASecurityDevOps & CI/CDGit & Pull RequestsDocumentationCode Review & QualityAI & Agent BuildingSkill Development
MCP Servers by Category
Sales & MarketingWeb & Browser AutomationDatabasesAI & LLM ToolsCloud & InfrastructureCommunication & MessagingDeveloper ToolsDesign & CreativeDocuments & KnowledgeSearch & Web Crawling
Marketplaces by Category
AI Agents & OrchestrationLLM IntegrationDevelopment ToolsFrontend & UIBackend & APIsDatabasesTesting & Code QualityDevOps & CloudSecurity & ComplianceGit & Version Control

Claude Code Marketplaces

Discover Claude Code plugins, extensions, and tools. Automatically updated directory of Anthropic Claude AI marketplaces with development tools, productivity plugins, and integrations.

Resources

  • Browse Skills
  • Browse MCP Servers
  • Browse Marketplaces
  • Skill index
  • MCP index
  • Marketplace index
  • Plugins Reference

Community

  • About
  • Tools
  • Feedback
  • Privacy Policy
  • Advertise

Built for the Claude Code community with Claude Code by mertbuilds.com

Independent project, not affiliated with Anthropic
hamelsmu avatar

Validate Evaluator

hamelsmu/evals-skills
566 installs1.6k stars
Summary

This walks you through calibrating an LLM judge against human labels using proper train/dev/test splits and TPR/TNR metrics. You'd use it after writing a judge prompt when you need to verify it actually agrees with human judgment before trusting it in production. The workflow is methodical: split your labeled data, iterate on the dev set until you hit 90% TPR and TNR, then measure once on the held-out test set. It includes the Rogan-Gladen bias correction formula for estimating true success rates from biased judge scores, plus bootstrap confidence intervals. The anti-pattern section is worth reading since most people skip validation entirely and just assume judges work.

Install to Claude Code

npx -y skills add hamelsmu/evals-skills --skill validate-evaluator --agent claude-code

Installs into .claude/skills of the current project.

CodeRabbit
CodeRabbit
AI writes the code. CodeRabbit catches the slop.
Try For Free →
inference shell
inference shell
create and run specialised agents in minutes
build now →
MCP-ready Email SendingMCP-ready Email Sending
MCP-ready Email Sending
Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →
Make your agent a DeFi expert
Make your agent a DeFi expert
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
Capacitor - Shared memory for your team’s coding agents.
Capacitor - Shared memory for your team’s coding agents.
Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →
CodeScene MCP ServerCodeScene MCP Server
CodeScene MCP Server
Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →
Give your AI the whole web as clean markdownGive your AI the whole web as clean markdown
Give your AI the whole web as clean markdown
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
belt - the only tool your agent needs
belt - the only tool your agent needs
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →
CodeRabbit
CodeRabbit
AI writes the code. CodeRabbit catches the slop.
Try For Free →
inference shell
inference shell
create and run specialised agents in minutes
build now →
MCP-ready Email SendingMCP-ready Email Sending
MCP-ready Email Sending
Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →
Make your agent a DeFi expert
Make your agent a DeFi expert
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
Capacitor - Shared memory for your team’s coding agents.
Capacitor - Shared memory for your team’s coding agents.
Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →
CodeScene MCP ServerCodeScene MCP Server
CodeScene MCP Server
Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →
Give your AI the whole web as clean markdownGive your AI the whole web as clean markdown
Give your AI the whole web as clean markdown
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
belt - the only tool your agent needs
belt - the only tool your agent needs
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →
Files
SKILL.md

Select a file.

Featured
CodeRabbit
CodeRabbit
AI writes the code. CodeRabbit catches the slop.
Try For Free →
inference shell
inference shell
create and run specialised agents in minutes
build now →
MCP-ready Email SendingMCP-ready Email Sending
MCP-ready Email Sending
Plug Mailtrap into your AI workflow and let it handle the email.
Connect Mailtrap MCP →
Make your agent a DeFi expert
Make your agent a DeFi expert
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →
Capacitor - Shared memory for your team’s coding agents.
Capacitor - Shared memory for your team’s coding agents.
Make coding agent sessions - Searchable, Shareable, Vendor-neutral & Scored.
Try For Free →
CodeScene MCP ServerCodeScene MCP Server
CodeScene MCP Server
Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →
Give your AI the whole web as clean markdownGive your AI the whole web as clean markdown
Give your AI the whole web as clean markdown
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
belt - the only tool your agent needs
belt - the only tool your agent needs
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →
Categories
Testing & QADocumentationAI & Agent Building
First SeenJun 3, 2026
View on GitHub

More from hamelsmu/evals-skills

All 7 skills →
  • Build Review Interface527
  • Eval Audit1.2k
  • Evaluate Rag654
  • Write Judge Prompt625
  • Error Analysis604
  • Generate Synthetic Data575

Recommended

More Testing & QA →
alirezarezvani avatar
scenario-war-room

alirezarezvani/claude-skills

Cross-functional what-if modeling for cascading multi-variable scenarios. Unlike single-assumption stress testing, this models compound adversity across all business functions simultaneously. Use when facing complex risk scenarios, strategic decisions with major downside, or when the user asks 'what if X AND Y both happen?'
565
24.6k
machina-sports avatar
world-cup

machina-sports/sports-skills

Premium FIFA World Cup 2026 market & match intelligence — a hosted, read-only layer that fuses official match truth (fixtures, standings, squads, injuries, player performance) with live prediction markets (Kalshi + Polymarket: prices, order books, price history, movers, cross-venue edges) and AI-grounded context (prematch briefs, move explanations, fan/social pulse). Every entity carries a canonical machina URN cross-walked across api-football, sportradar, opta, entain and ESPN, so a market resolves to a fixture resolves to two teams. This skill is prompt-only and premium: it routes the agent to the hosted World Cup Intelligence project (a per-project Machina MCP server) via `machina-cli`. It runs no code locally and ships no API keys. Use when: the user wants World Cup 2026 odds + match context together, asks "what moved and why", wants a grounded market brief or fan-sentiment read on a fixture, or needs one stable id that joins markets ↔ fixtures ↔ teams across providers. Don't use w
563
202
readwiseio avatar
quiz

readwiseio/readwise-skills

Quiz yourself on documents you've recently read to test understanding and retention
563
281
product-on-purpose avatar
measure-experiment-design

product-on-purpose/pm-skills

Designs an A/B test or experiment with variants, success metrics, sample size, and duration for an existing hypothesis. Use when planning an experiment to validate a product change or test an assumption you have already framed. To articulate the hypothesis itself first, use define-hypothesis.
562
520
alirezarezvani avatar
statistical-analyst

alirezarezvani/claude-skills

Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes. Use when you need to validate whether observed differences are real, size an experiment correctly before launch, or interpret test results with confidence.
560
24.6k
longbridge avatar
longbridge-stock-research

longbridge/skills

Comprehensive equity research snapshot — integrates analyst consensus estimates, company fundamentals (revenue / profit / valuation), 60-day price history, and recent major news to produce an investment research snapshot similar to a sell-side equity research brief. Triggers: "股票研究", "个股分析", "研究报告", "个股快照", "综合分析", "股票调研", "股票深度", "個股分析", "研究報告", "個股快照", "綜合分析", "股票研究", "stock research", "equity research", "stock analysis", "research snapshot", "investment brief", "stock deep dive", "comprehensive analysis", "NVDA research", "700.HK analysis".
560
47