Rewardkit

Turns a directory of Python checks and TOML judge files into a scored verifier for Harbor tasks. Drop criterion functions or LLM judge configs into your tests/ folder, run one uvx command, and get a reward.json back. The multi-reward setup is clean: subdirectories become separate dimensions (correctness/, quality/, structure/), each with its own score. Built-ins cover the usual suspects (file checks, command output, JSON comparisons), and you can write custom criteria with a @criterion decorator. The agent-as-a-judge mode is interesting but slow, it actually shells out to a CLI that can explore files. Honestly feels like the right level of abstraction if you're already writing Harbor tasks and don't want to hand-roll scoring logic.

Install to Claude Code

npx -y skills add harbor-framework/harbor --skill rewardkit --agent claude-code