Summary
Wraps Ultralytics YOLOE to expose zero-shot object detection and segmentation through two MCP tools. You pass an image as a local path, URL, or Base64 string, optionally specify text prompts like "blue coffee cup next to the spoon", and get back bounding boxes or polygon masks without training a model. Uses yoloe-26l-seg.pt by default, which hits 55.0 mAP on COCO at 6.2ms on T4. Reach for this when you need Claude or another agent to parse objects from images using arbitrary natural language queries instead of fixed class lists.
Notes, actions and memory. Without a meeting bot. First month 100% off.
Download for free →
Keep your Mac awake while Claude Code and 40+ AI agents run. Sleeps when they're idle.
One time payment $9 →
Give your AI agent a complete email layer—sending, inbound inboxes, and sandbox testing.
Get 4K emails/month free →
Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →
Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →
Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →MCP-YOLO
MCP-YOLO is an agent-first development platform that provides Zero-Shot Object Detection and Segmentation as a Model Context Protocol (MCP) server. Powered by Ultralytics YOLOE, it enables developers and AI agents to detect and segment objects using arbitrary text prompts without retraining.
Key Features
- Zero-Shot Detection: Detect any object using natural language (e.g., "the blue coffee cup next to the spoon").
- Instance Segmentation: Precise polygon masks for discovered objects.
- Flexible Image Inputs: Supports local file paths, remote URLs, and Base64 encoded strings.
- Agent Optimized: Includes custom "Skills" for autonomous deployment and benchmarking.
YOLOE Performance Reference
YOLOE builds upon the latest YOLO architectures (like YOLO11 and YOLO26) to provide state-of-the-art open-vocabulary performance.
| Model | Based On | mAP (COCO) | Speed (T4/ms) | Params (M) |
|---|---|---|---|---|
| YOLOE26-N | YOLO26-N | 40.9 | 1.7 | ~3.0 |
| YOLOE26-S | YOLO26-S | 48.6 | 2.5 | ~10.0 |
| YOLOE26-L | YOLO26-L | 55.0 | 6.2 | ~40.0 |
| YOLOE-L | YOLO11-L | ~52.0 | ~5.0 | ~26.0 |
Note: Performance varies depending on the hardware and input resolution. mcp-yolo uses yoloe-26l-seg.pt by default for high precision.
Quick Start
Installation
uv pip install mcp-yolo
Running the Server
uv run mcp-yolo
MCP Tools
detect_objects
Performs zero-shot detection.
- Arguments:
image_source(str): Path, URL, or Base64.classes(list[str], optional): Custom text prompts to detect.
segment_objects
Performs zero-shot instance segmentation.
- Arguments:
image_source(str): Path, URL, or Base64.classes(list[str], optional): Custom text prompts to segment.
Publishing
This project is configured for automated PyPI publishing. See the pypi_setup_guide.md for details.
Featured
Notes, actions and memory. Without a meeting bot. First month 100% off.
Download for free →Keep your Mac awake while Claude Code and 40+ AI agents run. Sleeps when they're idle.
One time payment $9 →Give your AI agent a complete email layer—sending, inbound inboxes, and sandbox testing.
Get 4K emails/month free →Integrate web data into your AI product. One API to scrape website & brand data.
Get API Key Now →Your agent targets a perfect 10 Code Health score. Deterministic. Every commit.
Try For Free →Agent, run crypto. Access onchain data & trade routes via 1inch.
Install now →