Summary
Wraps Ultralytics YOLOE to expose zero-shot object detection and segmentation through two MCP tools. You pass an image as a local path, URL, or Base64 string, optionally specify text prompts like "blue coffee cup next to the spoon", and get back bounding boxes or polygon masks without training a model. Uses yoloe-26l-seg.pt by default, which hits 55.0 mAP on COCO at 6.2ms on T4. Reach for this when you need Claude or another agent to parse objects from images using arbitrary natural language queries instead of fixed class lists.
Fastest browser for AI agents to run web automation tasks, always free.
Download Free life-time →
belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →
Free an open-source skills that make AI coding agents easier to understand, verify, and control.
Download for free →
Connect your Claude agent to live crypto prices and trading routes via 1inch
Get the MCP →
Notes, actions and memory. Without a meeting bot. First month 100% off.
Download for free →MCP-YOLO
MCP-YOLO is an agent-first development platform that provides Zero-Shot Object Detection and Segmentation as a Model Context Protocol (MCP) server. Powered by Ultralytics YOLOE, it enables developers and AI agents to detect and segment objects using arbitrary text prompts without retraining.
Key Features
- Zero-Shot Detection: Detect any object using natural language (e.g., "the blue coffee cup next to the spoon").
- Instance Segmentation: Precise polygon masks for discovered objects.
- Flexible Image Inputs: Supports local file paths, remote URLs, and Base64 encoded strings.
- Agent Optimized: Includes custom "Skills" for autonomous deployment and benchmarking.
YOLOE Performance Reference
YOLOE builds upon the latest YOLO architectures (like YOLO11 and YOLO26) to provide state-of-the-art open-vocabulary performance.
| Model | Based On | mAP (COCO) | Speed (T4/ms) | Params (M) |
|---|---|---|---|---|
| YOLOE26-N | YOLO26-N | 40.9 | 1.7 | ~3.0 |
| YOLOE26-S | YOLO26-S | 48.6 | 2.5 | ~10.0 |
| YOLOE26-L | YOLO26-L | 55.0 | 6.2 | ~40.0 |
| YOLOE-L | YOLO11-L | ~52.0 | ~5.0 | ~26.0 |
Note: Performance varies depending on the hardware and input resolution. mcp-yolo uses yoloe-26l-seg.pt by default for high precision.
Quick Start
Installation
uv pip install mcp-yolo
Running the Server
uv run mcp-yolo
MCP Tools
detect_objects
Performs zero-shot detection.
- Arguments:
image_source(str): Path, URL, or Base64.classes(list[str], optional): Custom text prompts to detect.
segment_objects
Performs zero-shot instance segmentation.
- Arguments:
image_source(str): Path, URL, or Base64.classes(list[str], optional): Custom text prompts to segment.
Publishing
This project is configured for automated PyPI publishing. See the pypi_setup_guide.md for details.
Featured
Fastest browser for AI agents to run web automation tasks, always free.
Download Free life-time →belt cli automatically finds the best tools and skills for your agent. image, video, music, tts...
one prompt install →Free an open-source skills that make AI coding agents easier to understand, verify, and control.
Download for free →Connect your Claude agent to live crypto prices and trading routes via 1inch
Get the MCP →Notes, actions and memory. Without a meeting bot. First month 100% off.
Download for free →