Isaacsim Mcp Server

Isaac Sim MCP Server

Natural language control for NVIDIA Isaac Sim through the Model Context Protocol (MCP).

Connect any MCP-compatible IDE (Cursor, VS Code, Claude Code, Windsurf, Antigravity, JetBrains) to a running Isaac Sim instance and control it with plain-English prompts -- create robots, build scenes, run simulations, and debug physics all from your editor.

Highlights

• 42 tools across 9 categories -- scene, objects, lighting, robots, sensors, materials, assets, simulation, graphs
• 107+ robots auto-discovered from the Isaac Sim asset library (Franka, UR, Unitree, Boston Dynamics, and more)
• Step-and-observe debugging -- step the simulation and inspect prim positions, joint states, and physics in one call
• Hot-reload -- iterate on Python controllers without restarting Isaac Sim
• Multi-instance -- run multiple Isaac Sim sessions side by side on different ports
• Built for Isaac Sim 5.1.0 - 6.0.1 (PhysX + Newton) with a modular adapter layer for version isolation

Installation

Option A: pip install (recommended)

pip install isaacsim-mcp-server

This installs the MCP server and the isaacsim-mcp-server CLI. You still need the Isaac Sim extension from the repo (see Launching Isaac Sim below).

Option B: From source

git clone https://github.com/whats2000/isaacsim-mcp-server
cd isaacsim-mcp-server
./scripts/setup_python_env.sh

Requirements

[!IMPORTANT] Linux and Windows are supported. On Windows, use the PowerShell launcher scripts/run_isaac_sim.ps1 in place of the .sh scripts (see below). macOS is not supported because NVIDIA Isaac Sim does not run on macOS.

[!NOTE] We are welcoming contributions to support other Isaac Sim versions. The adapter layer is designed for easy version isolation.

Quick Start

1. Set up the environment

If you installed from source:

./scripts/setup_python_env.sh

On Windows, uv sync creates the virtual environment (.venv) and installs the package plus its dependencies:

uv sync

2. Launch Isaac Sim with the extension

./scripts/run_isaac_sim.sh

You should see in the logs:

Registered 42 command handlers
Isaac Sim MCP server started on localhost:8766

The script looks for Isaac Sim in $HOME/isaacsim; set ISAACSIM_ROOT to use a different install.

Choosing the physics engine. Isaac Sim 6.0+ ships PhysX (default) and Newton backends. Select one with --newton / --physx, or ISAACSIM_ENGINE:

./scripts/run_isaac_sim.sh                  # PhysX (default)
./scripts/run_isaac_sim.sh --newton         # Newton
ISAACSIM_ENGINE=newton ./scripts/run_isaac_sim.sh

The same flags work with scripts/launch_isaac_sim_mcp.sh. Everything else on the command line is forwarded to Kit untouched. The server auto-detects the active engine, so no MCP-side configuration changes. Newton requires 6.0 or newer; asking for it on 5.1.0 fails with a clear message.

On Windows, use the PowerShell launcher instead. It takes the same engine selection and forwards extra arguments to Kit:

.\scripts\run_isaac_sim.ps1                          # PhysX (default)
.\scripts\run_isaac_sim.ps1 -Engine newton           # Newton
$env:ISAACSIM_ENGINE = 'newton'; .\scripts\run_isaac_sim.ps1

The script resolves the install from -IsaacSimRoot, then $env:ISAACSIM_ROOT, then a local source build, then C:\isaacsim, then %USERPROFILE%\isaacsim. It also creates a writable USD working directory (.cache\usd) since Windows has no /tmp.

export BEAVER3D_MODEL="<your beaver3d model name>"
export ARK_API_KEY="<your beaver3d api key>"
export NVIDIA_API_KEY="<your nvidia api key>"

$env:BEAVER3D_MODEL = "<your beaver3d model name>"
$env:ARK_API_KEY = "<your beaver3d api key>"
$env:NVIDIA_API_KEY = "<your nvidia api key>"

3. Connect your IDE

Add the MCP server to your editor. Replace the path with your actual repo location. The command examples are for Linux/macOS; each guide shows the Windows equivalent, which wraps the PowerShell launcher scripts\run_mcp_server.ps1.

claude mcp add isaac-sim /path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh

{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}

claude mcp add isaac-sim -- powershell -NoProfile -ExecutionPolicy Bypass -File C:\path\to\isaacsim-mcp-server\scripts\run_mcp_server.ps1

{
  "mcpServers": {
    "isaac-sim": {
      "command": "powershell",
      "args": ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", "C:\\path\\to\\isaacsim-mcp-server\\scripts\\run_mcp_server.ps1"]
    }
  }
}

{
  "servers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}

{
  "servers": {
    "isaac-sim": {
      "command": "powershell",
      "args": ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", "C:\\path\\to\\isaacsim-mcp-server\\scripts\\run_mcp_server.ps1"]
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "powershell",
      "args": ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", "C:\\path\\to\\isaacsim-mcp-server\\scripts\\run_mcp_server.ps1"]
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "powershell",
      "args": ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", "C:\\path\\to\\isaacsim-mcp-server\\scripts\\run_mcp_server.ps1"]
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "powershell",
      "args": ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", "C:\\path\\to\\isaacsim-mcp-server\\scripts\\run_mcp_server.ps1"]
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}

{
  "mcpServers": {
    "isaac-sim": {
      "command": "powershell",
      "args": ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", "C:\\path\\to\\isaacsim-mcp-server\\scripts\\run_mcp_server.ps1"]
    }
  }
}

4. Start prompting

Check the connection with get_scene_info.
If the scene is empty, create a physics scene.
Add a Franka robot at the origin and a Go1 quadruped at [2, 0, 0].

Architecture

MCP Client (IDE)
      |
      v
isaacsim-mcp-server          (PyPI package / CLI)
      |
      v  TCP socket (localhost:8766)
      |
isaac.sim.mcp_extension      (Omniverse extension)
      |
      v
Handlers -> Adapter -> Isaac Sim 5.1 / 6.0 APIs

Tools

42 tools across 9 categories:

Known Limitations

Open defects a normal session can hit. Each is warned about at the point of use where that is possible; this list is for choosing a runtime before you start.

Example Prompts

Scene bootstrap

Check the connection with get_scene_info. If the scene is empty, create a physics scene.
Add stronger lighting and place a camera that looks at the workspace.

Robot layout

Create three Franka robots in a row at [0,0,0], [2,0,0], and [4,0,0].
Then add a Go1 robot at [1, 3, 0].

Environment loading

List available environments, choose a warehouse-like one, and load it.
Create a camera and capture an image.

Asset search and 3D generation

Search for a rusty desk, load the best result near [0, 5, 0], scaled to [2, 2, 2].

Advanced Usage

Multiple Instances

Run multiple Isaac Sim sessions side by side. Each uses a different port (auto-assigned from 8766).

# First instance (default port 8766)
claude mcp add isaac-sim /path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh

# Second instance (port 8767)
claude mcp add isaac-sim-2 -e ISAAC_MCP_PORT=8767 -- /path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh

{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    },
    "isaac-sim-2": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh",
      "env": { "ISAAC_MCP_PORT": "8767" }
    }
  }
}

Desktop Launcher (Linux)

Install a dedicated Isaac Sim MCP application icon:

./scripts/install_desktop_entry.sh

This creates a launcher that auto-assigns ports, waits for the extension socket, and cleans up on exit.

Recommended Workflow

1. Start with get_scene_info to verify the connection
2. Create a physics scene if the stage is empty
3. Prefer purpose-built tools before execute_script
4. Use list_available_robots / list_environments before loading
5. Use create_action_graph to wire OnPlaybackTick → ScriptNode controllers
6. Use step_simulation with observe_prims and observe_joints for debugging
7. Use reload_script to iterate on controllers without restarting

Demo: Franka Pick-and-Place

A ready-to-run demo at demo/franka_pick_place.py using RMPflow for motion planning:

Please use the Isaac MCP tool complete this:

Create a physics scene with a ground plane, then spawn a Franka FR3 robot at the origin.

Add two textured tables with a gap along Y. Place a small textured cube with physics enabled on top of the first table.

Use `create_action_graph` to wire `OnPlaybackTick` → `ScriptNode`, and write a pick-and-place controller script using RMPflow for motion planning. Save the script to the `demo/` directory.

Use `get_prim_info` to query actual positions and sizes of the tables and cube before writing the controller — do not hardcode coordinates.

Start the simulation with Play. The robot should pick the cube from table 1 and place it on table 2. Verify the process using `step_simulation` with `observe_prims` on the cube to confirm it reaches table 2.

Uses create_action_graph with script_file for one-step Action Graph + ScriptNode setup, plus the observability tools: get_joint_config, step_simulation with observe_prims, get_physics_state, and edit_action_graph for script hot-reload.

Development

# Run the MCP inspector
./.venv/bin/python -m mcp dev ./isaac_mcp/server.py

The inspector is available at http://localhost:5173.

Setup Notes

Override defaults:

PYTHON_SPEC=3.11 ./scripts/setup_python_env.sh
ISAACSIM_ROOT=/opt/isaacsim ./scripts/run_isaac_sim.sh
ISAACSIM_ENGINE=newton ./scripts/run_isaac_sim.sh

Engine selection lives in scripts/lib/isaac_launcher.sh: each engine maps to the launcher script Isaac Sim ships for it. Adding an entry to that map is all a new backend needs — it enables both ISAACSIM_ENGINE=<name> and --<name> in every launcher script.

# Make sure you're in the repo root
pwd
test -f ./isaac.sim.mcp_extension/config/extension.toml && echo OK

Contributing

Pull requests are welcome. Improvements to tools, docs, adapters, and tests are all useful.

License

MIT License. Copyright (c) 2023-2025 omni-mcp, Copyright (c) 2026 whats2000. See LICENSE.