Isaac Sim MCP

The Isaac Sim MCP extension controls NVIDIA Isaac Sim through natural language, enabling robot simulation, scene creation, and dynamic interactions, and connects the MCP ecosystem with embodied intelligence applications.

Virtualization Developer tools #Natural Language Control #Robot Simulation #Scene Interaction .Python

rating : 2.5 points

downloads : 39

update time : 2025-04-28

What is the MCP Server?

The MCP server is a tool based on the Model Context Protocol framework that allows users to operate and control NVIDIA Isaac Sim through simple natural language instructions. It supports dynamic robot positioning, scene editing, lighting adjustment, and complex tasks such as navigation and obstacle handling.

How to Use the MCP Server?

First, install the MCP server and enable the extension plugin. Then, set up the MCP server connection in Cursor AI. Finally, enter natural language instructions to start the simulation.

Applicable Scenarios

Suitable for developers and technicians who need to quickly build robot environments, test AI algorithms, or conduct virtual simulation training.

Main Features

Natural Language ControlSupports users to operate Isaac Sim through simple and understandable language instructions, such as 'Create a Franka robot'.

Dynamic Robot PositioningCan precisely define the initial positions of robots and move them according to requirements.

Custom Lighting EffectsAllows adjustment of lighting conditions in the scene to meet the needs of specific experiments or demonstrations.

Obstacle NavigationSimulates complex robot movement scenarios, including the ability to bypass obstacles.

Code PreviewView the generated Python code before execution to ensure the instructions are accurate.

Advantages and Limitations

Advantages

Easy to get started, no need to have in - depth programming knowledge to operate.

Supports multi - robot collaboration and complex task planning.

Highly flexible, can customize scenes according to requirements.

Provides real - time feedback, facilitating debugging and optimizing model performance.

Limitations

Has certain requirements for hardware configuration, it is recommended to run on a high - performance computer.

Some advanced functions may require additional learning costs.

Network latency may affect the real - time interaction experience.

How to Use

Install the MCP Server

Clone the GitHub repository and point the project folder to your local directory.

Enable the Extension Plugin

Load the extension plugin in Isaac Sim and configure the API key.

Start the MCP Server

Run the script in the terminal to start the MCP server.

Configure Cursor AI

Add the MCP server address in Cursor preferences.

Usage Examples

Robot PartyCreate multiple Franka robots and arrange lights on the stage.

Factory LayoutDeploy a series of industrial robots for production line simulation.

Generate Beaver3D ModelGenerate a 3D model from an image and place it in the grid area.

Frequently Asked Questions

Does the MCP server support multiple robot types?

How to solve network connection problems?

Can I use this tool without being familiar with programming?

Related Resources

GitHub Repository

Get the latest version and source code.

Official Documentation

Learn the basics and best practices of Isaac Sim.

Tutorial Video

Watch detailed installation and usage tutorials.

🚀 Isaac Sim MCP Extension and Server

The NVIDIA Isaac Sim MCP extension offers a suite of professional tools accessible via natural language in Cursor AI. These tools empower you to control and manipulate NVIDIA Isaac Sim with straightforward commands.

🚀 Quick Start

The NVIDIA Isaac Sim MCP extension provides a set of professional tools that can be accessed via natural language in Cursor AI. These tools allow you to control and operate NVIDIA Isaac Sim through simple commands.

✨ Features

get_scene_info - Check the status of the Isaac Sim extension server and obtain basic scene information.
create_physics_scene - Create a physics scene with configuration parameters:
- objects: A list of objects to create (each object has a type and position).
- floor: Whether to create a ground plane (default: true).
- gravity: A vector defining the direction and magnitude of gravity (default: [0, -0.981, 0]).
- scene_name: The name of the scene (default: "physics_scene").
create_robot - Create a robot in the scene:
- robot_type: The type of robot to create (options: "franka", "jetbot", "carter", "g1", "go1").
- position: [x, y, z] coordinates.
omni_kit_command - Execute an Omni Kit command:
- command: The Omni Kit command to execute (e.g., "CreatePrim").
- prim_type: The primitive type of the command (e.g., "Sphere").
execute_script - Execute arbitrary Python code in Isaac Sim:
- code: The Python code to execute.