What Is Isaac Sim?¶
NVIDIA Omniverse™ Isaac Sim is a robotics simulation toolkit for the NVIDIA Omniverse™ platform. Isaac Sim has essential features for building virtual robotic worlds and experiments. It provides researchers and practitioners with the tools and workflows they need to create robust, physically accurate simulations and synthetic datasets. Isaac Sim supports navigation and manipulation applications through ROS/ROS2. It simulates sensor data from sensors such as RGB-D, Lidar, and IMU for various computer vision techniques such as domain randomization, ground-truth labeling, segmentation, and bounding boxes.
Omniverse Isaac Sim uses the NVIDIA Omniverse™ Kit SDK, a toolkit for building native Omniverse applications and microservices. Omniverse Kit provides a wide variety of functionality through a set of light-weight plugins. Plugins are authored with C interfaces for persistent API compatibility; however, a Python interpreter is also provided for accessible scripting and customization.
The Python API can be used to write new extensions to Omniverse Kit or new experiences for Omniverse.
For a more in-depth look at developing in Kit, see the Kit Programming Manual.
Omniverse Isaac Sim uses NVIDIA Omniverse™ Nucleus to access content such as USD files for environments and robots. Omniverse Nucleus services allow a variety of client applications, renderers, and microservices to share and modify representations of virtual worlds in Omniverse Isaac Sim.
Nucleus operates under a publish/subscribe model. Subject to access controls, Omniverse clients can publish modifications to digital assets and virtual worlds to the Nucleus Database (DB) or subscribe to their changes. Changes are transmitted in real-time between connected applications. Digital assets can include geometry, lights, materials, textures and other data that describe virtual worlds and their evolution through time.
This allows a variety of Omniverse-enabled client applications (Apps, Connectors, and others) to share and modify authoritative representations of virtual worlds.
See Nucleus Overview for a more in-depth look at the Nucleus data model, architecture, and distribution platforms.
Omniverse Isaac Sim uses the USD interchange file format to represent scenes. Universal Scene Description (USD) is an easily extensible, open-source 3D scene description and file format developed by Pixar for content creation and interchange among different tools. Because of its power and versatility, USD is being adopted widely, not only in the visual effects community, but also in architecture, design, robotics, manufacturing, and other disciplines.
The documentation is split into the following sections: Introduction, Installation Guide, API Documentation, Tutorials, Manuals, and Reference Materials.
The Introduction section is a basic overview of Omniverse Isaac Sim and how it fits into the ecosystem of NVIDIA Omniverse™. It also outlines the architecture, workflow of Omniverse Isaac Sim, and the documentation layout.
The Basic Isaac Sim Installation is the recommended installation process for all users.
Omniverse Isaac Sim also supports more advanced, headless, and remote installation options. Docker, Cloud, and Streaming-related setups are covered in the Deploy to Cloud guide.
Python API Documentation for both Omniverse and Isaac Sim extensions can be found here.
Tutorials are grouped in the categories listed below. We strongly recommend completing the Introductory Tutorials, GUI Tutorials, and Core API Tutorials in order to understand all the ways Omniverse Isaac Sim can be used, before moving on to the more focused tutorials.
Introductory Tutorials: The fundamentals of working in Omniverse Kit and different workflows in Omniverse Isaac Sim.
GUI Tutorials: The fundamental concepts of robotics in Omniverse Isaac Sim via GUI.
Core API Tutorials: Building environments, robots, and tasks using Omniverse Isaac Sim Core Python APIs.
Isaac Gym Tutorials: Reinforcement learning framework and Cloner APIs.
Replicator Tutorials: Synthetic data generation.
ROS Tutorials: Examples of various ROS bridges and interfaces.
ROS2 Tutorials: Examples of various ROS2 bridges and interfaces.
Isaac Cortex Tutorials: High level behavioral programming.
Advanced Tutorials: Specific topics.
The Manuals section provides documentation for various extensions, additional examples, and features provided in Isaac Sim. These generally cover APIs or details about a specific topic and are less tutorialized.
The Reference Material section provides references, links, and resources to assist developers.
This section also provides a suite of useful examples for users to quickly try intermediate and advanced applications in Omniverse Isaac Sim. Each example includes well-commented source code that users grab and modify for their own use cases.
See the complete list of examples in the Isaac Examples Menu.