4. Running External Reinforcement Learning Examples

A set of reinforcement learning examples are provided in an external GitHub repository: OmniIsaacGymEnvs

4.1. Learning Objectives

In this tutorial, we will set up our external reinforcement learning example repository: OmniIsaacGymEnvs. We will

  1. Install OmniIsaacGymEnvs for Isaac Sim

  2. Running inferencing and training example in OmniIsaacGymEnvs

10-15 Minute Tutorial

4.2. Getting Started

4.3. Installing Examples Repository

To set up these examples, first clone the repository:

git clone https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs.git

We can install the examples as a python module in Isaac Sim. Locate the Isaac Sim python executable, which by default should be python.sh on Linux or python.bat on Windows, located at the root of the Isaac Sim directory. We will refer to this path as PYTHON_PATH.

To set a PYTHON_PATH variable in the terminal that links to the python executable, we can run a command that resembles the following. Make sure to update the paths to your local path.

For Linux: alias PYTHON_PATH=~/.local/share/ov/pkg/isaac_sim-*/python.sh
For Windows: doskey PYTHON_PATH=C:\Users\user\AppData\Local\ov\pkg\isaac_sim-*\python.bat $*

Install OmniIsaacGymEnvs to PYTHON_PATH by running the following from the root of OmniIsaacGymEnvs:

PYTHON_PATH -m pip install -e .

The following error may appear during the initial installation. This error is harmless and can be ignored.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.

4.4. Running Examples

Example scripts should be launched from omniisaacgymenvs/omniisaacgymenvs.

4.4.1. Launching Training Examples

To train your first policy, run:

PYTHON_PATH scripts/rlgames_train.py task=Cartpole

We will see an Isaac Sim window pop up. Once Isaac Sim initialization completes (which may take a few minutes if launching for the first time), the Cartpole scene will be constructed and simulation will start running automatically. The process will terminate once training finishes.

../_images/isaac_sim_rl_cartpole.gif

4.4.2. Running Inference

To load a trained checkpoint and perform inference (no training), pass test=True as an argument, along with the checkpoint name.

PYTHON_PATH scripts/rlgames_train.py task=Cartpole test=True checkpoint=runs/Cartpole/nn/Cartpole.pth

4.4.3. Inferencing with Pre-Trained Checkpoints

Pre-trained checkpoints are provided for each task on the Nucleus server, under Assets/Isaac/2022.1/Isaac/Samples/OmniIsaacGymEnvs/Checkpoints.

To load a pre-trained checkpoint and run inferencing, run:

PYTHON_PATH scripts/rlgames_train.py task=Cartpole test=True checkpoint=omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/OmniIsaacGymEnvs/Checkpoints/cartpole.pth

4.5. Summary

This tutorial covered the following topics:

  1. Installation of OmniIsaacGymEnvs

  2. Running training examples in OmniIsaacGymEnvs

  3. Running inferencing examples in OmniIsaacGymEnvs

4.5.1. Next Steps

Continue on to the next tutorial in our Reinforcement Learning Tutorials series, Transferring Policies from Isaac Gym Preview Releases, to read about tips on transferring policies from standalone Isaac Gym to Isaac Sim.

4.5.2. Further Learning

  • For more details on the RL examples, please refer to the README page in OmniIsaacGymEnvs.