4. Running External Reinforcement Learning Examples¶
A set of reinforcement learning examples are provided in an external GitHub repository: OmniIsaacGymEnvs
4.1. Learning Objectives¶
In this tutorial, we will set up our external reinforcement learning example repository: OmniIsaacGymEnvs. We will
Install OmniIsaacGymEnvs for Isaac Sim
Running inferencing and training example in OmniIsaacGymEnvs
10-15 Minute Tutorial
4.2. Getting Started¶
4.3. Installing Examples Repository¶
To set up these examples, first clone the repository:
git clone https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs.git
We can install the examples as a python module in Isaac Sim. Locate the Isaac Sim python executable, which by default should be python.sh on Linux or python.bat on Windows, located at the root of the Isaac Sim directory. We will refer to this path as PYTHON_PATH.
To set a PYTHON_PATH variable in the terminal that links to the python executable, we can run a command that resembles the following. Make sure to update the paths to your local path.
For Linux: alias PYTHON_PATH=~/.local/share/ov/pkg/isaac_sim-*/python.sh For Windows: doskey PYTHON_PATH=C:\Users\user\AppData\Local\ov\pkg\isaac_sim-*\python.bat $*
Install OmniIsaacGymEnvs to PYTHON_PATH by running the following from the root of OmniIsaacGymEnvs:
PYTHON_PATH -m pip install -e .
The following error may appear during the initial installation. This error is harmless and can be ignored.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
4.4. Running Examples¶
Example scripts should be launched from omniisaacgymenvs/omniisaacgymenvs.
4.4.1. Launching Training Examples¶
To train your first policy, run:
PYTHON_PATH scripts/rlgames_train.py task=Cartpole
We will see an Isaac Sim window pop up. Once Isaac Sim initialization completes (which may take a few minutes if launching for the first time), the Cartpole scene will be constructed and simulation will start running automatically. The process will terminate once training finishes.
4.4.2. Running Inference¶
To load a trained checkpoint and perform inference (no training), pass test=True as an argument, along with the checkpoint name.
PYTHON_PATH scripts/rlgames_train.py task=Cartpole test=True checkpoint=runs/Cartpole/nn/Cartpole.pth
4.4.3. Inferencing with Pre-Trained Checkpoints¶
Pre-trained checkpoints are provided for each task on the Nucleus server, under Assets/Isaac/2022.1/Isaac/Samples/OmniIsaacGymEnvs/Checkpoints.
To load a pre-trained checkpoint and run inferencing, run:
PYTHON_PATH scripts/rlgames_train.py task=Cartpole test=True checkpoint=omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/OmniIsaacGymEnvs/Checkpoints/cartpole.pth
This tutorial covered the following topics:
Installation of OmniIsaacGymEnvs
Running training examples in OmniIsaacGymEnvs
Running inferencing examples in OmniIsaacGymEnvs
4.5.1. Next Steps¶
Continue on to the next tutorial in our Reinforcement Learning Tutorials series, Transferring Policies from Isaac Gym Preview Releases, to read about tips on transferring policies from standalone Isaac Gym to Isaac Sim.