RL [omni.isaac.gym]¶
Base Environment Wrapper¶
-
class
SimulationApp
(launch_config: Optional[dict] = None, experience: str = '')¶ Helper class to launch Omniverse Toolkit.
Omniverse loads various plugins at runtime which cannot be imported unless the Toolkit is already running. Thus, it is necessary to launch the Toolkit first from your python application and then import everything else.
Usage:
# At top of your application from omni.isaac.kit import SimulationApp config = { width: "1280", height: "720", headless: False, } simulation_app = SimulationApp(config) # Rest of the code follows ... simulation_app.close()
Note
The settings in
DEFAULT_LAUNCHER_CONFIG
are overwritten by those inconfig
.- Parameters
config (dict) – A dictionary containing the configuration for the app. (default: None)
experience (str) – Path to the application config loaded by the launcher (default: “”, will load app/omni.isaac.sim.python.kit if left blank)
-
DEFAULT_LAUNCHER_CONFIG
= {'active_gpu': None, 'anti_aliasing': 3, 'denoiser': True, 'display_options': 3094, 'headless': True, 'height': 720, 'livesync_usd': None, 'max_bounces': 4, 'max_specular_transmission_bounces': 6, 'max_volume_bounces': 4, 'memory_report': False, 'multi_gpu': True, 'open_usd': None, 'renderer': 'RayTracedLighting', 'samples_per_pixel_per_frame': 64, 'subdiv_refinement_level': 0, 'sync_loads': True, 'width': 1280, 'window_height': 900, 'window_width': 1440}¶ The config variable is a dictionary containing the following entries
- Parameters
headless (bool) – Disable UI when running. Defaults to True
active_gpu (int) – Specify the GPU to use when running, set to None to use default value which is usually the first gpu, default is None
multi_gpu (bool) – Set to true to enable Multi GPU support, Defaults to true
sync_loads (bool) – When enabled, will pause rendering until all assets are loaded. Defaults to True
width (int) – Width of the viewport and generated images. Defaults to 1280
height (int) – Height of the viewport and generated images. Defaults to 720
window_width (int) – Width of the application window, independent of viewport, defaults to 1440,
window_height (int) – Height of the application window, independent of viewport, defaults to 900,
display_options (int) – used to specify whats visible in the stage by default. Defaults to 3094 so extra objects do not appear in synthetic data. 3286 is another good default, used for the regular isaac-sim editor experience
subdiv_refinement_level (int) – Number of subdivisons to perform on supported geometry. Defaults to 0
renderer (str) – Rendering mode, can be RayTracedLighting or PathTracing. Defaults to PathTracing
anti_aliasing (int) – Antialiasing mode, 0: Disabled, 1: TAA, 2: FXAA, 3: DLSS, 4:RTXAA
samples_per_pixel_per_frame (int) – The number of samples to render per frame, increase for improved quality, used for PathTracing only. Defaults to 64
denoiser (bool) – Enable this to use AI denoising to improve image quality, used for PathTracing only. Defaults to True
max_bounces (int) – Maximum number of bounces, used for PathTracing only. Defaults to 4
max_specular_transmission_bounces (int) – Maximum number of bounces for specular or transmission, used for PathTracing only. Defaults to 6
max_volume_bounces (int) – Maximum number of bounces for volumetric materials, used for PathTracing only. Defaults to 4
open_usd (str) – This is the name of the usd to open when the app starts. It will not be saved over. Default is None and an empty stage is created on startup.
livesync_usd (str) – This is the location of the usd that you want to do your interactive work in. The existing file is overwritten. Default is None
memory_report (bool) – Set to true to print a memory usage report on exit. Default is False
-
property
app
¶ omniverse kit application object
- Type
omni.kit.app.IApp
-
close
() → None¶ Close the running Omniverse Toolkit.
-
property
context
¶ the current USD context
- Type
omni.usd.UsdContext
-
is_exiting
() → bool¶ bool: True if close() was called previously, False otherwise
-
is_running
() → bool¶ bool: convenience function to see if app is running. True if running, False otherwise
-
reset_render_settings
()¶ Reset render settings to those in config.
Note
This should be used in case a new stage is opened and the desired config needs to be re-applied.
-
set_setting
(setting: str, value) → None¶ Set a carbonite setting
- Parameters
setting (str) – carb setting path
value – value to set the setting to, type is used to properly set the setting.
-
update
() → None¶ Convenience function to step the application forward one frame
-
class
VecEnvBase
(headless: bool)¶ This class provides a base interface for connecting RL policies with task implementations. APIs provided in this interface follow the interface in gym.Env. This class also provides utilities for initializing simulation apps, creating the World, and registering a task.
-
action_space
= None¶
-
close
() → None¶ Closes simulation.
-
metadata
= {'render.modes': []}¶
-
property
num_envs
¶ Retrieves number of environments.
- Returns
Number of environments.
- Return type
num_envs(int)
-
observation_space
= None¶
-
render
(mode='human') → None¶ Step the renderer.
- Parameters
mode (str) – Select mode of rendering based on OpenAI environments.
-
reset
()¶ Resets the task and updates observations.
-
reward_range
= (-inf, inf)¶
-
seed
(seed=- 1)¶ Sets a seed. Pass in -1 for a random seed.
- Parameters
seed (int) – Seed to set. Defaults to -1.
- Returns
Seed that was set.
- Return type
seed (int)
-
set_task
(task, backend='numpy', sim_params=None, init_sim=True) → None¶ - Creates a World object and adds Task to World.
Initializes and registers task to the environment interface. Triggers task start-up.
- Parameters
task (RLTask) – The task to register to the env.
backend (str) – Backend to use for task. Can be “numpy” or “torch”. Defaults to “numpy”.
sim_params (dict) – Simulation parameters for physics settings. Defaults to None.
init_sim (Optional[bool]) – Automatically starts simulation. Defaults to True.
-
spec
= None¶
-
step
(actions)¶ - Basic implementation for stepping simulation.
Can be overriden by inherited Env classes to satisfy requirements of specific RL libraries. This method passes actions to task for processing, steps simulation, and computes observations, rewards, and resets.
- Parameters
actions (Union[numpy.ndarray, torch.Tensor]) – Actions buffer from policy.
- Returns
Buffer of observation data. rewards(Union[numpy.ndarray, torch.Tensor]): Buffer of rewards data. dones(Union[numpy.ndarray, torch.Tensor]): Buffer of resets/dones data. info(dict): Dictionary of extras data.
- Return type
observations(Union[numpy.ndarray, torch.Tensor])
-
property
unwrapped
¶ Completely unwrap this env.
- Returns
The base non-wrapped gym.Env instance
- Return type
gym.Env
-
-
abstractmethod
(funcobj)¶ A decorator indicating abstract methods.
Requires that the metaclass is ABCMeta or derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods are overridden. The abstract methods can be called using any of the normal ‘super’ call mechanisms.
Usage:
- class C(metaclass=ABCMeta):
@abstractmethod def my_abstract_method(self, …):
…
Multi-Threaded Environment Wrapper¶
-
exception
TaskStopException
¶ Exception class for signalling task termination.
-
args
¶
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
class
TrainerMT
¶ A base abstract trainer class for controlling starting and stopping of RL policy.
-
abstract
run
()¶ Runs RL loop in a new thread
-
abstract
stop
()¶ Stop RL thread
-
abstract
-
class
VecEnvBase
(headless: bool)¶ This class provides a base interface for connecting RL policies with task implementations. APIs provided in this interface follow the interface in gym.Env. This class also provides utilities for initializing simulation apps, creating the World, and registering a task.
-
action_space
= None¶
-
close
() → None¶ Closes simulation.
-
metadata
= {'render.modes': []}¶
-
property
num_envs
¶ Retrieves number of environments.
- Returns
Number of environments.
- Return type
num_envs(int)
-
observation_space
= None¶
-
render
(mode='human') → None¶ Step the renderer.
- Parameters
mode (str) – Select mode of rendering based on OpenAI environments.
-
reset
()¶ Resets the task and updates observations.
-
reward_range
= (-inf, inf)¶
-
seed
(seed=- 1)¶ Sets a seed. Pass in -1 for a random seed.
- Parameters
seed (int) – Seed to set. Defaults to -1.
- Returns
Seed that was set.
- Return type
seed (int)
-
set_task
(task, backend='numpy', sim_params=None, init_sim=True) → None¶ - Creates a World object and adds Task to World.
Initializes and registers task to the environment interface. Triggers task start-up.
- Parameters
task (RLTask) – The task to register to the env.
backend (str) – Backend to use for task. Can be “numpy” or “torch”. Defaults to “numpy”.
sim_params (dict) – Simulation parameters for physics settings. Defaults to None.
init_sim (Optional[bool]) – Automatically starts simulation. Defaults to True.
-
spec
= None¶
-
step
(actions)¶ - Basic implementation for stepping simulation.
Can be overriden by inherited Env classes to satisfy requirements of specific RL libraries. This method passes actions to task for processing, steps simulation, and computes observations, rewards, and resets.
- Parameters
actions (Union[numpy.ndarray, torch.Tensor]) – Actions buffer from policy.
- Returns
Buffer of observation data. rewards(Union[numpy.ndarray, torch.Tensor]): Buffer of rewards data. dones(Union[numpy.ndarray, torch.Tensor]): Buffer of resets/dones data. info(dict): Dictionary of extras data.
- Return type
observations(Union[numpy.ndarray, torch.Tensor])
-
property
unwrapped
¶ Completely unwrap this env.
- Returns
The base non-wrapped gym.Env instance
- Return type
gym.Env
-
-
class
VecEnvMT
(headless: bool)¶ This class provides a base interface for connecting RL policies with task implementations in a multi-threaded fashion. RL policies using this class will run on a different thread than the thread simulation runs on. This can be useful for interacting with the UI before, during, and after running RL policies. Data sharing between threads happen through message passing on multi-threaded queues.
-
action_space
= None¶
-
clear_queues
()¶ Clears all queues.
-
close
() → None¶ Closes simulation.
-
get_actions
(block=True)¶ Retrieves actions from policy by waiting for actions to be sent to the queue from the RL thread.
- Parameters
block (Optional[bool]) – Whether to block thread when waiting for data.
- Returns
actions buffer retrieved from queue.
- Return type
actions (Union[np.ndarray, torch.Tensor, None])
-
get_data
(block=True)¶ Retrieves data from task by waiting for data dictionary to be sent to the queue from the simulation thread.
- Parameters
block (Optional[bool]) – Whether to block thread when waiting for data.
- Returns
data dictionary retrieved from queue.
- Return type
actions (Union[np.ndarray, torch.Tensor, None])
-
initialize
(action_queue, data_queue, timeout=30)¶ Initializes queues for sharing data across threads.
- Parameters
action_queue (queue.Queue) – Queue for passing actions from policy to task.
data_queue (queue.Queue) – Queue for passing data from task to policy.
timeout (Optional[int]) – Seconds to wait for data when queue is empty. An exception will be thrown when the timeout limit is reached. Defaults to 30 seconds.
-
metadata
= {'render.modes': []}¶
-
property
num_envs
¶ Retrieves number of environments.
- Returns
Number of environments.
- Return type
num_envs(int)
-
observation_space
= None¶
-
render
(mode='human') → None¶ Step the renderer.
- Parameters
mode (str) – Select mode of rendering based on OpenAI environments.
-
reset
()¶ Resets the task and updates observations.
-
reward_range
= (-inf, inf)¶
-
run
(trainer)¶ Main loop for controlling simulation and task stepping. This method is responsible for starting simulation, stepping task and simulation, collecting buffers from task, sending data to policy, and retrieving actions from policy. It also deals with the case when the policy terminates on completion and continues the simulation thread so that UI does not get affected.
- Parameters
trainer (TrainerMT) – A Trainer object that implements APIs for starting and stopping RL thread.
-
seed
(seed=- 1)¶ Sets a seed. Pass in -1 for a random seed.
- Parameters
seed (int) – Seed to set. Defaults to -1.
- Returns
Seed that was set.
- Return type
seed (int)
-
send_actions
(actions, block=True)¶ Sends actions from RL thread to simulation thread by adding actions to queue.
- Parameters
actions (Union[np.ndarray, torch.Tensor]) – actions buffer to be added to queue.
block (Optional[bool]) – Whether to block thread when writing to queue.
-
send_data
(data, block=True)¶ Sends data from task thread to RL thread by adding data to queue.
- Parameters
data (dict) – Dictionary containing task data.
block (Optional[bool]) – Whether to block thread when writing to queue.
-
set_task
(task, backend='numpy', sim_params=None, init_sim=True) → None¶ - Creates a World object and adds Task to World.
Initializes and registers task to the environment interface. Triggers task start-up.
- Parameters
task (RLTask) – The task to register to the env.
backend (str) – Backend to use for task. Can be “numpy” or “torch”. Defaults to “numpy”.
sim_params (dict) – Simulation parameters for physics settings. Defaults to None.
init_sim (Optional[bool]) – Automatically starts simulation. Defaults to True.
-
spec
= None¶
-
step
(actions)¶ - Basic implementation for stepping simulation.
Can be overriden by inherited Env classes to satisfy requirements of specific RL libraries. This method passes actions to task for processing, steps simulation, and computes observations, rewards, and resets.
- Parameters
actions (Union[numpy.ndarray, torch.Tensor]) – Actions buffer from policy.
- Returns
Buffer of observation data. rewards(Union[numpy.ndarray, torch.Tensor]): Buffer of rewards data. dones(Union[numpy.ndarray, torch.Tensor]): Buffer of resets/dones data. info(dict): Dictionary of extras data.
- Return type
observations(Union[numpy.ndarray, torch.Tensor])
-
property
unwrapped
¶ Completely unwrap this env.
- Returns
The base non-wrapped gym.Env instance
- Return type
gym.Env
-