.. _isaac_sim_python_environment:

===================================
Python Environment
===================================

This document will cover:

- Details about how running standalone Python scripts works.
- A short list of interesting/useful standalone Python scripts to try.
- How Jupyter Notebook support works.
- How Visual Studio Code support works.

Details: How ``python.sh`` works
#################################

.. note::
    - On windows use python.bat instead of python.sh
    - The details of how python.sh works below are similar to how python.bat works

This script first defines the location of the apps folder so the contained .kit files can be located at runtime.

.. code-block:: bash

    # Get path to the script
    SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
    # The apps directory is relative to where the script lives
    export EXP_PATH=$SCRIPT_DIR/apps

Then we source the |isaac-sim| Python environment so all extension interfaces can be loaded correctly.

.. code-block:: bash

    source ${SCRIPT_DIR}/setup_python_env.sh

The setup_python_env.sh script update/defined the following environment variables:

- ISAAC_PATH: Path to the main isaac folder
- PYTHONPATH: Paths to each extensions Python interfaces
- LD_LIBRARY_PATH: Paths to binary interfaces required to find symbols at runtime
- CARB_APP_PATH: path to the core |omni| kit executable

Finally, we execute the Python interpreter that is packaged with |omni|:

.. code-block:: bash

    python_exe=${PYTHONEXE:-"${SCRIPT_DIR}/kit/python/bin/python3"}
    ...
    $python_exe $@

.. _isaac_sim_python_simulationapp:

SimulationApp
#################################

The |simulation_app| provides convenience functions to manage the lifetime of a |isaac-sim| application.

    .. |simulation_app| raw:: html

        <a href="../../py/isaacsim/source/extensions/omni.isaac.kit/docs/index.html">SimulationApp Class</a>


Usage Example:
--------------

The following code provides a usage example for how SimulationApp can be used to create an app, step forward in time and then exit.

.. note::
    Any |omni| level imports **must** occur after the class is instantiated.
    Because APIs are provided by the extension/runtime plugin system, it must be loaded before they will be available to import.

.. important::
    When running headless:

    - Set ``"headless": True`` in the config when initializing ``SimulationApp``
    - Any calls that create/open a matplotlib window need to be commented out

.. code-block:: python

    import os
    from omni.isaac.kit import SimulationApp

    # Simple example showing how to start and stop the helper
    simulation_app = SimulationApp({"headless": True})

    ### Perform any omniverse imports here after the helper loads ###

    simulation_app.update()  # Render a single frame
    simulation_app.close()  # Cleanup application


Details: How SimulationApp works
---------------------------------

The sample code shown below can be executed via:

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.kit.app/app_framework.py

There are three steps to loading an |omni| application in Python. The first is to get the carbonite framework:

.. code-block:: python

    import carb
    self.framework = carb.get_framework()
    self.framework.load_plugins(
            loaded_file_wildcards=["omni.kit.app.plugin"],
            search_paths=[os.path.abspath(f'{os.environ["CARB_APP_PATH"]}/plugins')],
        )

Here ``CARB_APP_PATH`` was previously defined by running using `python.sh`.


We then set up the startup arguments for the application:

.. code-block:: python

    # Inject experience config:
    sys.argv.insert(1, f'{os.environ["EXP_PATH"]}/omni.isaac.sim.python.kit')

    # Add paths to extensions
    sys.argv.append(f"--ext-folder")
    sys.argv.append(f'{os.path.abspath(os.environ["ISAAC_PATH"])}/exts')
    # Run headless
    sys.argv.append("--no-window")

    # Set some settings
    sys.argv.append("--/app/asyncRendering=False")

And then start the application:

.. code-block:: python

    self.app.startup("Isaac-Sim", app_root, sys.argv)

Shutting down a running application is done by calling shutdown and then unloading the framework:

.. code-block:: python

    self.app.shutdown()
    self.framework.unload_all_plugins()


.. _isaac_sim_python_additional_extensions:

Enabling additional extensions
------------------------------

There are two methods for adding additional extensions:

#. Add them to the ``[dependencies]`` section of ``apps/omni.isaac.sim.python.kit``:

    .. code-block:: python

        [dependencies]
        # Enable the layers and stage windows in the UI
        "omni.kit.window.stage" = {}
        "omni.kit.widget.layers" = {}

#.  Enable them from Python directly:

    .. code-block:: python

        # Start the omniverse application
        simulation_app = SimulationApp(CONFIG)
        from omni.isaac.core.utils.extensions import enable_extension
        # Enable the layers and stage windows in the UI
        enable_extension("omni.kit.widget.stage")
        enable_extension("omni.kit.widget.layers")
        simulation_app.update()


Standalone Example Scripts:
#################################

Time Stepping
-------------

This sample shows how to start an |kit| Python app and then create callbacks which get called each rendering frame and each physics timestep. It also shows the different ways to step physics and  rendering.

The sample can be executed by running the following:

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.isaac.core/time_stepping.py

Load USD Stage
------------------

This sample demonstrates how to load a USD stage and start simulating it.

The sample can be executed by running the following, specify ``usd_path`` to a location on your nucleus server:

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.isaac.kit/load_stage.py --usd_path /Isaac/Environments/Simple_Room/simple_room.usd


URDF Import
-----------

This sample demonstrates how to use the |urdf_api|, configure its physics and then simulate it for a fixed number of frames.

.. |urdf_api| raw:: html

    <a href="../../py/isaacsim/source/extensions/omni.isaac.urdf/docs/index.html">URDF Python API</a>


The sample can be executed by running the following:

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.isaac.urdf/urdf_import.py


Change Resolution
------------------

This sample demonstrates how to change the resolution of the viewport at runtime.

The sample can be executed by running the following:

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.isaac.kit/change_resolution.py

.. _isaac_sim_python_convert_assets:

Convert Assets to USD
---------------------

This sample demonstrates how to batch convert OBJ/STL/FBX assets to USD.

To execute it with sample data, run the following:

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.kit.asset_converter/asset_usd_converter.py --folders standalone_examples/data/cube standalone_examples/data/torus

The input folders containing OBJ/STL/FBX assets are specified as argument
and it will output in terminal the path to converted USD files.

.. code-block:: bash

    Converting folder standalone_examples/data/cube...
    ---Added standalone_examples/data/cube_converted/cube_fbx.usd

    Converting folder standalone_examples/data/torus...
    ---Added standalone_examples/data/torus_converted/torus_stl.usd

This sample leverages Python APIs from the  `Asset Importer <../../prod_extensions/prod_extensions/ext_asset-converter.html>`_ extension.

The details about the import options can be found `here <../../prod_extensions/prod_extensions/ext_asset-importer.html#programming-guide>`_.


Livestream
------------------

This sample demonstrates how to enable livestreaming when running in native Python.

See the :ref:`livestream documentation<isaac_sim_setup_kit_remote>` for more information on running the client.

The sample supports the Omniverse Streaming Client backend.

.. code-block:: bash

    ./python.sh standalone_examples/api/omni.isaac.kit/livestream.py

.. note::
    - Running livestream.py will not have all of the default isaac sim extensions enabled. See :ref:`enabling additional extensions <isaac_sim_python_additional_extensions>` for more information.


.. _isaac_sim_python_jupyter_notebook:

Jupyter Notebooks
#################################

In order for |isaac-sim_short| to work inside of a Jupyter Notebook we provide a custom Jupyter kernel that is installed when you run:

.. code-block:: bash

    ./jupyter_notebook.sh

.. note::
    - Jupyter notebook support is linux only currently

The kernel.json itself is fairly simple:

.. code-block:: python

    {
        "argv": ["AUTOMATICALLY_REPLACED", "-m", "ipykernel_launcher", "-f", "{connection_file}"],
        "display_name": "Isaac Sim Python 3",
        "language": "python",
        "env": {
            "ISAAC_JUPYTER_KERNEL": "1"
        },
        "metadata": {
            "debugger": true
        }
    }

The important part is that ``AUTOMATICALLY_REPLACED`` gets replaced by ``jupyter_notebook.sh`` with the absolute path to the python executable that is located in the kit/python directory at runtime. Once the variable is replaced, the kernel is installed and the notebook is started. There is an extra variable ``ISAAC_JUPYTER_KERNEL`` that is used inside of |isaac-sim_short| to setup for notebook usage properly.

Because notebooks require asyncio support, and |isaac-sim_short| itself uses asyncio internally, we automatically execute the following two lines when loading the ``omni.isaac.kit`` extension which provides the ``SimulationApp`` class:

.. code-block:: python

    import nest_asyncio
    nest_asyncio.apply()

This ensures that asyncio calls can be nested inside of the Jupyter Notebook properly.


.. _isaac_sim_python_vscode:

Visual Studio Code (VSCode) Support
####################################

The |isaac-sim_short| package provides a ``.vscode`` workspace with a pre-configured environment that provides the following three files:

.. code-block:: bash

    .vscode/launch.json
    .vscode/settings.json
    .vscode/tasks.json


launch.json
-----------

This file provides three different configurations that can be executed using the ``Run & Debug`` section in VSCode.

- **Python: Current File**: Debug the currently open standalone Python file, should not be used with extension examples/code.
- **Python: Attach**: Attach to a running |isaac-sim_short| application for debugging purposes, most useful when running an interactive GUI application. See :ref:`isaac_sim_app_tutorial_advanced_attach_debugger` for usage information.
- **(Linux) isaac-sim** Run the main |isaac-sim_short| application with an attached debugger.

settings.json
--------------

This file sets the default Python executable that comes with |isaac-sim_short|:

.. code-block:: python

    "python.pythonPath": "${workspaceFolder}/kit/python/bin/python3",

As well as a configuration for ``"python.analysis.extraPaths"`` which by default includes all of the extensions that are provided by default. You can add additional paths here if needed.

tasks.json
--------------

This is a helper file that contains a task used to automatically setup the Python environment when using the ``Python: Current File`` option in ``Run & Debug``.

.. code-block:: python

    "tasks": [
            {
                "label": "setup_python_env",
                "type": "shell",
                "linux": {
                    "command": "source ${workspaceFolder}/setup_python_env.sh && printenv >${workspaceFolder}/.vscode/.standalone_examples.env"
                }
            }
        ]

Once executed, the task generates the ``.standalone_examples.env`` file used by VSCode to launch the Python debug process.