9. ShapeNet Importer Tutorial

9.1. Learning Objectives

This tutorial demonstrates how to import the ShapeNet V2 3D model database into Omniverse Isaac Sim as USD files. After this tutorial, you will know how to create a local ShapeNet repository and import assets from the ShapeNet database into your experiments and simulations.

5-10 Minute Tutorial

9.2. Getting Started

Prerequisites

  • Please review the Essential Tutorial series, especially Isaac Sim Workflows, prior to beginning this tutorial.

  • This extension requires Google Chrome to function fully since it uses the selenium python package for login automation.

  • This extension requires a valid ShapeNet login to download and import ShapeNet models into Omniverse. Register a ShapeNet account at shapenet.org prior to using this extension.

Note

If the ShapeNet database is already downloaded, or you want to use a more permanent location for the local model storage, you can set the environment variable SHAPENET_LOCAL_DIR, for example: export SHAPENET_LOCAL_DIR=$HOME/shapenet

Creating the Database

Let’s begin by creating the Shapenet Index Database. This procedure will only happen one time if you have SHAPENET_LOCAL_DIR set or after each install of Omniverse Isaac Sim.

If you already have per opening the ShapeNet Extension and signing in to your ShapeNet account:

  1. Go to the top menu bar and click Isaac Utils > ShapeNet Importer.

  2. Press the Create Shapenet Database Index File button to enter your ShapeNet account info, or leave it blank if you have a local copy of shapenet already. Click Sign In.

  3. The extension launches your browser and automatically logs into shapenet.org.

  4. If a browser window opens, wait for the browser to close before proceeding.

Upon successful login or finding of local shapenet copy, the ShapeNet Importer Extension uses the CSV files from the ShapeNet V1 database. These CSV files are used to build a Shapenet Index Database, which is just a list of valid shapnetId mapped to the class names associated with them. You will know the Shapenet Index Database was created successfully if you see the Add A Model button instead of the Create Shapenet Database Index File button.

The ShapeNet Importer Extension is now ready for use!

9.3. Using the ShapeNet Importer Extension

There are three ways to add models using the ShapeNet Importer extension:

  1. Use the Omniverse Isaac Sim User Interface

  2. Use the Python API

  3. Use a Jupyter Notebook (via HTTPS Requests).

This section details the use and benefits of each method. Each of the methods will check the Omniverse server for the shapenet model converted to USD first and add it to the scene. This allows you to make changes to the USD models in Omniverse to improve them. If the model is not in Omniverse, the ShapeNet Importer extension will go through the following steps automatically:

  1. Download the modal to local storage from shapenet.org without change.

  2. Convert it to USD and store it on Omniverse.

  3. Add it to the Scene.

9.3.1. Add a Model via User Interface

The ShapeNet Importer UI is under the Isaac Utils menu in Omniverse Isaac Sim, and can also be accessed with the Geometry radio button on the Isaac Sim Utilities selector. The UI is recommended when adding a small number of models.

To add a model:

  1. Use the synsetId and modelId to set exact ids, or set the the string to “random”. If the synsetId is set to an exact id, the modelId may still be “random”, and that will produce a random model from within the set synset category.

  2. Set the transform parameters to place the model in the scene; default scale and rotation from the scene are added automatically.

  3. Check the Automatically add physics checkbox if you want physics, and the Use convex decomposition checkbox for convex decomposition of complex collision objects.

  4. Click the Add A Model button at the top of the ShapeNet Loader tab.

9.3.2. Add a Model via Python API

The second method to use the ShapeNet Importer extension is by using its Python API.

Users can get to the ShapeNet Importer extension folder by clicking on the Folder button in the Extension Manager, or by looking in exts/omni.isaac.shapenet/omni/isaac/shapenet/. The main function, addShapePrim, is in the ShapeNet Importer extension folder’s shape.py file.

  • omniverseServer is the name of the server that stores the shapes once converted to USD.

  • synsetId and modelId can be specific or not set for random values

  • The transform is set with pos for the translation, rot for the rotation as a Gf.Rotation in axis angle notation, and scale

  • Set auto_add_physics and / or use_convex_decomp to True for physics and decompostion of complex collision objects

  • do_not_place is a specialized parameter for if you want to use the plugin to mass convert and download models but do not want to place them in the scene.

For an example of how to use addShapePrim, refer to how settings.py uses that function.

9.3.3. Add Shapes via Jupyter Notebook

When the Isaac Shapenet extension is added to Isaac Sim, it starts an http server listening for POST commands. This allows the user to interact with the ShapeNet Importer extension via any web-based interface that can send HTTPS based POST commands, like python or a Jupyter notebook. The default address is localhost:7011. Omniverse Isaac Sim provides a jupyter notebook in the ShapeNet Importer extension folder at exts/omni.isaac.shapenet/omni/isaac/shapenet/ShapeNet Python Example.ipynb.

The Jupyter Notebook example relies on HTTPS Requests to connect to Omniverse Isaac Sim when the ShapeNet Loader extension is running.

Run the following command to open the notebook and see how the http command is formatted there.

./jupyter_notebook.sh exts/omni.isaac.shapenet/omni/isaac/shapenet/ShapeNet Python Example.ipynb

The comm_kit library is located in the ShapeNet Importer extension folder mentioned in the previous Add a Model via Python API sub section. Here is an example of how to use it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# import and place one model
from comm_kit import *

comm = setup()
command = {}

command['omniverseServer'] = 'localhost'
command['synsetId'] = '02691156'
command['modelId'] = 'dd9ece07d4bc696c2bafe808edd44356'
# The following command components are optional
command['pos'] = (10.0, 11.0, 120.0)
command['rot'] = ((0.0,0.0,1.0), 90.0)
command['scale'] = 1.1
command['auto_add_physics'] = 0 # 1 if you want to add physics to the shapenet object.
command['use_convex_decomp'] = 0 # 1 if you want to break complex shapes into pieces, for more accurate collisions.
command['do_not_place'] = 0 # 1 if you want to only convert and skip placing the shape in kit's current stage.
command['g_omni_shape_loc'] = "/Projects/shapenet" # This is where the converted files are placed, if they don't
                                                        # already exist on the server.  Or pulled from if they do.
command['g_local_shape_loc'] = "F:/shapenet/ShapeNetCorev2" # This is the local hard drive location used where the obj
                                                            # obj files that are converted are kept.  This should just
                                                            # be a local copy of the original shapenet core 2
command['g_root_usd_namespace_path'] = "/shapenet" # this is the prim path in the scene used to store the models.


print(comm.post_command(command))

9.4. Summary

This tutorial covered the following topics:

  1. Connecting Omniverse Isaac Sim to ShapeNet.

  2. Adding Models from ShapeNet into a scene via User Interface.

  3. Adding Models from ShapeNet into a scene via Python API.

  4. Adding Models from ShapeNet into a scene via Jupyter Notebook and HTTPS Requests.

9.4.1. Further Learning

For a more in-depth look into the concepts covered in this tutorial, refer to the following reference materials: