Using Python Pip Packages#

In some cases during application development, you may need to include a Python package that is not part of the default Python environment in the Kit. To do this, you can use the pip.toml file located at .\kit-app-template\tools\deps\pip.toml to specify which additional Python packages should be included during the build process. This informs the build system about the required dependencies.

Process Overview:

  1. Add pip packages to pip.toml

  2. Create a pip_bundle extension to manage the additional Python dependencies

  3. Add the pip_bundle extension into your application .kit file as a dependency

Prerequisites#

The following prerequisites are required to complete this tutorial:

Tutorial#

In this tutorial, we will walk through how to add third-party Python libraries to your application via an extension using pip. The recommended approach is to add the additional dependencies at build time, and this guide will walk you through that process. With the approach taken in this tutorial, the additional dependencies will be accessible to all other extensions in your application.

1. Setup#

We first need to create a basic application using the kit-app-template tooling. This step ensures that we have a foundation to which dependencies can be added.

Create a New Editor Application

From a new project or a fresh clone kit-app-template repository, create a new editor application using the template new tool. For this example, we will use Kit Base Editor.

Linux:

./repo.sh template new

Windows:

.\repo.bat template new

Follow the prompt instructions, choosing:

  • ? Select what you want to create with arrow keys ↑↓: Application

  • ? Select desired template with arrow keys ↑↓: Kit Base Editor

  • ? Enter name of application .kit file [name-spaced, lowercase, alphanumeric]: tutorial.editor.pip

  • ? Enter application_display_name: Tutorial Pip Prebundle Application

  • ? Enter version: 0.1.0

2. Create an Extension for Pip Packages#

Next, we will create an extension to manage all the Python dependencies. This extension will ensure that the required packages are included and available for use in our application.

Create a New Extension

Create a Basic Python Extension using the template new tool.

Linux:

./repo.sh template new

Windows:

.\repo.bat template new

Follow the prompt instructions, choosing:

  • ? Select what you want to create with arrow keys ↑↓: Extension

  • ? Select desired template with arrow keys ↑↓: Basic Python Extension

  • ? Enter name of extension .py file [name-spaced, lowercase, alphanumeric]: tutorial.pip_prebundle_extension

  • ? Enter extension_display_name: Tutorial Pip Prebundle Extension

  • ? Enter version: 0.1.0

3. Add Python Dependencies via an Extension#

Once the application and extension are set up, we will add the chosen Python dependencies via the extension. We will then add the extension to the main application as a dependency. This will ensure the new Python dependencies are made available for use within the application.

3a - Update the pip.toml File#

In your project, navigate to the .\kit-app-template\tools\deps\pip.toml file. This is where you’ll list the Python packages you want to add to your project. You will also specify the folder where the packages will be installed.

Required pip.toml changes:

To add a new package, you can either uncomment the existing [[dependency]] section or add a new code snippet. For this example, we will use the openpyxl package, but feel free to replace it with any other required dependency.

[[dependency]]
python = "../../_build/target-deps/python"
packages = [
   "openpyxl==3.1.5",
]
target = "../../_build/target-deps/pip_prebundle"

  • python : This is the path to the Python interpreter that will be used for installation. It can be an absolute path or relative to the location of this configuration file.

  • packages : List the Pip packages you want to install, along with their specific versions to avoid compatibility issues. For example, with Kit 106.4, Python 3.10 is used. Some packages, like openpyxl==3.1.5, may not work with other Python versions.

  • target : This specifies the path where the packages will be installed. The installed packages will later be linked from your extension.

Verify the pip.toml file changes

After you have updated the pip.toml file, you can verify the requested packages are added by building the project.

Linux:

./repo.sh build

Windows:

.\repo.bat build

Now, locate to the .\_build\target-deps\pip_prebundle. If you see openpyxl and openpyxl-3.1.5.dist-info folders, you have successfully added Pip packages during the build.

3b - Point the Extension to Installed Dependencies#

After making the required changes to the pip.toml file, we now need to let the build system know where the new dependencies are located for use in pip_prebundle_extension.

Within.\kit-app-template\source\extensions\tutorial.pip_prebundle_extension\premake5.lua, add a new prebuild link:

-- Link only those files and folders into the extension target directory
repo_build.prebuild_link {
    { "data", ext.target_dir.."/data" },
    { "docs", ext.target_dir.."/docs" },
    { "tutorial", ext.target_dir.."/tutorial" },
    { "%{root}/_build/target-deps/pip_prebundle", ext.target_dir.."/pip_prebundle" }, -- added line
}

3c - Update extension.toml#

To make sure that the Python interpreter can find the installed packages, you need to add the pip_prebundle path in the [[python.module]] section of the extension.toml file.

Edit the extension.toml File

In your .\kit-app-template\source\extensions\tutorial.pip_prebundle_extension\config\extension.tomlfile, please replace [[python.module]] section with this:

[[python.module]]
path = "pip_prebundle"

note: remove the name=tutorial.pip_prebundle_extension

This configuration tells Python to include the pip_prebundle folder in the module search path, ensuring that your extension can import the installed packages.

3d - Adding the Extension to the Application#

Once the pip_prebundle_extension is properly configured, we can add it to the application as a dependency and verify that the new packages are available.

Add tutorial.pip_prebundle_extension to the application .kit file

Within the .kit file for the application (\source\apps\tutorial.editor.pip.kit), add the new extension to the [dependencies] section:

[dependencies]
"tutorial.pip_prebundle_extension" = {}

4. Build, Launch, and Test#

Finally, we can build, launch, and test it.

4a - Build#

Use this command to build

Linux:

./repo.sh build

Windows:

.\repo.bat build

4b - Launch#

After the build is complete, we can launch. For the sake of this simple test, we will use Script Editor in the developer bundle to verify our changes. The developer bundle (omni.kit.developer.bundle) should be enabled in the application .kit file by default using the standard application templates. If the developer bundle is not in the application .kit file it can be activated at launch by adding the -d flag to the launch command.

Linux:

./repo.sh launch -d

Windows:

.\repo.bat launch -d

The launch tool will prompt you to select an application .kit file to launch. Select tutorial.editor.pip.kit

Note that the first time startup of a rendering application may take 5-8 minutes as the shader cache is built.

4c - Test#

Once the application launches, go to the Developer menu and click Script Editor. You should be able to use the dependencies you specified in the Script Editor.

Using Python Pip Packages Successfully Implemented

If successful, you will be able to import the openpyxl package.

5. Adding Multiple Dependencies#

This is an optional section to demonstrate a use case where multiple additional Python dependencies are required. It is recommended that Python dependencies that will be used together are managed by the same pip_prebundle_extension.

5a - Update the pip.toml File#

[[dependency]]
python = "../../_build/target-deps/python"
packages = [
   "openpyxl==3.1.5",
   "requests==2.32.3",
]
target = "../../_build/target-deps/pip_prebundle"

5b - Build#

We cannot build as we did before because if the folder already exists, it will not be updated. Since the pip_bundle folder already exists, no changes would be made. To make sure the repository is cleaned before building, we need to add x after the build command.

Linux:

./repo.sh build -x

Windows:

.\repo.bat build -x

5c - Launch#

After the build is complete, we can launch.

Linux:

./repo.sh launch

Windows:

.\repo.bat launch

The launch tool will prompt you to select an application .kit file to launch. Select tutorial.editor.pip.kit

5d - Test#

Once the application launches, go to the Developer menu and click Script Editor. You should be able to use the dependencies you specified in the Script Editor.

Using Python Pip Packages Successfully Implemented Part 2

If successful, you will be able to import openpyxl and requests packages.

6. Cleanup#

Before continuing to the next section, it is recommended to clean up the applications, extensions, and any other repository changes that were created during this exploration.

Either revert all local changes or remove your existing repository/project entirely. Then, recreate it by cloning the repository again or by running the new project script from the Kit SDK.

Typical cleanup steps include:

  • Revert changes to top level premake5.lua file

  • Revert changes to repo.toml

  • Delete the source/ directory

  • Delete the _build/ directory, or run ./repo.sh build -c or .\repo.bat build -c

Additional Considerations#

Dependency Management and Compatibility#

When adding Python packages to your application through extensions, it’s important to consider the dependency management implications. When a Python package is installed via an extension and added as an application dependency (as done in this tutorial), all other extensions in the same application gain access to it. However, if an extension that relies on these shared packages is moved to another application without them, it may fail due to missing dependencies.

To ensure smooth functionality, consider these options:

  • Add the pip_bundle extension to the new application to make the required packages available to all extensions.

  • Set an extension dependency (in the extension.toml file) on the package-managing extension. This approach will ensure that all necessary Python packages are accessible to the extension regardless of where the extension is used.

Important Note: In both cases, additions to the pip.toml file are required as shown in the tutorial above.

Python Version Compatibility#

When selecting specific versions of Python dependencies, it is important to ensure that the selected version is compatible with the version of Python being used by Kit SDK. In the case Kit 106.4, this is Python 3.10.

Additional Resources#

On this page