Substance 3D Painter

A wide view of headphones painted in Painter and viewed in Omniverse Create

Overview

Adobe Substance 3D Painter is a world leading solution for 3D texturing. It’s a go-to 3D texturing app for creative professionals with hundreds of smart materials at your disposal and a powerful brush engine that allows you to texture any 3D model in minutes.

The Substance 3D Painter Connector works in tandem with the Create Painter Live Link extension to provide real time visualization of painted meshes in Omniverse Create. The Painter Connector can:

  • Open a USD of the active Painter mesh in Create

  • Duplicate the Painter environment HDR in Create as a Dome Light

  • Live-update texture maps and materials in Create

  • Import a USD mesh that’s currently open in Create from Omniverse

    • If the USD is using OmniPBR then the Albedo, Normal, and Roughness textures are imported into the project in Painter

Installation

There are two components to install, the Painter Connector and the Create Painter Link extension.

Painter Connector

The Painter Connector is available from the Omniverse Launcher. It requires that you have Substance 3D Painter installed.

The Substance 3D Painter Connector is available from the Omniverse Launcher

Settings and Actions

Painter Connector

The Omniverse Connector is a plugin that automatically exports texture maps whenever a user paints or changes any properties in the texture set. There is a settings and actions panel that allows the user to configure the Connector, export, and import meshes.

Sometimes the panel is hidden or mostly minimized after installation in the bottom right corner.

If the Settings panel is completely hidden, open it by selecting Window > Views > Omniverse-Connector:

Making the settings pane in Painter visible

If it’s stuck in the bottom right corner drag the title bar up to expose all of the parameters:

Exposing the settings pane in Painter

The Settings pane:

The extension settings pane in Painter

Setting

Effect

Directory

This folder is where the plugin initially stores Textures and FBX meshes so Create can import them. This folder is not related to the Create extension’s “Nucleus Folder” setting.

Texture Resolution

The size of exported textures. When changed all texture sets are immediately exported and a message is sent to the Create Painter Link extension

Texture Preset

Defines what textures are output and how they’re formatted. For now, only two are supported:
PBR Metallic Roughness
Unreal Engine 4 (Packed)

Texture Bit Depth

The bit depth of the exported textures. Note that not all bit depths are supported by some formats

Texture Format

The texture file format (PNG, TGA, etc.)

Kit/Create Link Port

The port on which the Create extension is listening for texture update, mesh import, and mesh export notifications. The default is 8111, but if you’re running more than one instance of Create or you’re using View/Code/etc. this will change.

Tick Interval (ms)

The interval between texture exports in milliseconds

Action

Effect

Export Mesh to Omniverse

This button will open the current Painter mesh in Create on Omniverse

Export Environment Map

This button will create a Dome Light in Create using Painter’s environment map

Make Environment Map Visible (option)

If set, the environment map will be visible in Create

Import Mesh from Omniverse

Import the current USD prop that’s open in Create into a new Painter project

Split Materials by UDIM (option) 102.1

If the USD uses UDIMs then check the box so that the new Painter project supports UV Tiling

Enable/Disable Live-Link 102.1

This is effectively a pause button that will keep the plugin from exporting any maps or communicating with Create while disabled

102.1 The “Idle/Exporting” indicator shows the user when the plugin is busy exporting textures (larger textures take time to export and the delay can be noticeable).

Note

The two available Texture Presets (PBR Metallic Roughness and Unreal Engine 4 (Packed)) are currently the only two presets that will work with the Painter Link extension in Create. The specific format of the map names - $mesh_$textureSet_BaseColor(_$colorSpace)(.$udim) - is linked to the constructing of the OmniPBR texture within Create. In the future we intend to support more custom presets and possibly even custom presets so creators can format things as they require.

Texture Set Settings

Only the channels specified in the Channels list are initialized in the OmniPBR material that Omniverse Create updates. This can be used to make a material more efficient. For example, if there’s no opacity required, the Opacity channel can be deleted and that parameter will be unchecked in the generated OmniPBR material.

Show the extension window settings pane

Create Painter Live Link Extension

To bring up the Painter Live Link settings, go to Window > Substance 3D Painter Link

Show the extension window settings pane

This will show the Painter Live Link settings:

The Painter Link extension settings pane

Setting

Effect

Nucleus Folder

Exported meshes and textures from painter are created and copied to this folder
NOTE: for testing purposes, this can be changed to a local disk folder instead of a Nucleus folder

Upload to Nucleus

If checked, copy textures to the Nucleus folder with every update
Note, if this is not checked then the root layer edits will point to the rotating number texture folders within the Painter export folder

Update Materials

If checked, when textures are update materials within the stage will be updated with the correct parameters and maps according to the Texture Preset defined in the Painter Connector

Create Materials

If checked, materials will be created for any stage, whether it has a mesh in it or not

Autosave Stage

If checked, the stage will be saved with every texture update from Painter

Link Port

This shows the current port on which the Painter Link extension is listening. It can be configured in the Painter Connector settings.

102.1 The “Idle/Working” indicator shows the user when the extension is busy uploading textures (larger textures take time to upload and the delay can be noticeable).

Exporting Omniverse and Painting

The Connector and extension allow a user to export a Painter mesh into Omniverse for live painting.

  1. Open or create a project in Painter

  2. To link Painter with Create, find the Omniverse Connector dock widget, it might be in the bottom right corner. It might be better to dock it under the viewport so it’s slightly wider.

  3. In the Settings pane, ensure that the Omniverse Create Link Port matches the number shown in the Create 3D Painter Link window. This number can change if you have multiple instances of Create running or if you are running Omniverse Code instead of Create

    Ensure the ports match between the Connector and Extension
  4. Use the Painter Connector Settings Export Mesh to Omniverse button to send the current mesh and texture sets to Omniverse Create

  5. If you want to match the environment, you can create a matching dome light in Create by exporting the Environment Map using the Export Environment Map button

  6. Start painting

UV Tile (UDIM) Support

The Connector supports exporting Painter projects with the UV Tile workflow enabled. When a change is made to any individual tile all of the tiles are exported, so take note that every brush stroke can generate a lot of work exporting textures.

Importing a Mesh from Omniverse

With the Connector and Create Painter Link extension it is possible to open a USD prop stage in Create and import it into

Example video: (maybe)

Setting up the USD import

To properly setup a USD prop for import + editing into Painter, it’s good to know where things will be placed so the prop is modular and there aren’t texture references that point to crazy places. When Create copies textures to Nucleus, they always land in a folder inside the Nucleus Painter Folder location.

Ideally, prior to importing a USD into Painter, the file hierarchy should look like this:

Nucleus Painter Folder
└───Mesh_Name (USD prop stage name)
    │   Mesh_Name.usd
    │
    └───Textures
            OmniPBR referenced textures

The Import Process Explained

  1. Close any open projects in Painter (the Import button creates a new project)

  2. Open a USD prop in Create that you intend to import into Painter

    • NOTE: it’s advised that the USD stage contains a single mesh for painting, not a massive set with many props

    • NOTE: This USD prop/stage will be edited by the Painter Link:

      • Existing materials that are OmniPBR-based will be edited

      • It is suggested that if you want to keep the original OmniPBR-based materials to duplicate them before importing

      • Existing material that are not OmniPBR-based will be renamed and new OmniPBR-based materials will be created and assigned to the USD mesh

  3. Hit the Import Mesh from Omniverse button and this sequence starts:

    • The current USD stage is exported as FBX to the Painter Connector Export Folder

    • A new project is created in Painter based on the mesh

    • The Texture Sets in Painter are all sent back to Create and it will generate and assign materials to the mesh

    • NOTE: the current solution doesn’t preserve the USD’s materials and textures initially, so it will appear that all of the textures and materials are wiped and replaced with white.

  4. If the USD contained OmniPBR materials, they can be recreated because the textures should have been imported to the project

Recreating OmniPBR Materials in Painter

If the USD contained OmniPBR materials then the import process should have imported textures into the Painter project. They can be applied with these steps:

  1. Find the imported textures from the Project Assets (select the Texture filter to isolate them in the panel)

    Find the imported textures in the Painter project
  2. For each texture set, add a fill layer

  3. Drag the imported textures from the Project Textures onto the fill layer’s channels

    Drag the imported textures onto the fill layer's channels

At this point the original material exists in Painter for further editing.

Operating on a USD Mesh Within a Stage

The real power of being able to visualize an exported mesh from Painter comes when you include it in a composed stage in Create. The Connector will update textures for any material found in the stage that uses them. It is possible to export a mesh prop to Omniverse, setup the materials a bit, then actually drag that prop into another stage and continue editing it. The Painter Link extension should find the mesh and update it as painting continues in Painter.

The extension can edit textures used by a referenced USD prop within a stage

Existing Issues

Here are a collection of issues that can cause issues and we are working on:

  • In Adobe Substance 3D Painter the OMNIVERSE-CONNECTOR Settings pane is missing

    • If the Settings panel is completely hidden, open it by selecting Window > Views > Omniverse-Connector. If it’s stuck in the bottom right corner drag the title bar up to expose all of the parameters.

  • If the Omniverse Nucleus server you’re using is remote and texture copies are less than immediate, Create may freeze when the textures are uploading to Nucleus.

  • In Omniverse Create there are error messages like Failed to read texture file... or Couldn't process the file... in regard to texture files being updated while also read by the renderer. We are working to resolve the source of these messages.

  • If the Texture Set names are different than the material names in the FBX you may run into issues when exporting the mesh to Create. Painter doesn’t export the FBX with updated material names and there will be a mismatch. This can be fixed in one of two ways. You can either:

    • Rename the materials in Create to match the Texture Set names

      Rename materials and re-export the texture sets to link up the materials
    • Or use the Create Materials Painter Link option to generate new materials and assign them to the meshes yourself

      Rename materials and re-export the texture sets to link up the materials

Implementation Details

The Substance 3D Painter Connector and Create Painter Link Extension work in tandem to handle texture, material, mesh, and lighting synchronization between Painter and Create.

Mesh

Painter Export

  1. The Painter FBX is exported out of the Painter project and placed in the mesh’s folder under the Export Folder

  2. The Painter Connector notifies the Create Painter Link extension through and HTTP post message to convert the FBX mesh to USD. The Create extension performs the conversion and opens the stage in Create. The stage is located in the Nucleus Painter Folder under this hierarchy.

  3. The Painter Connector exports all of the textures for all of the texture sets to the mesh’s folder under the Export Folder

  4. The Painter Connector notifies the Create extension through HTTP post messages that the texture set maps are available and the Painter Link extension updates the OmniPBR materials in the USD stage with the Painter maps and parameters.

An overview of the interaction between the Painter Connector and Create Painter Link extension

Painter Import

  1. A USD prop stage, preferably located in the correct folder hierarchy under the Painter Link’s Nucleus Painter Folder is open in Create

  2. Painter requests to “Import a Mesh from Omniverse” with an HTTP post message to the Painter Link extension

  3. The Painter Link extension exports the current open stage as FBX. If the USD mesh had an OmniPBR material then the textures are exported in a folder next to the mesh (under materials/textures)

  4. When the export is complete Painter is notified and it creates a new project, setting the normal map convention to DirectX and adding any textures from materials/textures

Textures

Textures are exported using the Painter plugin API into time-stamped folders. The Create extension is notified of these new textures and updates the materials to reference them. The textures are copied to the Nucleus Folder if the Upload to Nucleus setting is checked.

Materials

When the Painter Connector exports textures it notifies the Painter Link extension to update materials with all of the correct meshes and texture preset channels

Note

Regarding the Omniverse Physically Based Rendering (OmniPBR) Material: Currently the only material that the Painter Link extension generates or updates in Create is OmniPBR. Also, the only material exported by Create’s FBX exporter is OmniPBR, so if there are OmniGlass, OmniSurface, or any other materials on the mesh the textures won’t be imported to Painter.

Environment Lighting

The Painter plugin API provides the current environment map path. When the environment map is exported a messages is sent to the Painter Link extension with the environment map path on local disk. A Dome Light is created in the stage under the /Preview/DomeLight prim path. Note that this is outside the defaultPrim so that there are no lighting collisions when referencing the USD prop in another stage. The environment map is also copied to the Nucleus Painter Folder if the Copy Texture to Nucleus setting is checked.

Session Layers and Local vs. Nucleus Textures

In an effort to make local Painter changes appear very quickly and still make the updates replicate through Omniverse, the Painter Link extension uses the Session Layer. The Session Layer is “scratch space” to configure, override, and experiment with the data contained in files backing the stage. An anonymous layer is a layer that cannot be saved to disk and will only reside in memory.

The Create Painter Link extension uses an anonymous sublayer within the Session Layer to store local texture paths. While working in the live session with Painter, these local texture paths are what a user sees and they update very quickly because all access is local. The Nucleus Paths to the textures are stored in the Root Layer so that the file-backed version of the stage contains the Nucleus-resident textures. If the stage is opened from another client on another computer the textures are referenced from Nucleus. They will update automatically, but not as quickly as the local textures update.

To show the Session Layer in the Layer tab, select Show Session Layer in the hamburger button (menu or collapsed menu button).

This is the Layer tab from Create. The Session Layer is on top because it is the strongest layer in the stage. The Painter Link extension creates the anon:####:PainterLinkLocalFileReferences sublayer and sets that to the authoring layer whenever textures are updated within the material.

The anonymous session layer with the material's local texture "overs" expanded

Release Notes

102.1 Point Release

The emphasis for this release is on stability, bug fixes, and some light workflow improvement with the UI.

Features

  • Added a button that will toggle enabling/disabling of all live-link features.

  • Added a checkbox that allows a mesh to be imported from Omniverse using UDIMs (OM-44584)

  • Added busy indicators to the Painter Connector and Create Extension for feedback during long-running operations

    Busy indicators in both the Connector and Extension

Bug Fixes

  • Painter now rotates the folder that textures are written to to make GPU access in Create safer. (OM-46027)

  • Installer fails and pops up a dialog when Painter not installed (OM-47068)

  • Installer uses proper exception handling on file/directory access (OM-47002)

  • Installer will overwrite files in the plugin folder (OM-49033)

  • The Texture Settings dialog will not export textures if none of the properties were changed from their original value.

  • Nucleus folder will not be created if not requested by having “Copy Textures To Nucleus” on (OM-48623)

  • Added support for OpenColorIO color management projects (OM-48645)

    • Note: detailed color space support is not working, but exported maps will link up correctly with the Omniverse PBR material

  • Doc link from the extension manager, now includes a link to the documentation (OM-47530)

  • Settings - version renamed to Substance 3D Painter Settings - version (OM-47531)

  • Re-exporting a mesh from painter twice no longer crashes Create (OM-46084)

Changes

  • Local only workflow is supported. Base layer now maps paths locally when copy textures to nucleus is off. (OM-48689)

  • The text on button that allows user to select a directory is replaced with a folder icon.

  • Added an icon that indicates if exporting is taking place (OM-49002).

  • Added a button for force exporting all textures (OM-48696).

  • Updated http post protocol from version 1.0 to 1.1 to send local textures relative directory with post.

101.1 Point Release

Features

  • Export the Painter mesh and texture sets as USD and MDL then open in Omniverse Create

  • Duplicate the Painter environment HDR in Create as a Dome Light

  • Live-update texture maps and materials in Create

  • Import a USD mesh that’s currently open in Create from Omniverse