Generative AI Reference Architecture

With NVIDIA NIM microservices, teams can experiment with backgrounds, visuals, and 3D assets to create visual content using Generative AI. This Reference Architecture diagram illustrates an example of a development workflow for this purpose.

Workflow Diagram

In this sample workflow diagram, the user leverages a web-based front end that sends prompts to search for, query, and generate brand-approved assets, and visualize an environment.

This workflow is powered by an Omniverse Kit Application, streamed through the Omniverse App Streaming API, which generates real-time updates and photorealistic rendering of the USD scene. User prompts are processed through the Kit Application using USD NIM microservices like USD Code, USD Search, and NVIDIA Edify to create assets using Python code, fetch assets from a USD Database, generate 3D assets using text prompts, and create backgrounds.

Diagram of the parts of the Generative AI Reference Architecture diagram

Exploring the Sample Workflow

Web-Based Front End

This is the part of the application that users interact with directly. It encompasses everything users visually experience on their screens, and how they digest the product.

NVIDIA Omniverse™ Cloud

NVIDIA Omniverse Cloud is a platform-as-a-service designed for developers and enterprises. It provides a full-stack cloud environment to create, develop, deploy, and manage industrial metaverse applications. Omniverse Cloud is fully managed by NVIDIA.

Agent

This interacts with your Kit-based application and data. The agent puts your USD data in the database and pulls from it when necessary for a query or for addition to your scene.

Brand-Approved Assets

Brand-approved assets are design elements, content, or materials that align with a company’s brand guidelines and have received official approval for use. These assets ensure consistent branding across different channels, including marketing materials, websites, and product packaging.

USD Database

This database is the repository for all USD assets for users – meshes, materials, textures, etc. It includes both assets created by traditional 3D workflows as well as assets created by Gen AI. This is the source for USD Search queries.

USD Search NIM

USD Search is an NVIDIA NIM microservice enabling developers, creators, and workflow specialists to take advantage of AI-powered search for OpenUSD data, 3D models, images, and assets using text or image-based inputs.

By combining a collection of cloud-native microservices, USD Search is capable of searching and indexing 3D asset databases, searching in-scene, and performing spatial searches, without requiring any manual tagging of assets.

USD Code NIM

USD Code is an NVIDIA NIM microservice and Large Language Model (LLM) capable of answering OpenUSD knowledge questions and generating Python USD code in response to text prompts, packaged as a NIM microservice.

USD Code enables developers to learn and develop with OpenUSD more productively in existing 3D development workflows.

3D Asset Generator

Leveraging the Edify NIM to train a 3D Asset Generator enables rapid deployment of a model that can create 3D assets from text or image prompt. APIs powered by NVIDIA Edify enable developers to integrate 3D asset generators into their workflows.

Edify.Getty

Edify.Getty is a model built by Getty Images on NVIDIA Edify. It creates images with realistic aesthetics, ideal for workflows like editing scenes or creating backgrounds. It receives a prompt condition and outputs a background image based on this condition.