.. _ogn_user_guide: OGN User Guide ============== Now that you are ready to write an OmniGraph node the first thing you must do is create a node definition. The .ogn format (short for **O** mni **G** raph **N** ode) is a JSON file that describes the node and its attributes. Links to relevant sections of the :ref:`ogn_reference_guide` are included throughout, where you can find the detailed syntax and semantics of all of the .ogn file elements. OmniGraph nodes are best written by creating a .ogn file with a text editor, with the core algorithm written in a companion C++ or Python file. There is also the :ref:`omnigraph_node_description_editor`, a work in progress that will give you a user interface assist in populating your node description. This document walks through the basics for writing nodes, accessing attribute data, and explains how the nodes fit into the general ecosystem of the OmniGraph. To get a walkthrough of the node writing process by way of examples that build on each other, from the simplest to most complex node go to the :ref:`ogn_tutorial_nodes`. This document will reference relevant tutorials when appropriate, but is intended to be more of a one-stop shop for all features of OmniGraph nodes. In the interests of clarity the code samples are kept in a separate document for :ref:`C++` and :ref:`Python` and referred to from here, rather than having everything embedded. If you are reading this from a web browser you probably want to open a new tab for those links when you visit them. .. contents:: .. .. note:: For the purpose of these examples the extension **ogn.examples** will be assumed, and names will follow the established naming conventions. .. warning:: The code referenced is for illustrative purposes only and some necessary elements may have been elided for clarity. It may not work as-is. Generated Files +++++++++++++++ Before you can write any nodes you must first teach your extension how to build them. These instructions are tailored for building using premake inside Omniverse Kit, with more generic information being provided to adapt them to any build environment. The core of the OmniGraph nodes is the .ogn file. Before actually writing a node you must enable processing of these files in the build of your extension. If your extension doesn't already support it you can follow the steps in :ref:`ogn_build_conversion` to add it. What the build process adds is a step that runs the :ref:`OGN Generator Script` on your .ogn file to optionally generate several files you will need for building, testing, running, and documenting your node. Once you have your .ogn file created, with your build .ogn-enabled as described above, you can run the build with just that file in place. If it all works you should see the following files added to the build directory. (PLATFORM can be *windows-x86_64* or *linux-x86_64*, and VARIANT can be *debug* or *release*, depending on what you are building.) - *_build/ogn/include/OgnMyNodeDatabase.h* - *_build/PLATFORM/VARIANT/exts/ogn.examples/docs/OgnMyNode.rst* - *_build/PLATFORM/VARIANT/exts/ogn.examples/ogn/examples/ogn/OgnMyNode.py* - *_build/PLATFORM/VARIANT/exts/ogn.examples/ogn/examples/tests/TestOgnMyNode.py* - *_build/PLATFORM/VARIANT/exts/ogn.examples/ogn/examples/tests/data/OgnMyDatabaseTemplate.usda* If these are not created, go back and check your build logs to confirm that your build is set up correctly and your .ogn file was processed correctly. .. note:: If your node is written in Python then the file *_build/ogn/include/OgnMyNodeDatabase.h* is unused and will not be generated. .. tip:: If you have an existing node you wish to convert to .ogn format then you can follow along with the detailed example of a node's conversion found in :ref:`ogn_node_conversion`. The Split OmniGraph Extension +++++++++++++++++++++++++++++ Most extensions are implemented atomically, with all code supporting the feature in a single extension. The OmniGraph core, however, was split into two. `omni.graph.core` is the basic support for nodes and their evaluation, and `omni.graph` is the added support for Python bindings and scripts. You almost always want your extension to have a dependency on `omni.graph`. The main reason for just using `omni.graph.core` is if you have a headless evaluation engine that has no scripting or UI, just raw calculations, and all of your nodes are written in C++. The Compute +++++++++++ The primary function of a node is to use a set of attribute values as input to its algorithm, which generates a set of output values. In its purest form the node compute operation will be purely functional; reading only received input attributes and writing its defined output attributes. In this form the node is capable of taking advantage of the maximum performance provided by threading and distributed computing. However, we recognize that not all interesting calculations can be expressed in that way, and many times should not, so OmniGraph is set up to handle more complex configurations such as self-contained subgraphs, internal structures, and persistent state data, as well as combining all types of nodes into arbitrarily complex graphs. As the node writer, what happens within the compute function is entirely up to you. The examples here are one possible approach to these algorithms. .. important:: It is important to note here that you should consider your node to be an island unto itself. It may live on a different thread, CPU, GPU, or even physical computer than other nodes in the graph. To guarantee correct functioning in all situations you should never inject or extract data to or from locations outside of your node. It should behave as a standalone evaluation engine. This includes other nodes, user interfaces, USD data, and anything else that is not part of the node's input or output attributes. Should your node require access to such data then you must provide OmniGraph with the :ref:`ogn_keyword_node_scheduling` information. Mandatory Node Properties +++++++++++++++++++++++++ There are properties on the node that are required for every legal file. The node must have a name, a :ref:`ogn_keyword_node_description`, and a :ref:`ogn_keyword_node_version`. Minimal node definition which includes only those elements. .. code-block:: json { "NoOp" : { "description": "Minimal node that does nothing", "version": 1 } } .. note:: As described in :ref:`omnigraph_naming_conventions` the actual unique name of this node will include the extension, and will be ``ogn.examples.NoOp``. These examples also illustrate some convenience functions added to the database that facilitate the reporting of warnings or errors encountered during a node's operation. A warning might be something incidental like a deformer running on an empty set of points. An error is for something serious like a divide-by-zero error in a calculation. Using this reporting methods makes debugging node operations much easier. Generally speaking a warning will still return true as the compute is successful, just not useful, whereas an error will return false indicating that the compute could not be performed. +---------------------------------------+-----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +---------------------------------------+-----------------------------------------+ Relevant tutorial - :ref:`ogn_tutorial_empty`. Although it's not mandatory in every file, the keyword :ref:`ogn_keyword_node_language` is required when you intend to implement your node in Python. For the above, and all subsequent examples, using the Python node implementation requires this one extra line in your .ogn file. (C++ is the default so it isn't necessary for nodes written in C++.) .. code-block:: json :emphasize-lines: 5 { "NoOp" : { "description": "Minimal node that does nothing in Python", "version": 1, "language": "python" } } Secondary Node Properties +++++++++++++++++++++++++ Some other node properties have simple defaults and need not always be specified in the file. These include :ref:`ogn_keyword_node_exclude`, :ref:`ogn_keyword_node_memoryType`, :ref:`ogn_keyword_node_cudaPointers`, :ref:`ogn_keyword_node_metadata`, :ref:`ogn_keyword_node_scheduling`, :ref:`ogn_keyword_node_tags`, :ref:`ogn_keyword_node_tokens`, and :ref:`ogn_keyword_node_uiName`. Providing Scheduling Hints -------------------------- The scheduler will try to schedule execution of the nodes in as efficient a manner as possible while still maintaining safe evaluation constraints (e.g. by not scheduling two nodes in parallel that are not threadsafe). Although it's not (yet) mandatory it is a good idea to provide a value for the :ref:`ogn_keyword_node_scheduling` keyword so that the scheduler has as much information as possible on how to efficiently scheduler your nodes. The ideal node has _"scheduling": "threadsafe"_, meaning it is safe to schedule that node in parallel with any other nodes. Excluding Generated Files ------------------------- If for some reason you want to prevent any of the normally generated files from being created you can do so within the .ogn file with the :ref:`ogn_keyword_node_exclude` keyword. For example you might be in a C++-only environment and want to prevent the Python test scripts and database access file from being created. .. code-block:: json :emphasize-lines: 5 { "NoOp" : { "description": "Minimal node that does nothing without Python support", "version": 1, "exclude": ["python", "tests"] } } In addition to the five generated file types listed above the reference guide shows that you can also exclude something called **"template"**. This file, if generated, would be a blank implementation of your node, in the language you've selected. It's not normally generated by the build, though it is useful for manual generation when you first start implementing a node. The :ref:`omnigraph_node_description_editor` uses this option to give you a blank node implementation to start with. Adding it to the exclusion list will prevent that. Relevant tutorial - :ref:`ogn_tutorial_abi`. .. _ogn_using_gpu_data: Using GPU Data -------------- Part of the benefit of using the .ogn format is that it's purely descriptive so it can handle nodes implemented in different languages and nodes that run on the CPU, the GPU, or both. The keyword :ref:`ogn_keyword_node_memoryType` is used to specify where the attribute data on a node should live. By default all of the node data lives on the CPU, however you can use this keyword to tell :ref:`omnigraph_concept_fabric` that the data instead lives on the GPU, in particular in CUDA format. .. code-block:: json :emphasize-lines: 5 { "NoOp" : { "description": "Minimal node that does nothing on the GPU", "version": 1, "memoryType": "cuda" } } Until you have attributes, though, this keyword has not effect. It is only the attribute's data that lives on :ref:`omnigraph_concept_fabric`. See :ref:`ogn_overriding_memory_location` for details on how it affects the code that access the attribute data. Relevant tutorials - :ref:`ogn_tutorial_cudaData` and :ref:`ogn_tutorial_cpuGpuData`. By default the memory references of CUDA array data will be GPU-pointer-to-GPU-pointer, for convenience in facilitating the use of arrays of arrays in an efficient manner. For single arrays, though, this may not be desirable and you might wish to just use a CPU-pointer-to-GPU-pointer so that it can be dereferenced on the CPU side. To do so you can add the *cudaPointers* keyword with your memory definition. .. code-block:: json :emphasize-lines: 6 { "NoOp" : { "description": "Minimal node that does nothing on the GPU", "version": 1, "memoryType": "cuda", "cudaPointers": "cpu" } } Adding Metadata To A Node Type ------------------------------ Node types can have a metadata dictionary associated with them that can be added through the :ref:`ogn_keyword_node_metadata` keyword. .. code-block:: json :emphasize-lines: 5-7 { "NodeMetadata" : { "description": "Minimal node that has some metadata", "version": 1, "metadata": { "author": "Bertram P. Knowedrighter" } } } .. note:: This is not the same as USD metadata. It is only accessible through the OmniGraph node type. .. tip:: Although all metadata is stored as a string:string mapping in OmniGraph, you can specify a list of strings in the .ogn file. It will be changed into a single CSV formatted comma-separated string. For example the list ["red", "green", "blue"] results in a single piece of metadata with the value "red,green,blue". The CSV escape mechanism is used for strings with embedded commas, so the list ["red,green", "blue"] results in the similar but different metadata "'red,green',blue". Any CSV parser can be used to safely extract the list of values. If your metadata does not contain commas then a simple tokenizer will also work. +----------------------------------------+------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +----------------------------------------+------------------------------------------+ Alternative Icon Location ------------------------- If the node file *OgnMyNode.ogn* has a file in the same directory named *OgnMyNode.svg* then that file will automatically be promoted to be the node's icon. If you wish to arrange your icons in a different way then you can specify a different location for the icon file using the :ref:`ogn_keyword_node_icon` keyword. The icon path will be relative to the directory in which the *.ogn* file lives so be sure to set your path accordingly. (A common location might be the *icons/* subdirectory.) .. code-block:: json :emphasize-lines: 5-7 { "NodeWithOtherIcon" : { "description": "Minimal node that uses a different icon", "version": 1, "icon": "icons/CompanyLogo.svg" } } .. note:: This file will be installed into the build area in your extension directory, under the subdirectory *ogn/icons/* so you don't have to install it into the build separately. When the icon is installed you can get at it by using the extension manager's ability to introspect its own path. Sometimes you might also wish to change the coloring of the icon. By default all of the colors are the same. Using this extended syntax for the icon specification lets you override the shape, border, and background color of the icon using either a **#AABBGGRR** hexadecimal format or a **[R, G, B, A]** decimal format. .. code-block:: json :emphasize-lines: 5-10 { "NodeWithOtherColoredIcon" : { "description": "Minimal node that uses a different colored icon", "version": 1, "icon": { "path": "icons/CompanyLogo.svg", "color": "#FF223344", "backgroundColor": [255, 0, 0, 0], "borderColor": [255, 128, 0, 128] } } } +-----------------------------------------+-------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-----------------------------------------+-------------------------------------------+ .. tip:: Although the node type icon information is set through the generated code, it is encoded in metadata and as such can be modified at runtime if you wish to further customize your look. Singleton Node Types -------------------- For some types of nodes it is undesirable to have more than one of them per graph, including any child graphs. To add this restriction a node can be marked as a "singleton" using the :ref:`ogn_keyword_node_singleton` keyword. It is a shortcut to defining specially named metadata whose presence will prevent more than one node of that type being instantiated. .. code-block:: json :emphasize-lines: 5-7 { "SingletonNode" : { "description": "Minimal node that can only be instantiated once per graph", "version": 1, "singleton": true } } .. note:: Node types with this flag set are not true singletons in the programming sense. You can instantiate more than one of them. The restriction is that they have to be in different graphs. +-----------------------------------------+---------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-----------------------------------------+---------------------------------------------+ Node Tags --------- Nodes can often be grouped in collections orthogonal to their extension owners or names - e.g. you might want the nodes *Add*, *Multiply*, and *Divide* to appear in a math collection, even though they may have been implemented in three unrelated extensions. This information appears in the internal metadata value ``tags``. Since it is so common, a more succinct method of specifying it is available with the :ref:`ogn_keyword_node_tags` keyword. It is a shortcut to defining that specially named metadata. Also, if it is specified as a list the tags string will contain the list of names separated by a comma, so these two definitions generate identical code: .. code-block:: json :emphasize-lines: 5-7 { "NodeTagged" : { "description": "Minimal node with keyword tags", "version": 1, "metadata": { "tags": "cool,new,improved" } } } .. code-block:: json :emphasize-lines: 5 { "NodeTagged" : { "description": "Minimal node with keyword tags", "version": 1, "tags": ["cool", "new", "improved"] } } +------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------+----------------------------------------+ Relevant tutorial - :ref:`ogn_tutorial_tupleData`. String Tokens ------------- A token is a unique ID that corresponds to an arbitrary string. A lot of the ABI makes use of tokens where the choices of the string values are limited, e.g. the attribute types, so that fast comparisons can be made. Using tokens requires accessing a token translation ABI, leading to a lot of duplicated boilerplate code to perform the common operation of translating a string into a token, and vice-versa. In addition, the translation process could be slow, so in order to experience the benefits of using a token it should only be done once where possible. To make this easier, the :ref:`ogn_keyword_node_tokens` keyword is provided in the .ogn file to predefine a set of tokens that the node will be using. For example if you are going to look up a fixed set of color names at runtime you can define the color names as tokens. .. code-block:: json :emphasize-lines: 5 { "Tokens" : { "description": "Minimal node that has some tokens", "version": 1, "tokens": ["red", "green", "blue"] } } When you use the alternative token representation you still access the tokens by the simplified name. So this definition, although the actual token values are different, uses the same code to access those values. .. code-block:: json :emphasize-lines: 5 { "Tokens" : { "description": "Minimal node that has some tokens", "version": 1, "tokens": {"red": "Candy Apple Red", "green": "English Racing Green", "blue": "Sky Blue"} } } As an added simplification, a simple interface to convert between tokens and strings is added to the database code for nodes in C++. It isn't necessary in Python since Python represents tokens directly as strings. +--------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +--------------------------------------+----------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_tokens`, :ref:`ogn_tutorial_bundle_manipulation`, :ref:`ogn_tutorial_extended_types`, and :ref:`ogn_tutorial_simpleData`. .. caution:: Although the simplified token access is implemented in Python, ultimately Python string comparisons are all done as strings, not as token IDs, due to the nature of Python so that code is for convenience, not efficiency. Providing A User-Friendly Node Type Name ---------------------------------------- While the unique node type name is useful for keeping things well organized it may not be the type of name you would want to see, e.g. in a dropdown interface when selecting the node type. A specially named metadata value has been reserved for that purpose, to give a consistent method of specifying a more user-friendly name for the node type. Since it is so common, a more succinct method of specifying it is available with the :ref:`ogn_keyword_node_uiName` keyword. It is a shortcut to defining that specially named metadata, so these two definitions generate identical code: .. code-block:: json :emphasize-lines: 5-7 { "NodeUiName" : { "description": "Minimal node with a UI name", "version": 1, "metadata": { "uiName": "Node With A UI Name" } } } .. code-block:: json :emphasize-lines: 5 { "NodeUiName" : { "description": "Minimal node with a UI name", "version": 1, "uiName": "Node With A UI Name" } } +--------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +--------------------------------------+----------------------------------------+ Almost every tutorial in :ref:`ogn_tutorial_nodes` make use of this special piece of metadata. Attribute Definitions --------------------- Attributes define the data flowing in and out of the node during evaluation. They are divided into three different locations with different restrictions on each location. .. code-block:: json :emphasize-lines: 5-13 { "NodeWithEmptyAttributes" : { "description": "Minimal node with empty attribute lists", "version": 1, "inputs": { "NAME": { "ATTRIBUTE_PROPERTY": "PROPERTY_VALUE" } }, "outputs": { "NAME": { "ATTRIBUTE_PROPERTY": "PROPERTY_VALUE" } }, "state": { "NAME": { "ATTRIBUTE_PROPERTY": "PROPERTY_VALUE" } } } } Each attribute section contains the name of an attribute in that location. See :ref:`omnigraph_naming_conventions` for a description of allowable names. The properties in the attribute definitions are described below in the sections on :ref:`ogn_mandatory_attribute_properties` and :ref:`ogn_secondary_attribute_properties`. **inputs** are treated as read-only during a compute, and within the database interface to the attribute data. The input values can only be set through a command, or the ABI. This is intentional, and should not be overridden as it could cause graph evaluation to become incorrect or unstable. **outputs** are values the node is to generate during a compute. From one evaluation to another they are not guaranteed to be valid, or even exist, so it is the node's responsibility to define them and set their values during the compute. (Optimizations to this process exist, but are beyond the scope of this document.) **state** attributes persist from one evaluation to the next and are readable and writable. It is the node's responsibility to ensure that they initialize correctly, either by explicit initialization in the node or through use of a recognizable default value that indicate an uninitialized state. Other than the access restrictions described above the attributes are all described in the same way so any of the keywords descriptions shown for one attribute location type can be used for any of them. Automatic Test Definitions -------------------------- It is always a good idea to have test cases for your node to ensure it is and continues to be operating correctly. The .ogn file helps with this process by generating some simple test scenarios automatically, along with a script that will exercise them within the test environment. .. code-block:: json :emphasize-lines: 5 { "NodeWithEmptyAttributes" : { "description": "Minimal node with empty attribute lists", "version": 1, "tests": [ { "TEST_PROPERTY": "TEST_VALUE" } ] } } This subsection will contain a list of such test definitions. More detail on the **TEST_PROPERTY** values is available in the discussion on :ref:`ogn_defining_automatic_tests`. .. _ogn_mandatory_attribute_properties: Mandatory Attribute Properties ++++++++++++++++++++++++++++++ All attributes in any location subsection has certain minimally required properties. The attribute must have a name, a :ref:`ogn_keyword_attribute_description`, and a :ref:`ogn_keyword_attribute_type`. This is a minimal node definition with one simple integer value attribute. .. code-block:: json { "Ignore" : { "description": "Ignore an integer value", "version": 1, "inputs": { "x": { "description": "Value to be ignored", "type": "int" } } } } The value of the **"type"** property can create very different interfaces to the underlying data. Although the syntax in the file is the same for every type (with one exception, explained below) the generated access methods are tuned to be natural for the type of underlying data. See the document on :ref:`ogn_attribute_types` for full details on the accepted attribute types and how they correspond to C++, Python, JSON, and USD types. The data types can be divided into categories, explained separately here though there can be any arbitrary amount of type mixing. .. note:: The attribute type **"execution"** can also be specified. These attributes do not carry any data, they merely exist to form connections to trigger node sequences to evaluate based on external conditions. This behavior can only be seen at the graph level, not at the individual node level. Simple Data Attribute Types --------------------------- These denote individual values with a fixed size such as float, int, etc. In Fabric they are stored directly, using the size of the type to determine how much space to allocate. This example will illustrate how to access simple data of type float and token. A full set of compatible types and how they are accessed can be found in :ref:`ogn_attribute_types`. .. code-block:: json { "TokenStringLength" : { "description": "Compute the length of a tokenized string, in characters", "version": 1, "inputs": { "token": { "description": "Value whose length is to be calculated", "type": "token" } }, "outputs": { "length": { "description": "Number of characters in the input token's string", "type": "int64" } } } } +------------------------------------------+--------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------------+--------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_simpleData`, :ref:`ogn_tutorial_simpleDataPy` .. note:: Tokens are simple data types as they have a fixed size in Fabric, however strings do not. Using them is a special case described in :ref:`ogn_string_attribute_type`. Tuple Data Attribute Types -------------------------- These denote fixed numbers of simple values, such as double[4], vectord[3], etc. Each tuple value can be treated as a single entity, but also provide access to individual tuple elements. In Fabric they are stored directly, using the size of the simple type and the tuple count to determine how much space to allocate. .. code-block:: json { "VectorMultiply" : { "description": "Multiple two mathematical vectors to create a matrix", "version": 1, "inputs": { "vector1": { "description": "First vector to multiply", "type": "double[4]" }, "vector2": { "description": "Second vector to multiply", "type": "double[4]" }, "outputs": { "product": { "description": "Matrix equal to the product of the two input vectors", "type": "matrixd[4]" } } } } +------------------------------------------+--------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------------+--------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_simpleData`, :ref:`ogn_tutorial_abi`, :ref:`ogn_tutorial_cudaData`, :ref:`ogn_tutorial_cpuGpuData`, :ref:`ogn_tutorial_simpleDataPy`, :ref:`ogn_tutorial_abi_py`, :ref:`ogn_tutorial_state_py`, :ref:`ogn_tutorial_defaults`, and :ref:`ogn_tutorial_state_attributes_py`. Role Data Attribute Types ------------------------- Roles are specially named types that assign special meanings to certain tuple attribute types. See the details of what types are available in :ref:`ogn_attribute_roles`. .. code-block:: json { "PointsToVector" : { "description": "Calculate the vector between two points", "version": 1, "inputs": { "point1": { "description": "Starting point of the vector", "type": "pointf[4]" }, "point2": { "description": "Ending point of the vector", "type": "pointf[4]" } }, "outputs": { "vector": { "description": "Vector from the starting point to the ending point", "type": "vectorf[4]" } } } } +------------------------------------+--------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------+--------------------------------------+ Relevant tutorial - :ref:`ogn_tutorial_roleData`. Array Data Attribute Types -------------------------- These denote variable numbers of simple values, such as double[], bool[], etc. Although the number of elements they contain is flexible they do not dynamically resize as a ``std::vector`` might, the node writer is responsible for explicitly setting the size of outputs and the size of inputs is fixed when the compute is called. In Fabric they are stored in two parts - the array element count, indicating how many of the simple values are contained within the array, and as a flat piece of memory equal in size to the element count times the size of the simple value. .. code-block:: json { "PartialSum" : { "description": [ "Calculate the partial sums of an array. Element i of the output array", "is equal to the sum of elements 0 through i of the input array" ], "version": 1, "inputs": { "array": { "description": "Array whose partial sum is to be computed", "type": "float[]" } }, "outputs": { "partialSums": { "description": "Partial sums of the input array", "type": "float[]" } } } } .. important:: There is no guarantee in Fabric that the array data and the array size information are stored together, or even in the same memory space. The generated code takes care of this for you, but if you decide to access any of the data directly through the ABI you should be aware of this. +------------------------------------------+--------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------------+--------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_arrayData`, :ref:`ogn_tutorial_cudaData`, :ref:`ogn_tutorial_cpuGpuData`, :ref:`ogn_tutorial_complexData_py`, :ref:`ogn_tutorial_defaults`, and :ref:`ogn_tutorial_tokens`. Tuple-Array Data Attribute Types -------------------------------- These denote variable numbers of a fixed number of simple values, such as pointd[3][], int[2][], etc. In principle they are accessed the same as regular arrays, with the added capability of accessing the individual tuple values on the array elements. In Fabric they are stored in two parts - the array element count, indicating how many of the tuple values are contained within the array, and as a flat piece of memory equal in size to the element count times the tuple count times the size of the simple value. The tuple elements appear contiguously in the data so for example the memory layout of a **float[3][]** named `t` implemented with a struct containing x, y, z, would look like this: +--------+--------+--------+--------+--------+--------+--------+------+ | t[0].x | t[0].y | t[0].z | t[1].x | t[1].y | t[1].z | t[2].x | etc. | +--------+--------+--------+--------+--------+--------+--------+------+ .. code-block:: json { "CrossProducts" : { "description": "Calculate the cross products of an array of vectors", "version": 1, "inputs": { "a": { "description": "First set of vectors in the cross product", "type": "vectord[3][]", "uiName": "First Vectors" } "b": { "description": "Second set of vectors in the cross product", "type": "vectord[3][]", "uiName": "Second Vectors" } }, "outputs": { "crossProduct": { "description": "Cross products of the elements in the two input arrays", "type": "vectord[3][]" } } } } +-------------------------------------------+---------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-------------------------------------------+---------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_tupleArrays`, :ref:`ogn_tutorial_cudaData`, :ref:`ogn_tutorial_cpuGpuData`, and :ref:`ogn_tutorial_complexData_py`. .. _ogn_string_attribute_type: String Attribute Type --------------------- String data is slightly different from the others. Although it is conceptually simple data, being a single string value, it is treated as an array in Fabric due to its size allocation requirements. Effort has been made to make the data accessed from string attributes to appear as much like a normal string as possible, however there is a restriction on modifications that can be made to them as they have to be resized in Fabric whenever they change size locally. For that reason, when modifying output strings it is usually best to do all string operations on a local copy of the string and then assign it to the output once. .. code-block:: json { "ReverseString" : { "description": "Output the string in reverse order", "version": 1, "inputs": { "original": { "description": "The string to be reversed", "type": "string" } }, "outputs": { "reversed": { "description": "Reversed string", "type": "string" } } } } .. caution:: At this time there is no support for string arrays. Use tokens instead for that purpose. +--------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +--------------------------------------+----------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_simpleData` and :ref:`ogn_tutorial_simpleDataPy`. .. _ogn_extended_attribute_types: Extended Attribute Type - Any ----------------------------- Sometimes you may want to create a node that can accept a wide variety of data types without the burden of implementing a different attribute for every acceptable type. For this case the **any** type was introduced. When an attribute has this type it means "allow connections to any type and resolve the type at runtime". Practically speaking this type resolution can occur in a number of ways. The main way it resolves now is to create a connection from an **any** type to a concrete type, such as **float**. Once the connection is made the **any** attribute type will be resolved and then behave as a **float**. The implication of this flexibility is that the data types of the **any** attributes cannot be assumed at build time, only at run time. To handle this flexibility, an extra wrapper layer is added to such attributes to handle identification of the resolved type and retrieval of the attribute data as that specific data type. .. code-block:: json { "Add" : { "description": "Compute the sum of two arbitrary values", "version": 1, "inputs": { "a": { "description": "First value to be added", "type": "any" }, "b": { "description": "Second value to be added", "type": "any" } }, "outputs": { "sum": { "description": "Sum of the two inputs", "type": "any" } } } } .. caution:: At this time the extended attribute types are not allowed to resolve to :ref:`ogn_bundle_attribute_types`. +-----------------------------------+-------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-----------------------------------+-------------------------------------+ Relevant tutorial - :ref:`ogn_tutorial_extended_types`. Extended Attribute Type - Union ------------------------------- The **union** type is similar to the **any** type in that its actual data type is only decided at runtime. It has the added restriction of only being able to accept a specific subset of data types, unlike the **any** type that can literally be any of the primary attribute types. The way this is specified in the .ogn file is, instead of using the type name **"union"**, you specify the list of allowable attribute types. Here's an example that can accept either double or float values, but nothing else. .. code-block:: json { "MultiplyNumbers" : { "description": "Compute the product of two float or double values", "version": 1, "inputs": { "a": { "description": "First value to be added", "type": ["double", "float"] }, "b": { "description": "Second value to be added", "type": ["double", "float"] } }, "outputs": { "product": { "description": "Product of the two inputs", "type": ["double", "float"] } } } } Other than this restriction, which the graph will attempt to enforce, the **union** attributes behave exactly the same way as the **any** attributes. +-------------------------------------+---------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-------------------------------------+---------------------------------------+ Relevant tutorial - :ref:`ogn_tutorial_extended_types`. .. _ogn_bundle_attribute_types: Bundle Attribute Types ---------------------- A bundle doesn't describe an attribute with a specific type of data itself, it is a container for a runtime-curated set of attributes that do not have definitions in the .ogn file. .. code-block:: json { "MergeBundles" : { "description": [ "Merge the contents of two bundles together.", "It is an error to have attributes of the same name in both bundles." ], "version": 1, "inputs": { "bundleA": { "description": "First bundle to be merged", "type": "bundle", }, "bundleB": { "description": "Second bundle to be merged", "type": "bundle", } }, "outputs": { "bundle": { "description": "Result of merging the two bundles", "type": "bundle", } } } } +--------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +--------------------------------------+----------------------------------------+ .. code-block:: json "CalculateBrightness": { "version": 1, "description": "Calculate the brightness value for colors in various formats", "tokens": ["r", "g", "b", "c", "m", "y", "k"], "inputs": { "color": { "type": "bundle", "description": [ "Color value, in a variety of color spaces. The bundle members can either be floats", "named 'r', 'g', 'b', and 'a', or floats named 'c', 'm', 'y', and 'k'." ] } }, "outputs": { "brightness": { "type": "float", "description": "The calculated brightness value" } } } +--------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +--------------------------------------+----------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_bundle_manipulation` :ref:`ogn_tutorial_bundle_data`, and :ref:`ogn_tutorial_bundle_add_attributes`. .. _ogn_secondary_attribute_properties: Secondary Attribute Properties ++++++++++++++++++++++++++++++ Some other attribute properties have simple defaults and need not always be specified in the file. These include :ref:`ogn_keyword_attribute_default`, :ref:`maximum `, and :ref:`ogn_keyword_attribute_memoryType`, :ref:`ogn_keyword_attribute_metadata`, :ref:`minimum `, and :ref:`ogn_keyword_attribute_optional`, :ref:`ogn_keyword_attribute_unvalidated`, :ref:`ogn_keyword_attribute_uiName`. Setting A Default ----------------- If you don't set an explicit default then the attributes will go to their "natural" default. This is False for a boolean, zeroes for numeric values including tuples, an empty array for all array types, an empty string for string and token types, and the identity matrix for matrix, frame, and transform types. Attributes whose types are resolved at runtime (any, union, and bundle) have no defaults and start in an unresolved state instead. Sometimes you need a different default though, like setting a scale value to (1.0, 1.0, 1.0), or a token to be used as an enum to one of the enum values. To do that you simply use the :ref:`ogn_keyword_attribute_default` keyword in the attribute definition. When it is created it will automatically assume the specified default value. .. code-block:: json :emphasize-lines: 5 { "HairColors": { "version": 1, "description": "Collect hair colors for various characters", "inputs": { "sabine": { "type": "token", "description": "Color of Sabine's hair", "default": "red" } } } } As there is no direct way to access the default values on an attribute yet, no example is necessary. Relevant tutorials - :ref:`ogn_tutorial_defaults`, :ref:`ogn_tutorial_simpleData` and :ref:`ogn_tutorial_tupleData`. .. _ogn_overriding_memory_location: Overriding Memory Location -------------------------- As described in the :ref:`ogn_using_gpu_data` section, attribute memory can be allocated on the CPU or on the GPU. If all attributes are in the same location then the node :ref:`ogn_keyword_node_memoryType` keyword specifies where all of the attribute memory resides. If some attributes are to reside in a different location then those attributes can override the memory location with their :ref:`ogn_keyword_attribute_memoryType` keyword. .. code-block:: json :emphasize-lines: 5,10,21 { "GpuSwap" : { "description": "Node that optionally moves data from the GPU to the CPU", "version": 1, "memoryType": "any", "inputs": { "sizeThreshold": { "type": "int", "description": "The number of points at which the computation should be moved to the GPU", "memoryType": "cpu" }, "points": { "type": "pointf[3][]", "description": "Data to move" } }, "outputs": { "points": { "type": "pointf[3][]", "description": "Migrated data, values unchanged" } } } } In this description the `inputs:sizeThreshold` data will live on the CPU due to the override, the `inputs:points` data and the `outputs:points` data will be decided at runtime. +------------------------------------------------+--------------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------------------+--------------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_cudaData` and :ref:`ogn_tutorial_cpuGpuData`. Attribute Metadata ------------------ Attributes have a metadata dictionary associated with them in the same way that node types do. Some values are automically generated. Others can be added manually through the :ref:`ogn_keyword_attribute_metadata` keyword. .. code-block:: json :emphasize-lines: 6-8 { "StarWarsCharacters": { "version": 1, "description": "Database of character information", "inputs" : { "anakin": { "description": "Jedi Knight", "type": "token", "metadata": { "secret": "He is actually Darth Vader" } } } } } .. note:: This is not the same as USD metadata. It is only accessible through the OmniGraph attribute type. One special metadata item with the keyword ``allowedTokens`` can be attached to attributes of type ``token``. It will be automatically be added to the USD Attribute's metadata. Like regular tokens, if the token string contains any special characters it must be specified as a dictionary whose key is a legal code variable name name and whose value is the actual string. +---------------------------------------------+-----------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +---------------------------------------------+-----------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_abi_py` Suggested Minimum/Maximum Range ------------------------------- Numeric values can specify a suggested legal range using the keywords :ref:`minimum ` and :ref:`maximum `. These are not used at runtime at the moment, only within the .ogn file to verify legality of the default values, or values specified in tests. Default values can be specified on simple values, tuples (as tuples), arrays (as simple values applied to all array elements), and tuple-arrays (as tuple values applied to all array elements). .. code-block:: json :emphasize-lines: 5-6,11-12,17-18,23-24 "MinMax": { "version": 1, "description": "Attribute test exercising the minimum and maximum values to verify defaults", "inputs": { "simple": { "description": "Numeric value in [0.0, 1.0]", "type": "double", "minimum": 0.0, "maximum": 1.0 }, "tuple": { "description": "Tuple[2] value whose first value is in [-1.0, 1.0] with second value in [0.0, 255.0]", "type": "double[2]", "minimum": [-1.0, 0.0], "maximum": [1.0, 255.0] }, "array": { "description": "Array value where every element is in [0, 255]", "type": "uint", "minimum": 0, "maximum": 255 }, "tupleArray": { "description": "Array of tuple[2] values whose first value is in [5, 10] and second value is at least 12", "type": "uchar[2][]", "minimum": [5, 12], "maximum": [10, 255] } } } Relevant tutorials - :ref:`ogn_node_conversion`. Optional Attributes ------------------- Usually an attribute value must exist and be legal in order for a node's compute to run. This helps the graph avoid executing nodes that cannot compute their outputs due to missing or illegal inputs. Sometimes a node is capable of computing an output without certain inputs being present. Those inputs can use the :ref:`ogn_keyword_attribute_optional` keyword to indicate to OmniGraph that it's okay to compute without it. .. code-block:: json :emphasize-lines: 5 { "Shoes": { "version": 1, "description": "Create a random shoe type", "inputs": { "shoelaceStyle": { "type": "token", "description": "If the shoe type needs shoelaces this will contain the style of shoelace to use", "optional": true } }, "outputs": { "shoeType": { "type": "string", "description": "Name of the randomly generated shoe" } } } } It is up to the node to confirm that such optional attributes have legal values before they use them. +-----------------------------------+-------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-----------------------------------+-------------------------------------+ Unvalidated Attributes For Compute ---------------------------------- Above you can see how attributes may optionally not be required to exist depending on your node function. There is also a slightly weaker requirement whereby the attributes will exist but they need not have valid values in order for ``compute()`` to be called. Those attributes can use the :ref:`ogn_keyword_attribute_unvalidated` keyword to indicate to OmniGraph that it's okay to compute without verifying it. The most common use of this is to handle the case of attributes whose values will only be used under certain circumstances, especially :ref:`ogn_extended_attribute_types`. .. code-block:: json :emphasize-lines: 13,18 { "ABTest": { "version": 1, "description": "Choose one of two inputs based on some input criteria", "inputs": { "selectA": { "type": "bool", "description": "If true then pass through input a, else pass through input b" }, "a": { "type": "any", "description": "First choice for the a/b test", "unvalidated": true }, "b": { "type": "any", "description": "Second choice for the a/b test", "unvalidated": true } }, "outputs": { "choice": { "type": "any", "description": "Result from the a/b test choice" } } } } It is up to the node to confirm that such attributes have legal values before they use them. Notice here that the output will be validated. In particular, it will have its resolved type validated before calling ``compute()``. After that the node will have to confirm that the selected input, a or b, has a type that is compatible with that resolved type. +--------------------------------------+----------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +--------------------------------------+----------------------------------------+ Providing A User-Friendly Attribute Name ---------------------------------------- While the unique attribute name is useful for keeping things well organized it may not be the type of name you would want to see, e.g. in a dropdown interface when selecting the attribute. A specially named metadata value has been reserved for that purpose, to give a consistent method of specifying a more user-friendly name for the attribute. Since it is so common, a more succinct method of specifying it is available with the :ref:`ogn_keyword_attribute_uiName` keyword. It is a shortcut to defining that specially named metadata, so these two definitions generate identical code: .. code-block:: json :emphasize-lines: 6-8 { "AttributeUiName": { "version": 1, "description": "No-op node showing how to use the uiName metadata on an attribute", "inputs" : { "x": { "description": "X marks the spot", "type": "pointf[3]", "metadata": { "uiName": "Treasure Location" } } } } } .. code-block:: json :emphasize-lines: 6 { "AttributeUiName": { "version": 1, "description": "No-op node showing how to use the uiName metadata on an attribute", "inputs" : { "x": { "description": "X marks the spot", "type": "pointf[3]", "uiName": "Treasure Location" } } } } +-------------------------------------------+---------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +-------------------------------------------+---------------------------------------------+ Almost every tutorial in :ref:`ogn_tutorial_nodes` make use of this special piece of metadata. .. _ogn_defining_automatic_tests: Defining Automatic Tests ++++++++++++++++++++++++ It is good practice to always write tests that exercise your node's functionality. Nodes that are purely functional, that is their outputs can be calculated using only their inputs, can have simple tests written that set certain input values and compare the outputs against expected results. To make this process easier the **"tests"** section of the .ogn file was created. It generates a Python test script in the Kit testing framework style from a set of input, output, and state values on the node. The algorithm is simple. For each test in the list it sets input and state attributes to the values given in the test description, using default values for any unspecified attributes, runs the compute on the node, then gathers the computed outputs and compares them against the expected ones in the test description, ignoring any that did not appear there. There are two ways of specifying test data. They are both equivalent so you can choose the one that makes your particular test data the most readable. The first is to have each test specify a dictionary of *ATTRIBUTE* : *VALUE*. This is a simple node that negates an input value. The tests run a number of example values to ensure the correct results are obtained. Four tests are run, each independent of each other. .. code-block:: json :emphasize-lines: 17-22 { "NegateValue": { "version": 1, "description": "Testable node that negates an input value", "inputs" : { "value": { "description": "Value to negate", "type": "float" } }, "outputs": { "result": { "description": "Negated value of the input", "type": "float" } }, "tests": [ { "inputs:value": 5.0, "outputs:result": -5.0 }, { "inputs:value": 0.0, "outputs:result": 0.0 }, { "inputs:value": -5.0, "outputs:result": 5.0 }, { "outputs:result": 0.0 } ] } } Note how the last test relies on using the default input value, which for floats is 0.0 unless otherwise specified. The tests illustrate a decent coverage of the different possible types of inputs. The other way of specifying tests is to use the same type of hierarchical dictionary structure as the attribute definitions. The attribute names are thus shorter. This .ogn file generates exactly the same test code as the one above, with the addition of test descriptions to add more information at runtime. .. code-block:: json :emphasize-lines: 17-51 { "NegateValue": { "version": 1, "description": "Testable node that negates an input value", "inputs" : { "value": { "description": "Value to negate", "type": "float" } }, "outputs": { "result": { "description": "Negated value of the input", "type": "float" } }, "tests": [ { "description": "Negate a positive number", "inputs": { "value": 5.0 }, "outputs": { "result": -5.0 } }, { "description": "Negate zero", "inputs": { "value": 0.0 }, "outputs": { "result": 0.0 } }, { "description": "Negate a negative number", "inputs": { "value": -5.0 }, "outputs": { "result": 5.0 } }, { "description": "Negate the default value", "outputs": { "result": 0.0 } } ] } } For this type of simple node you'd probably use the first, abbreviated, version of the test description. The second type is more suited to nodes with many inputs and outputs. In addition, if you require more than one node to properly set up your test you can use this format to add in a special section defining the state of the graph before the tests start. For example if you want to test two nodes chained together you could do this: .. code-block:: json :emphasize-lines: 24-42 { "AddTwoValues": { "version": 1, "description": "Testable node that adds two input values", "inputs" : { "a": { "description": "First value to add", "type": "float" }, "b": { "description": "Second value to add", "type": "float" } }, "outputs": { "result": { "description": "Sum of the two inputs", "type": "float" } }, "tests": [ { "description": "Sum a constant and a connected value", "setup": { "nodes": [ ["TestNode", "omni.examples.AddTwoValues"], ["InputNode", "omni.examples.AddTwoValues"] ], "prims": [ ["InputPrim", {"value": ["float", 5.0]}] ], "connections": [ ["InputPrim", "value", "TestNode", "inputs:a"] ] }, "outputs": { "result": 5.0 } }, { "inputs:b": 7.0, "outputs:result": 12.0 } ] } } When there is more than one test the setup that happened in the previous test will still be applied. It will be as though the tests are run on live data in sequence. To reset the setup configuration put a new one in your test, including simply *{}* if you wish to start with an empty scene. There is no C++ or Python code that access the test information directly, it is only used to generate the test script. In addition to your defined tests, extra tests are added to verify the template USD file, if it was generated, and the import of the Python database module, if it was generated. The test script itself will be installed into a subdirectory of your Python import directory, e.g. ``ogn.examples/ogn/examples/ogn/tests/TestNegateValue.py`` Relevant tutorials - :ref:`ogn_tutorial_simpleDataPy`, :ref:`ogn_tutorial_complexData_py`, :ref:`ogn_tutorial_abi_py`, :ref:`ogn_tutorial_state_py`, :ref:`ogn_tutorial_defaults`, :ref:`ogn_tutorial_state_attributes_py`, :ref:`ogn_tutorial_state`, :ref:`ogn_tutorial_simpleData`, :ref:`ogn_tutorial_tokens`, :ref:`ogn_tutorial_tokens`, :ref:`ogn_tutorial_abi`, :ref:`ogn_tutorial_tupleData`, :ref:`ogn_tutorial_arrayData`, :ref:`ogn_tutorial_tupleArrays`, :ref:`ogn_tutorial_roleData`, :ref:`ogn_tutorial_cudaData`, :ref:`ogn_tutorial_cpuGpuData`, and :ref:`ogn_tutorial_cpu_gpu_extended`. Internal State ++++++++++++++ In addition to having state attributes you may also need to maintain state information that is not representable as a set of attributes; e.g. binary data, arbitrary C++ structures, etc. Per-node internal state is the mechanism that accommodates this need. The approach is slightly different in C++ and Python but the intent is the same. The internal state data is a node-managed piece of data that persists on the node from one evaluation to the next (though not across file load and save). There is nothing to do in the .ogn file to indicate that internal state of this kind is being used. The ABI function ``hasState()`` will return true when it is being used, or when state attributes exist on the node. .. code-block:: json { "Counter" : { "description": "Count the number of times the node executes", "version": 1 } } +------------------------------------------+--------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------------+--------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_state` and :ref:`ogn_tutorial_state_py`. Versioning ++++++++++ Over time your node type will evolve and you will want to change things within it. When you do that you want all of the old versions of that node type to continue working, and update themselves to the newer version automatically. The ABI allows for this by providing a callback to the node that happens whenever a node with a version number lower than the current version number. (Recall the version number is encoded in the .ogn property **"version"** and in the USD file as the property **custom int node:typeVersion**.) The callback provides the version it was attempting to create and the version to which it should be upgraded and lets the node decide what to do about it. The exact details depend greatly on what changes were made from one version to the next. This particular node is in version 2, where the second version has added the attribute *offset* because the node function has changed from ``result = a * b`` to ``result = a * b + offset``. .. code-block:: json :emphasize-lines: 3,14-17 { "Multiply": { "version": 2, "description": "Node that multiplies two values and adds an offset", "inputs" : { "a": { "description": "First value", "type": "float" }, "b": { "description": "Second value", "type": "float" }, "offset": { "description": "Offset value", "type": "float" } }, "outputs": { "result": { "description": "a times b plus offset", "type": "float" } } } } +------------------------------------------+--------------------------------------------+ | :ref:`C++ Code` | :ref:`Python Code` | +------------------------------------------+--------------------------------------------+ Relevant tutorials - :ref:`ogn_tutorial_abi` and :ref:`ogn_tutorial_abi_py`. Other References ++++++++++++++++ - :ref:`Naming and File Conventions` - :ref:`Setting Up Your Build` - :ref:`OGN Reference Guide` - :ref:`Feature-Based Tutorials` - :ref:`Attribute Type Details` - :ref:`OmniGraph Python API`