Integrating an Authoring Layer
In this article, a toy example using the Execution Framework is used to describe an online bakery. While the simplistic subject matter of the example is contrived, the concepts demonstrated in the example have real-world applications.
The article is structured such that the example starts simple, and new concepts are introduced piecemeal.
The Authoring Layer
The Execution Framework, in particular the execution graph, is a common language to describe execution across disparate software components. It is the job of each component (or an intermediary) to populate the execution graph based on some internal description. We call this per-component, internal description the authoring layer. It is common to have multiple different authoring layers contribute to a single execution graph.
This example demonstrate a single authoring layer that describes several online bakeries. The data structures used by this authoring layer is as follows:
struct BakedGood
{
unsigned int bakeMinutes;
std::string name;
};
struct Order
{
std::string customer;
std::vector<BakedGood> bakedGoods;
};
struct Bakery
{
std::string name;
std::vector<Order> orders;
};
The example starts by describing two bakeries at the authoring layer:
std::vector<Bakery> bakeries {
Bakery {
"The Pie Hut", // bakery name
{
Order {
"Tracy", // customer
{
BakedGood { 20, "applePie" },
BakedGood { 30, "chickenPotPie" }
}
},
Order {
"Kai", // customer
{
BakedGood { 22, "peachPie" },
}
}
}
},
Bakery {
"Sam's Bakery", // bakery name
{
Order {
"Alex", // customer
{
BakedGood { 20, "blueberryPie" },
}
}
}
}
};
Setting Up the Execution Graph
With the authoring layer defined, the following code is then used to populate the execution graph based on the authoring layer description:
// this example manually creates an execution graph. in most applications (e.g. kit-based applications) this will
// already be created for you
GraphPtr graph = Graph::create("exec.graph");
// as mentioned above, the builder context, pass pipeline, and builder will likely already be created for you in
// real-world scenarios.
GraphBuilderContextPtr builderContext{ GraphBuilderContext::create(graph, PassPipeline::create()) };
GraphBuilderPtr builder{ GraphBuilder::create(builderContext) };
// ef relies on the user to maintain a reference (i.e. a call omni::core::Object::acquire()) on each node in a graph
// definition. this can be done by simply holding an array of NodePtr objects in your definition. in this case,
// since we're populating the top-level graph definition, we simply store the NodePtrs here.
std::vector<NodePtr> nodes;
for (auto& bakery : bakeries) // for each bakery
{
auto node = Node::create(
graph, // this makes the node a part of the execution graph's top-level graph definition
BakeryGraphDef::create(builder, bakery), // bakery's definition (i.e. work description)
carb::fmt::format("node.bakery.{}", bakery.name)
);
// connect the bakery to the root of the execution graph's definition so that it will be executed. only nodes
// in a graph definition that can reach the definition's root node will be executed.
builder->connect(graph->getRoot(), node);
nodes.emplace_back(std::move(node));
}
The execution graph can be visualized as follows:
You can see the execution graph has several types of entities:
Nodes are represented by rounded boxes. Their name starts with “node.”.
Opaque Definitions are represented by angled boxes. Their name starts with “def.”.
Graph Definitions are represented by shaded boxes. Their name, at the top of the box, starts with “def.”.
Root Nodes are represented by circles. Their name is not shown.
Edges, represented by an arrow with a solid line, show the orchestration ordering between nodes.
Each node points to a definition, either an opaque definition or a graph definition. This relationship is represented by an arrow with a dotted line. Note, definitions can be pointed to by multiple nodes, though this example does not utilize the definition sharing feature of EF.
To simplify the example, this article focuses on a single bakery. Below, you can see a visualization of only The Pie Hut’s graph definition:
Building a Graph Definition
When creating the execution graph, most of the work is done in BakeryGraphDef, which is defined as follows:
class BakeryGraphDef : public NodeGraphDef // NodeGraphDef is an ef provided implementation of INodeGraphDef
{
public:
// implementations of ef interfaces are encouraged to define a static create() method. this method returns an
// ObjectPtr which correctly manages the reference count of the returned object.
//
// when defining api methods like create(), the use of ObjectParam<> to accept ONI object is encouraged. below,
// ObjectParam<IGraphBuilder> is a light-weight object that will accept either a raw IGraphBuilder* or a
// GraphBuilderPtr.
static omni::core::ObjectPtr<BakeryGraphDef> create(
omni::core::ObjectParam<IGraphBuilder> builder,
const Bakery& bakery) noexcept
{
// the pattern below of creating an graph definition followed by calling build() is common. in libraries like
// OmniGraph, all definitions are subclassed from a public interface that specifies a build_abi() method. since
// the build method is virtual (in OG, not here), calling build_abi() during the constructor would likely lead
// to incorrect behavior (i.e. calling a virtual method on an object that isn't fully instantiated is an
// anti-pattern). by waiting to call build() after the object is fully instantiated, as below, the proper
// build_abi() will be invoked.
auto def = omni::core::steal(new BakeryGraphDef(builder->getGraph(), bakery));
def->build(builder);
return def;
}
// build() (usually build_abi()) is a method often seen in ef definitions. it usually serves two purposes:
//
// - build the graph definition's graph
//
// - update the graph definition's graph when something has changed in the authoring layer
//
// note, this example doesn't consider updates to the authoring layer.
void build(omni::core::ObjectParam<IGraphBuilder> parentBuilder) noexcept
{
// when building a graph definition, a *dedicated* builder must be created to handle connecting nodes and
// setting the node's definitions.
//
// below, notice the use of 'auto'. use of auto is highly encouraged in ef code. in many ef methods, it is
// unclear if the return type is either a raw pointer or a smart ObjectPtr. by using auto, the caller doesn't
// need to care and "The Right Thing (TM)" will happen.
auto builder{ GraphBuilder::create(parentBuilder, this) };
// when using the build method to repspond to update in the authoring layer, we clear out the old nodes (if
// any). a more advanced implementation may choose to reuse nodes to avoid memory thrashing.
m_nodes.clear();
if (m_bakery.orders.empty())
{
// no orders to bake. don't turn on the oven.
return; // LCOV_EXCL_LINE
}
m_preheatOven = Node::create(
getTopology(), // each node must be a part of a single topology. getTopology() returns this defs topology.
PreHeatOvenNodeDef::create(m_bakery),
carb::fmt::format("node.bakery.{}.preHeatOven", m_bakery.name)
);
// connecting nodes in a graph must go through the GraphBuilder object created to construct this graph
// definition
builder->connect(getRoot(), m_preheatOven);
m_turnOffOven = Node::create(
getTopology(),
TurnOffOvenNodeDef::create(m_bakery),
carb::fmt::format("node.bakery.{}.turnOffOven", m_bakery.name)
);
for (auto& order : m_bakery.orders) // for each order
{
if (!order.bakedGoods.empty()) // make sure the order isn't empty
{
auto ship = Node::create(
getTopology(),
ShipOrderNodeDef::create(order),
carb::fmt::format("node.bakedGood.ship.{}", order.customer)
);
for (const BakedGood& bakedGood : order.bakedGoods) // for each item in the order
{
auto prepare = Node::create(
getTopology(),
PrepareBakedGoodGraphDef::create(builder, bakedGood),
carb::fmt::format("node.bakedGood.prepare.{}", bakedGood.name)
);
auto bake = Node::create(
getTopology(),
NodeDefLambda::create( // NodeDefLambda is an opaque def which uses a lambda to perform work
"def.bakedGood.bake",
[&bakedGood = bakedGood](ExecutionTask& info)
{
log("baking {} for {} minutes", bakedGood.name, bakedGood.bakeMinutes);
return Status::eSuccess;
},
SchedulingInfo::eParallel
),
carb::fmt::format("node.bakedGood.bake.{}", bakedGood.name)
);
builder->connect(getRoot(), prepare);
builder->connect(prepare, bake);
builder->connect(bake, ship);
builder->connect(m_preheatOven, bake);
builder->connect(bake, m_turnOffOven);
// ef expects graph definitions to maintain a ref (i.e. call omni::core::IObject::acquire()) on any
// nodes in the graph definition. an easy way to do this is to keep an array of NodePtr objects in
// the graph definition. NodePtr objects correctly call omni::core::IObject::acquire() and
// omni::core::IObject::release() at the proper times.
m_nodes.emplace_back(std::move(prepare));
m_nodes.emplace_back(std::move(bake));
}
m_nodes.emplace_back(std::move(ship));
}
}
}
private:
BakeryGraphDef(IGraph* graph, const Bakery& bakery) noexcept
: NodeGraphDef(graph, "def.bakery"), m_bakery(bakery)
{
}
const Bakery& m_bakery;
NodePtr m_preheatOven;
NodePtr m_turnOffOven;
std::vector<NodePtr> m_nodes;
};
In Figure 21, you can see:
Preparation of the ingredients can run concurrently with pre-heating the oven.
Once pre-heating and ingredient preparation has completed, the goods are baked in parallel.
Once all goods in an order have baked, they are shipped to the customer.
Once all goods across all orders have finished baking, the oven is turned off. This action can happen in parallel with shipping the goods.
There are two types of definitions seen in the graph: graph definitions and opaque definitions.
Opaque Definitions
Opaque definitions are represented by angled boxes. An example of an opaque definition is the definition used to pre-heat the oven:
class PreHeatOvenNodeDef : public NodeDef
{
public:
static omni::core::ObjectPtr<PreHeatOvenNodeDef> create(const Bakery& bakery) noexcept
{
auto def = omni::core::steal(new PreHeatOvenNodeDef(bakery));
return def;
}
protected:
Status execute_abi(ExecutionTask* info) noexcept override
{
log("pre-heating oven at {}", m_bakery.name);
return Status::eSuccess;
};
SchedulingInfo getSchedulingInfo_abi(const ExecutionTask* info) noexcept override
{
return SchedulingInfo::eParallel;
}
private:
PreHeatOvenNodeDef(const Bakery& bakery) noexcept
: NodeDef("def.oven.preHeat"), m_bakery(bakery)
{ }
const Bakery& m_bakery;
};
The two main methods to overload are INodeDef::execute()
and INodeDef::getSchedulingInfo()
. INodeDef::execute()
performs the definition’s work while INodeDef::getSchedulingInfo()
provides hints to EF about how to schedule the
work.
Because this is an example, the work performed in PreHeatOvenNodeDef is trivial. When performing uncomplicated work,
developers are often better served using NodeDefLambda::create()
which accepts a lambda, a SchedulingInfo
object,
and produces an INodeDef
object. See the bake
node in BakeryGraphDef for an example use of NodeDefLambda
.
Creating an opaque definition by sub-classing from NodeDef
(e.g. PreHeatOvenNodeDef) is useful when:
The developer wishes to provide additional methods on the definition.
The opaque definition needs to store authoring data whose ownership and lifetime can’t be adequately captured in the lambda provided to
NodeDefLambda
.
Graph Definitions
The second type of definitions seen in Figure 21 are graph definitions. Graph definitions
are represented by shaded boxes. Each graph definition has a root node, represented by a circle. In
Figure 21, there is one type of graph definition: PrepareBakedGoodGraphDef
.
Here, you can see the code behind PrepareBakedGoodGraphDef
:
class PrepareBakedGoodGraphDef : public NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IPrivateBakedGoodGetter>
{
public:
static omni::core::ObjectPtr<PrepareBakedGoodGraphDef> create(
omni::core::ObjectParam<IGraphBuilder> builder,
const BakedGood& bakedGood) noexcept
{
auto def = omni::core::steal(new PrepareBakedGoodGraphDef(builder->getGraph(), bakedGood));
def->build(builder);
return def;
}
void build(omni::core::ObjectParam<IGraphBuilder> parentBuilder) noexcept
{
auto builder = GraphBuilder::create(parentBuilder, this);
m_nodes.clear();
auto gather = Node::create(
getTopology(),
NodeDefLambda::create("def.bakedGood.gatherIngredients",
[this](ExecutionTask& info)
{
log("gather ingredients for {}", m_bakedGood.name);
return Status::eSuccess;
},
SchedulingInfo::eParallel
),
carb::fmt::format("node.bakedGood.gather.{}", m_bakedGood.name)
);
auto assemble = Node::create(
getTopology(),
NodeDefLambda::create("def.bakedGood.assemble",
[this](ExecutionTask& info)
{
log("assemble {}", m_bakedGood.name);
return Status::eSuccess;
},
SchedulingInfo::eParallel
),
carb::fmt::format("node.bakedGood.assemble.{}", m_bakedGood.name)
);
builder->connect(getRoot(), gather);
builder->connect(gather, assemble);
m_nodes.emplace_back(std::move(gather));
m_nodes.emplace_back(std::move(assemble));
}
const BakedGood& getBakedGood() const noexcept override
{
return m_bakedGood;
}
private:
using BaseType = NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IPrivateBakedGoodGetter>;
PrepareBakedGoodGraphDef(IGraph* graph, const BakedGood& bakedGood) noexcept
: BaseType(graph, "def.bakedGood.prepare"), m_bakedGood(bakedGood)
{ }
const BakedGood& m_bakedGood;
std::vector<NodePtr> m_nodes;
};
PrepareBakedGoodGraphDef creates a simple graph with two opaque nodes, one which gathers all of the ingredients of the baked good and another which assembles the baked good.
Population Passes
A powerful feature in EF are passes. Passes are user created chunks of code that transform the graph during graph construction. As an example, an oft performed transformation is one in which a generic graph definition is replaced with an optimized user defined graph definition.
Figure 21 shows node.bakedGood.prepare.chickenPotPie has a fairly generic graph
definition. An opportunity exists to replace this definition with one which can prepare ingredients in parallel. To do
this, a population pass
is used:
class PopulateChickenPotPiePass : public omni::graph::exec::unstable::Implements<IPopulatePass>
{
public:
static omni::core::ObjectPtr<PopulateChickenPotPiePass> create(omni::core::ObjectParam<IGraphBuilder> builder) noexcept
{
return omni::core::steal(new PopulateChickenPotPiePass(builder.get()));
}
protected:
PopulateChickenPotPiePass(IGraphBuilder*) noexcept
{
}
void run_abi(IGraphBuilder* builder, INode* node) noexcept override
{
// a common pattern when constructing the execution graph is for population passes to want to access the
// authoring data associated with a defintion. to do this, a custom interface which understands the authoring
// layer's data is used. here we see if the currently attached definition understands retrieving BakedGood
// objects. if so, we get a pointer to the part of the defintion that implements the retrival interface.
auto bakedGoodGetter = omni::graph::exec::unstable::cast<IPrivateBakedGoodGetter>(node->getDef());
if (!bakedGoodGetter)
{
// either the node or def matched the name we're looking for, but the def doesn't implement our private
// interface to access the baked good. so, this isn't a def we can populate. bail.
return; // LCOV_EXCL_LINE
}
// we can now grab the baked good from the definition and use it to create a better definition for chicken pot
// pie
const BakedGood& bakedGood = bakedGoodGetter->getBakedGood();
builder->setNodeGraphDef(
node,
ChickenPotPieGraphDef::create(builder, bakedGood)
);
}
};
Population passes are registered via OMNI_GRAPH_EXEC_REGISTER_POPULATE_PASS()
. The registration processes requires
supplying a name the pass will match. In this example, the name to match when registering PopulateChickenPotPiePass
is “node.bakedGood.prepare.chickenPotPie”.
Passes are run by the application’s PassPipeline
. The default PassPipeline
will match the population pass against
the node’s name. If a node’s name does not match, the PassPipeline
will check if a graph definition is attached to
the node and see if the graph definition’s name matches. For more details on how passes are applied, see
Pass Concepts.
Private Interfaces
Above, the population pass first checks if the given node’s definition implements the IPrivateBakedGoodGetter
interface. That interface is defined as follows:
class IPrivateBakedGoodGetter : public omni::core::Inherits<IBase, OMNI_TYPE_ID("example.IPrivateBakedGoodGetter")>
{
public:
virtual const BakedGood& getBakedGood() const noexcept = 0;
};
IPrivateBakedGoodGetter is an example of a private interface. Private interfaces are often used in graph construction and graph execution to safely access non-public implementation details.
To understand the need of private interfaces, consider what the run_abi()
method is doing in
Listing 44. The purpose of the method is to replace the generic “prepare”
graph definition with a higher-fidelity graph definition specific to the preparation of chicken pot pie. In order to
build that new definition, parameters in the chicken pot pie’s BakedGood
object are needed. Therein lies the
problem: EF has no concept of a “baked good”. The population pass is only given a pointer to an INode
EF interface.
With that pointer, the pass is able to get yet another EF interface, IDef
. Neither of these interfaces have a clue
what a BakedGood
is. So, how does one go about getting a BakedGood
from an INode
?
The answer lies in the type casting mechanism Omniverse Native Interfaces provides. The idea is simple, when creating
a definition (e.g. PrepareBakedGoodGraphDef or ChickenPotPieGraphDef
), do the following:
Store a reference to the baked good on the definition.
In addition to implementing the
INodeGraphDef
interface, also implement the IPrivateBakedGoodGetter private interface.
To demonstrate the latter point, consider the code used to define the ChickenPotPieGraphDef
class used in the
population pass:
class ChickenPotPieGraphDef : public NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IPrivateBakedGoodGetter>
NodeGraphDefT
is an implementation of INodeGraphDef
and INodeGraphDefDebug
. NodeGraphDefT
’s template arguments
allow the developer to specify all of the interfaces the subclass will implement, including interfaces EF has no notion
about (e.g. IPrivateBakedGoodGetter). Recall, much like ChickenPotPieGraphDef
, PrepareBakedGoodGraphDef’s
implementation also inherits from NodeGraphDefT
and specifies IPrivateBakedGoodGetter as a template argument.
To access the BakedGood
from the given INode
, the population pass calls omni::graph::exec::unstable::cast()
on
the node’s definition. If the definition implements IPrivateBakedGoodGetter, a valid pointer is returned, on which
getBakedGood()
can be called. With the BakedGood
in hand, it can be used to create the new
ChickenPotPieGraphDef
graph definition, which the builder attaches to the node, replacing the generic graph
definition.
Why Not Use dynamic_cast?
A valid question that may arise from the code above is, “Why not use dynamic_cast?” There are two things to note about dynamic_cast:
dynamic_cast is not ABI-safe. Said differently, different compilers may choose to implement its ABI differently.
dynamic_cast relies on runtime type information (RTTI). When compiling C++, RTTI is an optional feature that can be disabled.
In Listing 44, run_abi()
’s’ INode
pointer (and its attached IDef
) may
point to an object implemented by a DLL different than the DLL that implements the population pass. That means if
dynamic_cast is called on the pointer, the compiler will assume the pointer utilizes its dynamic_cast ABI contract.
However, since the pointer is from another DLL, possibly compiled with a different ABI and compiler settings, that
assumption may be bad, leading to undefined behavior.
In contrast to dynamic_cast, omni::graph::exec::unstable::cast()
is ABI-safe due to its use of Omniverse Native Interfaces (ONI). ONI defines iron-clad, ABI-safe contracts that work across different compiler tool chains. Calling
omni::graph::exec::unstable::cast()
on an ONI object (e.g. IDef
) has predictable behavior regardless of the compiler
and compiler settings used to compile the object.
Private vs. Public Interfaces
ONI objects are designed to be ABI-safe. However, it is clear that IPrivateBakedGoodGetter is not ABI-safe.
getBakedGood()
returns a reference, which is not allowed by ONI. Furthermore, the returned object, BakedGood
,
also is not ABI-safe due to its use of complex C++ objects like std::string
and std::vector
.
Surprisingly, since IPrivateBakedGoodGetter is a private interface that is defined, implemented, and only used within
a single DLL, the interface can violate ONI’s ABI rules because the ABI will be consistent once
omni::graph::exec::unstable::cast()
returns a valid IPrivateBakedGoodGetter. If IPrivateBakedGoodGetter was able
to be implemented by other DLLs, this scheme would not work due the ambiguities of the C++ ABI across DLL borders.
Private interfaces allow developers to safely access private implementation details (e.g. BakedGood
) as long as
the interfaces are truly private. The example above illustrates a common EF pattern, a library embedding implementation
specific data in either nodes or definitions, and then defining passes which cast the generic EF INode
and IDef
pointers to a private interface to access the private data.
But what if the developer wants to not be limited to accessing this data in a single DLL? What if the developer wants to allow other developers, authoring their own DLLs, to access this data? In the bakery example, such a system would allow anyone to create DLLs that implement passes which can optimize the production of any baked good.
The answer to these questions are public interfaces. Unlike private interfaces, public interfaces are designed to be implemented by many DLLs. As such, public interfaces must abide by ONI’s strict ABI rules.
In the context of the example above, the following public interface can be defined to allow external developers to access baked good information:
class IBakedGood_abi : public omni::core::Inherits<omni::core::IObject, OMNI_TYPE_ID("example.IBakedGood")>
{
protected:
virtual uint32_t getBakeMinutes_abi() noexcept = 0;
virtual const char* getName_abi() noexcept = 0;
};
To utilize the public interface, definitions simply need to inherit from it and implement its methods. For example:
class PrepareBakedGoodGraphDef : public NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IBakedGood>
Using public interfaces is often more work, but unlocks the ability for external developers to improve and extend a libary’s execution graph. When deciding whether to use a public or private interface, consult the following flowchart.
Back to the Population Pass
After PopulateChickenPotPiePass runs and replaces node node.bakedGood.prepare.chickenPotPie’s generic graph
definition with a new ChickenPotPieGraphDef
, the bakery’s definition is as follows:
Above, you can see node.bakedGood.prepare.chickenPotPie now points to a new graph definition which performs task such as preparing the crust and cooking the chicken in parallel.
Populating Graph Definitions
As mentioned earlier, population passes run on either matching node names or matching graph definition names. You are encouraged to inspect the names used in Figure 23. There, node names are fairly specific. For example, node.bakedGood.prepare.chickenPotPie rather than node.bakedGood.prepare. Definitions, on the other hand, are generic. For example, def.bakedGood.prepare instead of def.bakedGood.prepare.applePie. This naming scheme allows for a clever use of the rules for population passes.
Earlier, you saw a population pass that optimized chicken pot pie orders by matching node names. Here, a new pass is created: PopulatePiePass:
class PopulatePiePass : public omni::graph::exec::unstable::Implements<IPopulatePass>
{
public:
static omni::core::ObjectPtr<PopulatePiePass> create(omni::core::ObjectParam<IGraphBuilder> builder) noexcept
{
return omni::core::steal(new PopulatePiePass(builder.get()));
}
protected:
PopulatePiePass(IGraphBuilder*) noexcept
{
}
void run_abi(IGraphBuilder* builder, INode* node) noexcept override
{
auto bakedGoodGetter = omni::graph::exec::unstable::cast<IPrivateBakedGoodGetter>(node->getDef());
if (!bakedGoodGetter)
{
// either the node or def matches the name we're looking for, but the def doesn't implement our private
// interface to access the baked good, so this isn't a def we can populate. bail.
return; // LCOV_EXCL_LINE
}
const BakedGood& bakedGood = bakedGoodGetter->getBakedGood();
// if the baked good ends with "Pie" attach a custom def that knows how to bake pies
if (!omni::extras::endsWith(bakedGood.name, "Pie"))
{
// this baked good isn't a pie. do nothing.
return; // LCOV_EXCL_LINE
}
builder->setNodeGraphDef(
node,
PieGraphDef::create(builder, bakedGood)
);
}
};
PopulatePiePass’s purpose is to better define the process of baking pies. This is achieved by registering
PopulatePiePass with a matching name of “def.bakedGood.prepare”. Any graph definition matching
“def.bakedGood.prepare” with be given to PopulatePiePass. Above, PopulatePiePass’s run_abi()
method first
checks if the currently attached definition can provide the associated BakedGood
. If so, the name of the baked
good is checked. If the name of the baked good ends with “Pie” the node’s definition is replaced with a new
PieGraphDef
, which is a graph definition that better describes the preparation of pies. The resulting bakery graph
definition is as follows:
It is important to note that EF’s default pass pipeline
only matches population passes with either node names or graph
definition names. Opaque definition names are not matched.
The example above shows that knowing the rules of the application’s pass pipeline
can help EF developers name their
nodes and definitions in such as way to make more effective use of passes.
Conclusion
The bakery example is trivial in nature, but shows several of the patterns and concepts found in the wild when using the Execution Framework. An inspection of OmniGraph’s use of EF will reveal the use of all of the patterns outlined above.
A full source listing for the example can be found at source/extensions/omni.graph.exec/tests.cpp/TestBakeryDocs.cpp.