Omniverse Farm Architecture
As a platform for distributed task scheduling and execution, Omniverse Farm embraces the microservice architecture to seamlessly scale workflows.
Motivations
The main motivations for building workflows as microservices are that they:
Promote component reuse, leading to smaller code footprint.
Promote loose coupling between components, facilitating integration and evolution of systems.
Promote collaboration between components, as deploying a new one has limited impact on others.
By moving away from large, monolithic, opaque systems, services offer a more flexible approach to combining and chaining together tasks in order to build larger workflows.
For Omniverse Farm
These are the main drivers for the architecture of Omniverse Farm Queue, Omniverse Farm Agent and even the tasks they execute, as collections of shared features. Some of these include:
Reusing the same logging system, in order to have uniform logs across the system.
Reusing the same metrics collection, in order to instrument end-to-end performance.
Reusing the same data transports, in order for components to support common communication protocols.
etc.
As such, while we may refer to Omniverse Farm Queue and Omniverse Farm Agent as distinct entities, their roles — rather than their design — is what differentiates them.
At a high level, Farm Queue maintains a list of tasks to be executed, which Agents equipped with various capabilities visit in order to query for work to be done.
Briefly, the component architecture of Omniverse Farm can be illustrated as a constellation of shared features (note that a number of services have been voluntarily omitted for clarity):
