Overview#

omni.audiorecorder provides a straightforward API for capturing audio, either saving recordings to a file or streaming recorded audio through a callback. This extension supports audio recording in multiple PCM formats (16 bit, 32 bit, and float) and is available for both Python and C++ development. It is designed for applications that require low latency recording — achieving latencies as low as 20ms — and offers helper functions to render audio waveforms.

Concepts#

AudioRecorder Creation: The primary interface is exposed as the AudioRecorder class. An instance must be created before audio data is captured.
Flexible Recording Modes: Depending on your requirements, recording can be directed to a file or a user-provided callback for real-time processing.
PCM Format Options: The API supports multiple PCM data formats. For example, integer formats such as signed 16-bit PCM, and signed 32-bit PCM are supported. 24-bit integer PCM is also supported but will be recorded as a 32-bit per sample buffer where the low byte of each sample is zero. Also supported is 32-bit floating point PCM where each sample value ranges from -1.0 to 1.0.
Waveform Helpers: Helper functions (draw_waveform_from_blob_int16, draw_waveform_from_blob_int32, and draw_waveform_from_blob_float) allow rendering of audio waveforms from raw audio data.

Functionality#

Starting and Stopping Recording: The AudioRecorder includes methods to begin recording with a specified format — using callbacks to process data—and to stop recording, optionally flushing remaining buffered data.
Format Retrieval: Once recording is in progress, you can query the current recording format with a dedicated method.
Creation Helper: The create_audio_recorder API is used to create an AudioRecorder instance.
Low Latency Design: The design is optimized for low latency audio capture, providing the capability to handle real-time applications where timing precision is critical.

Relationships#

Dependencies: This extension depends on several core modules including carb.audio and omni.kit.audiodeviceenum extension. These dependencies ensure seamless integration with the broader Omniverse Kit SDK environment and audio systems.
C++ Integration: The module binds to the C++ omni::audio::IAudioRecorder interface, enabling consistent behavior and performance across both Python and C++ applications.

Considerations#

The waveform rendering helper functions are provided as a temporary solution. Their future availability and location may change.
When selecting a PCM format for callbacks, only 16 bit integer, 32 bit integer, and 32-bit float point PCM sample data is supported. For formats such as 24-bit PCM, it can be recorded as a 32-bit per sample buffer where the low byte of each sample is zero.

This extension serves as a practical tool to integrate audio capture and processing into your applications, with flexibility in output methods and format options for various use cases.