Carbonite Streamers

Streamer Overview

The carb::audio::IAudioPlayback interface supports sending its audio to one or more ‘streamer’ objects. These are host app defined objects that can be used to receive and process audio in a custom manner. The specific usage pattern for streamers is undefined and entirely up to the host app. The streamer’s interface is found in carb::audio::Streamer. The interface consists of five functions - two for reference counting management, and three for stream management.

A streamer may be used to perform any operation on an output stream as is needed by the host app. A common example would be to dump the stream to a file. Other uses could be to monitor the stream for volume peaks or overall loudness, consume the stream to submit it to a sound data object as input for further playback either in the current playback context or another context, etc. The host app is entirely responsible for deciding how the data should be processed and what should be done with it.

Every streamer object is reference counted. The streamer will only be destroyed once all acquired references have been released. Once a streamer is set as active on an carb::audio::IAudioPlayback context (with either carb::audio::IAudioPlayback::createContext or carb::audio::IAudioPlayback::setOutput) a reference will be taken internally and any external references can be released if no longer needed. It is the host app’s responsibility to ensure that all of its references to the streamer are released so that the streamer can be properly destroyed once it is no longer needed. The reference count management functions must always be thread safe.

The streamer interface has three data management operations that need to be implemented:

• ‘open’: this reports the proposed data format that will be used by the stream and signals that a new stream is being started. The implementation may request a different data format than what is proposed, but it must be a valid data format. The stream will silently fail if an invalid data format is given. Once the stream has been opened, another open call will not be expected until after a ‘close’ call is performed. The data format for the stream will not change once started and can be cached locally if needed.

• ‘write’: this submits a single buffer of interleaved audio data in the stream’s requested format to be processed by the streamer. The buffer and its size in bytes will be provided. It is up to the streamer’s implementation to properly interpret the data in the buffer according to the data format information given when the stream was opened. The given buffer will not be valid upon return so its contents must be copied if they are to be preserved or queued for later processing.

• ‘close’: this signals that the stream has ended and that any clean up tasks should be performed. These tasks may include cleaning up local working buffers, updating a file header, closing a file that was being written to, etc. Another ‘close’ operation will not be expected until after the next ‘open’ operation succeeds.

When a new stream is opened it is given a data format. This will typically be the output format of the playback context. The streamer should accept this data format if at all possible. The streamer is free to modify the data format and return it. However, if the format is changed, there will be a small performance penalty to convert each buffer to the requested data format. This will often be small, but if multiple streamers are used and each needs to have its buffers converted to a different format, these small performance penalties can add up to something significant enough to cause glitches in the audio output stream on the hardware device.

The streamer must process its write operations as quickly as possible. If the write operation takes too long, it could end up causing an audible glitch in the audio stream. This is mostly a concern when the audio stream is also playing to an actual audio device at the same time. If the streamer is attached to a baking context, it may take as long as needed, but with the caveat that any delay in its execution will also delay the overall audio processing. In general, if a long processing operation is needed during the write operation, the data buffer should be copied to an internal queue and processed on another thread to avoid stalling the audio processing. The ‘write’ operation will generally have strictly less time than the length of the buffer it is given to handle its overall processing information. Depending on the complexity of the audio scene being rendered, this may be as little as a few milliseconds. For example, if the given buffer is 10ms long, the ‘write’ operation may have as much as 9ms to handle its operation.

It is the host app’s responsibility to instantiate the streamer object that will be set as active on a playback context. Any required setup on the streamer object must be done before setting it as active with either carb::audio::IAudioPlayback::createContext or carb::audio::IAudioCapture::setSource. Any number of streamers may be set as active at any given time, but note that all active streamers must still share the same processing time requirements outlined above.

Once set as active, the host app may release its reference to the streamer if further direct interaction is no longer necessary. If needed, the current set of active streamers can be retrieved through the carb::audio::OutputDesc::streamer member of the carb::audio::ContextCaps::output object in the return value of carb::audio::IAudioPlayback::getContextCaps. It is up to the host app to provide a means of identifying its streamers and figuring out a way to safely cast them back to their original types from the active streamers table. Note that if the host app pulls a streamer out of the active streamer table for storage or reuse, it must acquire a reference to it for safety.

Streamer Wrapper Class Overview

Implementing a streamer interface object and attaching state to it can be unfortunately cumbersome. The carb::audio::StreamerWrapper class has been added to simplify this process. The wrapper class can simply be inherited from to provide a basic implementation of a streamer object. This will fully handle all reference counting operations, but leave the three data management operations to be implemented by the host app.

The three data management operations must be implemented by the host app. These are the carb::audio::StreamerWrapper::open(), carb::audio::StreamerWrapper::writeData() and carb::audio::StreamerWrapper::close() functions. The various utility helper functions in 'IAudioUtils.h' may be used to convert time and length values as needed. It is the host app’s responsibility to decide what to do with the sound data format object and the data it receives.

A sample implementation of a fully functional streamer is also available in the carb::audio::OutputStreamer class. This will stream data to a file on disk as it is produced. This is mostly useful for debugging purposes to ensure the processed data not only sounds correct but is also numerically correct, or to just capture the output data to a file for later processing or verification.