Streamlining production workflows for accessible audio using AI metadata assignment

Advancements in broadcast technology are granting new opportunities to improve listening experiences for consumers. One such advancement is object-based audio and the development of the accessible audio system termed Narrative Importance. This system provides users with an adjustable mix, that boosts sounds which are important to the story, such as dialogue, and attenuates non-essential background sounds, such as crowd chatter. This thesis will target one of the barriers to rolling out the Narrative Importance system - augmented production time due to implementation requirements. The specific focus will be furthering the investigation into whether machine learning can be trained to assign the requisite metadata for Narrative Importance.
A survey is deployed to collect label data for training the machine learning algorithm. Par- ticipants are asked to assign importance data to sounds in a mix for nine scenes. This data is then used to train a mixture model to categorise audio objects into 4 levels of importance. The results show that the method chosen here is not successful in its current form. Training with survey labels proves to be ineffective due to low levels of agreement amongst participants. Training with a single set of labels is shown to give better results.
The question of “What is object-based audio?” was also investigated, partially as a result of differing definitions in the existing literature. A survey of audio research professionals was undertaken. The results show that a robust definition for object-based audio does not exist in writing nor in practice. As a result, a definition is proposed in this thesis.