Read our latest blog piece
Read about why our ‘soft temporal modelling’ design philosophy is key to understanding sounds

Our technology design philosophy focuses on three core outcomes

Accuracy

Our models are designed to produce reliably high true positive rates and low false positive rates. This means our customers can deliver compelling and valuable consumer experiences that are helpful, reliable and irritation-free.

Robustness

Our models are trained and evaluated to be resilient in all target use cases and conditions, and on all target devices. This guarantees a consistent consumer experience irrespective of environmental factors.

Compactness

We design our technology to be as compact as possible. This enables our customers to deliver premium experiences alongside other features and functions; as well as meet privacy expectations by running 100% on the edge.

Three platforms, one outcome: An outstanding consumer experience

Our comprehensive and dedicated ML Pipeline spans our three core data, ML and software platforms:

Alexandria™ Data Platform – includes world-leading facilities, tools and techniques to support the data stages of our ML pipeline from collection and processing to labelling and augmentation.

AuditoryNET™ ML Platform – is dedicated to teaching machines to understand sound and encompasses powerful and versatile tools, frameworks and models – as well as access to the latest Audio Analytic technology and know-how – which are essential to delivering a compact sense of hearing.

ai3™ Software Platform – Our licensable lightweight sound recognition software platform empowers a diverse range of machines with a compact sense of hearing. Its size (from as little as 40KB ROM) and its power requirement (from as little as 1.4mA) means that it is suitable for even the most constrained devices.

Alexandria™ Data Platform

Robust data forms the foundation of good machine learning

To teach our technology to accurately model sounds in the real-world, we have to expose it to data that is representative of the real-world. Quantity matters but to achieve high-performance it is also about diversity, quality of labels and variability.

This is where our Alexandria™ Data Platform is important because it features all the necessary tools to support the data stages of the ML pipeline from collection and processing, to labelling and augmentation.

World-class data collection facilities

We record ‘acoustically-clean’ sounds in our dedicated anechoic Sound Labs, which effectively act as a green screen for audio. In addition, we collect data and recordings from a wide range of real-world environments through our dedicated data collection teams and a global network of volunteers.

Our approach to full data provenance means that we know everything about the data that we use to train and evaluate our models. This eliminates the technical risks, as well as the legal and reputational risks for our customers, as we have permission to use the data commercially.

You can find out why managing the legal aspects of data is so important in our whitepaper ‘Audio for ML: The Law and your Reputation’.

We explore the technical challenges and shortcomings of publicly available audio data in our whitepaper ‘Why Real Sounds Matter for Machine Learning’.

The better the labelling, the better the model

Labelling plays a key part in the augmentation and training phases of our ML pipeline and, as such, is critical to producing accurate and robust sound recognition models.

Our approach to labelling is specific to the challenges presented by sound and is assisted by our dedicated polyphonic labelling tools, which enforce consistency and accuracy, as well as support time-to-market.

We know that various levels of labelling are key to various levels of modelling, so we label simultaneously at the fine and episodic levels in addition to the basic level of weak labels. As a result, we have been able to develop temporal modelling methods which are tolerant to variations in sequence, which have a huge impact on end user experience.

Data that represents all edge-cases

By leveraging our large amounts of quality, well-labelled and diverse data, we can apply our Acoustically Smart Temporal Augmentation (ASTA) techniques to rapidly expand our training and evaluation data. Thus covering virtually every possible combination of environmental factors. This level of coverage would be impossible to fully sample with primary data collection alone.

Through ASTA we can construct acoustically accurate and high-fidelity polyphonic soundscapes that represent a wide range of real-world scenarios with realistic variability. This means that we train and evaluate our models to be robust in the real world. In turn, this means that our customers can trust our technology to deliver exceptional experiences for all of their consumers.

AuditoryNET™ ML Platform

A cutting-edge ML platform that delivers quality experiences

Defining the structure of sounds is challenging for machines because there is a vast array of acoustic and temporal features that compose each sound or acoustic scene. On top of this, the environment in which the sounds occur and additional sounds can interrupt or overlap, creating a lot of variability. As a result, machine learning models need the flexibility to take very diverse sound objects and sequences and translate these into very stable and unique labels.

Our ‘soft temporal modelling’ design philosophy runs right through our ML pipeline but the impact of that philosophy is clear to see in our AuditoryNET™ ML Platform, which includes all of the tools necessary for training and evaluating highly compressed sound recognition models that deliver exceptional consumer experiences.

Acoustic + temporal modelling

Our powerful Temporal Decision Engine transforms a series of acoustic observations into a semantic label that identifies an audio event or acoustic scene, rather than providing acoustic perceptions every few milliseconds. By modelling sounds acoustically and temporally, we optimise for end user experience by helping the system to make sense of the chaotic and unstructured nature of sound.

The patented Temporal Decision Engine plays a central role in the accuracy and robustness of our technology. To underline its significance, it delivers a 20x improvement in classification performance over an acoustic sound recognition model alone.

Guiding optimal outcomes

Our patented loss function frameworks are a key part of our model training arsenal. Rather than relying on a crude loss function like cross-entropy, we developed our own proprietary alternative which was designed around the specific challenges presented by sound.

The loss function frameworks guide model training by defining the ‘correct behaviour’ of the network and measuring how close the network performs against the desired output. This is achieved by factoring in the acoustic and temporal properties of sounds, as well as the tolerance to sequence interruptions, which help the network to find the boundaries between the sound objects in softer ways, instead of imposing hard and precise boundaries.

Click here to read more about the importance of the loss function.

Compression without compromise

A strong focus on compactness throughout the pipeline, as well as powerful ML-based compression tools ensures the gold standard of performance on even the most constrained end devices.

In these two figures, we illustrate the compactness of our technology by comparing the computational load and the network size of one of our models from AuditoryNET™ (recognizing six sound classes) with two network architectures used in the research arena: MobileNet (recognizing six sound classes) and ResNet (recognizing 10 sound classes). Both are familiar to a broad range of people working in machine learning and both selected network architectures are also often used by groups entering the annual academic DCASE Challenge for sound recognition tasks.

Evaluation based on the real-world

Machine learning is about learning patterns. ML engineers need all of the metrics, tools and data to guide and evaluate what the system has learnt. It is a process of train-evaluate-retrain.

Evaluation metrics are crucial to maintaining objectivity. We developed our own called Polyphonic Sound Detection Score (PSDS) which overcomes the significant limitations of generic approaches by redefining system errors, makes the evaluation more robust and expands the evaluation to include the factors which matter to user experience. PSDS was the subject of two published papers and has been adopted as the industry-standard evaluation metric.

Our ability to evaluate performance is also powered by the wealth of metadata at our fingertips – only possible because of our approach to data collection, labelling and augmentation. When evaluating model performance we can get a deep understanding of edge cases. This level of detail can be used to guide further training or direct further targeted data collection campaigns.

Plus, our wealth of representative, unbiased data enables us to prove accuracy and robustness in real-world environments on real devices, reducing the risks for our customers and ensuring that all end users get a consistent first-class experience.