Irish environmental monitoring equipment manufacturer Sonitus Systems partnered with Sparkd to design an API which automatically recognises and labels audio clips. Noise levels on construction sites represent a concern for health, safety, and environmental pollution. If you manage a construction site, you must be aware of the noise level regulations you must comply with to prevent complaints, fines, or, even worse, the closure of your site. Most construction companies use noise monitoring devices to measure noise levels and ensure they don’t exceed limits. Still, the big challenge is to understand which on-site activities are causing high noise levels. How can you do that in a quick, efficient and reliable way?
Rethinking noise level monitoring: Where is the noise coming from?
Sonitus manufactures noise level monitoring devices which are installed in construction sites, quarries, factories, smart cities and any sites where there is a need to monitor noise pollution levels. As noise level regulations have become stricter, Sonitus realised that measuring noise levels is no longer enough. There was a need to understand which activities caused high-level noises at any time. Their noise monitoring device already featured an audio-capture function, which enabled it to record an audio clip every time the noise level rose above the limits.
After being collected, the recordings were stored in the cloud. At this stage, the project or factory manager had to listen to the recordings individually and pinpoint where the noise came from. This was the only way to understand which machine or process was causing the high noise level on the site. Needless to say, this was a very time-consuming and expensive task. As the number of recordings increased, the number of man-hours required by this process became enormous. Still, this was a critical job, as the results of this analysis can influence important decisions, like the closure of a building site in case the noise level is too high. So, how could Sonitus help its clients speed up this task while maintaining the highest level of reliability?
At first, their team of engineers tested off-the-shelf solutions. They tried a few online tools which allowed them to upload audio samples, train them and test them. Not only did this approach turn out to be very time-consuming, but these tools were not fit for purpose because they listened out for specific sounds. For example, home security systems listen out only for breaking glasses. In Sonitus’ case, the problem was the opposite. Given the recordings of unidentified sounds, they needed to label (or classify) each sound to understand where it was coming from.
The Design of a New API for Sound Classification
From our very first meeting with Sonitus, it was clear that we needed to build a dataset containing the sounds captured by Sonitus’ devices and create an API which would:
- Work with unlabelled data.
- Label sounds using a large-scale library of known sounds.
- Let Sonitus’ clients classify sounds accurately and automatically to understand which activities cause the noise levels to rise above the limits.
- Segment each audio stream using temporal windows and classify the segment appropriately.
- Be based on a robust model which Sonitus could use in production and the clients could trust to make better and faster decisions (e.g. stop a building site, re-route).
- Run on the cloud.
- Require low power consumption.
The new API for sound classification had to be integrated into Sonitus’ devices. Our first step to assess which Machine Learning solutions better fit the available CPU & RAM budget was to benchmark different solutions (deep vs shallow) directly on Sonitus’ devices. We then studied the trade-off between the accuracy of each solution and computational costs. The result of this study paved the way for the development of the Machine Learning model.
The second step was to build the dataset. We included sounds captured by Sonitus’ devices and pertaining to a specific Use Case which covered most of Sonitus’ customer base, as well as a comprehensive range of background noise. We then took a sample of several thousand sounds and tried to understand which terminology better described them. Eventually, we decided to use taxonomies with which sounds are universally classified in the industry.
Particular attention was paid to creating a dataset with a uniform distribution of samples from different languages and sounds from all the different types of environments.
The third and last step was to use the Sparkd platform to launch an annotation campaign and train and test the classifier to produce “deployment-ready” models embedded into Sonitus’ devices.
We are Helping our Customers Understand their Data
We interviewed Paul McDonald, CEO of Sonitus Systems, to ask for feedback on the final result after the API was implemented into production.
“The product is a huge success with the customers who are facing that problem, who are those guys who need to classify thousand of audio sample files,” says Paul “now they log into the cloud platform, they know the measurement is automated, and they have an automatic dashboard that shows them that 90% of their highest noise level is coming from traffic, for example.”
The tool is now being used by Sonitus’ customers worldwide. The applications of the new API are many and varied, from redesigning hospitals to designing smart cities and predicting maintenance in industrial facilities.
“Being able to tell our customers that we (Sonitus) have a team of AI experts added value to them (the customers) because they are reassured that there are a lot of resources and expertise behind our tools”, Paul continues “, we are helping our customers understand where to focus. They have thousands of data, and we are telling them what data they have to pay more attention to.”
Author Manuela Armini