Filtering out wind from audio recordings

Question

I am currently working on building a deep learning model for snowscooter detections. The model does fairly well at detecting snowscooters but it triggers on wind noises, increasing the rate of false detections. Do you have any experience with this kind of problems and how do you deal with it?

The solutions I was thinking of include:

• Adding more wind noises when training the model so it can properly discriminate between snowscooter noises and wind noises.
• Using a high pass filter for filtering out the wind noises first and then use the model on the audio file
• For each file, compute a "wind noise" (is there any entropy metric for such purpose?) metric and if it is too windy simply don't analyse the specific file
• Use a model for detecting the segments containing wind and delete these segments. Is there any open source model for this?

Answer 1

Perhaps have a look at Ollie Metcalf's hardRain R package. It's designed to identify recordings with rain, I think based on passing an amplitude threshold between certain frequency bands. I'm sure you could modify it to pick out wind recordings instead. I don't know if it would be possible to discriminate wind from snow scooters, but it does have the benefit of being quite straightforward.

Metcalf et al (2020) "hardRain: An R package for quick, automated rainfall detection in ecoacoustic datasets using a threshold-based approach", Ecological Indicators. https://www.sciencedirect.com/science/article/pii/S1470160X19307873?casa_token=L13wgMv0i_cAAAAA:jn0YHr7l8v0VeSTrNSuU-FYe5hW6IGrc89qWPZ8BxyiZglxs64rcLptpC4KdQSzctSKsQ_B7dw

Answer 2

In line with the thinking of looking at the frequency content, you could take that a little further: I assume that wind noise generally has more energy at higher frequencies than snowscooters, thus the spectra should be shifted a bit to higher frequencies (but both probably broadband in nature). If you use a reasonably small FFT bin, your spectrum will be smooth-ish (low frequency resolution). Alternatively you could run an envelope function over a spectrum (e.g. Hilbert transform). Now you can run a correlation test between your recorded spectrum (probe/test in below figure) and some typical wind or scooter envelopes/spectra-shapes. In the below we have identified the noise as a scooter. I'm inclined to think that scooters have some strong tonals in their noise, given the use of rotary components. If true this opens up using kurtosis to have a measure of the "peakedness" of the spectrum (or maybe simply a running mean threshold will do).

Answer 3

I have in my other answer advocated for improving the deep learning model directly. But in case one wishes to use a separate model to filter out wind-noise, here is one approach:

Use a generic audio classifier model to detect wind noise. For example, the AudioSet ontology includes the class Wind, and also the subclass Wind noise (Microphone). There are several pre-trained models available that do AudioSet classification, with pretty good performance overall and are easy to use. Examples include PANNs and YAMNet.

If the performance of those models are not satisfactory out-of-the-box, it is possible to fine-tune them on your data. Use the models to extract a vector embedding from the audio, and then use a small dataset with wind-noise and not. Some 100-1000 labeled samples might be enough. For that approach, one can also use vector embedding mores such as OpenL3.

Answer 4

Adding more wind samples to the "non-scooter" / ambient noise class is probably the best solution (your first suggestion). That will make your existing model strictly better at these cases, without needing other pre- or post-processing steps.

But, it might be difficult to get enough samples to really make the network / training process take those into account. If that is the case, there are some tricks that you can do:

• Use sample weighting during training to make these samples get higher priority. In Tensorflow/Keras this can be done by passing the sample_weight argument to the fit() method.
• Use data-augmentation as a data balancing strategy (upsampling). Creating multiple samples with slight variations for the problematic/challenging cases.
• Use audio-mixing of event sounds (such as the wind noise) onto background samples, at different SNR to create more challenging background samples. Can be done using tools like Scaper.
• If you have a lot of simple/repetitive samples represented in the dataset, consider dropping some of them, to allow more focus on the tricky cases. This is a data balancing strategy based on undersampling.

Note that the particular failure-mode that you have identified (wind noise being used for), might be a symptom of a wider issue. It might be that your model is just be looking at (any kind of) soundlevel variation, or spectrogram variation - and decide that means "snow-scooter".

To diagnose and prevent that, consider including also other "event like" sound into (besides the wind noise) into your dataset. Preferably, things that could plausibly occur in your deployment scenario.