What methods exist to automatically define the start and end time of acoustic events?

Question

Event-level classifiers like BANTER incorporate multiple signal types and show promise for species-level classification, but defining the start and end time of acoustic events is not trivial, particularly when starting with an automated detector a with high false positive rate. Expert analysts can manually label events that may include multiple signal types (e.g. clicks, whistles and burst pusles in the case of odontocetes), but what strategies work well to define the start/end of acoustic events?

Answer 1

Generally people seem to tackle this with Convolutional Neural Networks (CNN), and train a labeling model to identify multiple soundclasses within a single detection period. There could be a variety of signal types within a specific soundclass, that variety just has to be reflected in the training data. That soundclass could be as narrow as all the calls of a single bird species, or as broad as all biophony present in the soundscape. I haven't built one of these models myself, but have helped develop the training dataset that our collaborators used. If you're familiar with python, I would explore whether or not there are any audio classification CNN templates that you could train to suit your own needs, incorporating data from your own dataset for training can be pretty important.

The most recent publication on the details of the model made by my collaborators is here: Çoban, E.B., Syed, A.R., Pir, D., Mandel, M.I., 2021. Large Scale Ecoacoustic Monitoring With Small Amounts of Labeled Data, in: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. pp. 1–5.

And then an guide to audio classifiers that might be helpful is here: https://towardsdatascience.com/audio-deep-learning-made-simple-sound-classification-step-by-step-cebc936bbe5

I wish I had a more concrete answer for you, I think a lot of people are working towards making such models more accessible to the general public, so depending on your needs there may already be something that you could plug your data into. We're working on publishing our labeled dataset from arctic soundscapes so that others can use it for their own purposes, but it's not out yet.

Answer 2

I've taken to using a semi-automated process that incorporates a suite of detectors and a grouping feature in PAMGuard. The modularity in connecting clicks and whistles to a single acoustic event is surely essential for a lot of my work, but of course, there is still the need for an acoustician to decide upon those annotations and group those call types accurately.

I agree with Megan that down the line, deep learning and versatile network development sure seems to be a promising route to reducing the experienced acoustician element, but so far it seems like those methods are targeted at single or a couple of species and even if they are broad in nature in terms of species and call type specificity, making those methods applicable to large acoustic datasets or real-time monitoring and available to a larger audience is still under development. Lots of progress though and hopefully with continued efforts this will become more useful in the near future!

Answer 3

I could recommend to look into TADARIDA (https://openresearchsoftware.metajnl.com/articles/10.5334/jors.154/). It was developed for bat signals but seems to be applicable to other groups. If I recall correctly it is using random forest algorithm to classify signals into groups.

Bas, Y., Bas, D. and Julien, J.-F., 2017. Tadarida: A Toolbox for Animal Detection on Acoustic Recordings. Journal of Open Research Software, 5(1), p.6. DOI: http://doi.org/10.5334/jors.154

Answer 4

Avisoft's analysis software (http://www.avisoft.com/) uses an intensity threshold to detect the beginning and end of events, allowing you to vary that threshold and visualize the results to calibrate for your specific species and background.