Best methods to categorize calls

Question

I have a dateset of many call types from one species and I would like to categorize the call types.

I know I have a few different options, but statistically I was wondering what would be the best approach. The call types range from broadband stacks to flat tonal whistles. Some call types grade into one another with both ends of the spectrum structurally very different. I am hesitant to group them by eye (even with volunteers blindly categorizing exemplars to ensure that the categories are the same).

However, I am running into the problem that every system/method I have run various measurements through will do good at separating some features, but not all of them, with many calls put in categories that make no sense - e.g., some harmonic stacks being grouped with tonal whistles rather than other stacks that are very similar.

I was wondering if instead of categorizing them all at once using a PCA or something similar, it would make sense to categorize them in batches. For example first separating by peak frequency, then by length, then by presence of harmonic stacks, then by frequency modulation, etc.

Other than that, are there any suggestions of other approaches I could try?

I only have access to a Mac which limits the programs I can use, but do have access to Raven Pro. Any advice would be appreciated.

Answer 1

It sounds like you are describing a decision tree analysis, which is typically constructed in custom code. This can be done in R or Matlab (or Python, really).

A decision tree looks like this, as recently published by Hamilton et al. 2021:

I second that you should look at R packages monitoR and tuneR to extract the features that you need which would feed into your decision tree. R, as it is free and runs on Macs, would be the best way to approach coding your algorithm. Unfortunately, I am not aware of a software that allows you to design a tree without some knowledge of coding.

Reference

Hamilton, R. A., Starkhammar, J., Gazda, S. K., & Connor, R. C. (2021). Separating overlapping echolocation: An updated method for estimating the number of echolocating animals in high background noise levels. The Journal of the Acoustical Society of America, 150(2), 709-717.

Answer 2

I would suggest looking into the R package, monitoR: https://cran.r-project.org/web/packages/monitoR/index.html

The reference manual contains a couple of very useful examples for getting a feel of the package and its functionality, and the audio examples to build correlation templates, etc are built into the package.

In terms of automating the process, I haven't progressed this far with monitoR, though some researchers I know recently published a journal article documenting nest monitoring of critically endangered black cockatoos on Kangaroo Island. The acoustic analysis of species and call recognition/workflow was handled in R and the monitoR package.

Reference

Teixeira, D., Linke, S., Hill, R., Maron, M., & van Rensburg, B. J. (2022). Fledge or fail: Nest monitoring of endangered black-cockatoos using bioacoustics and open-source call recognition. Ecological Informatics, 69, 101656.

Answer 3

I have clustered calls using the function seewave::diffcumspec in R before. It calculates a pairwise difference between cumulative spectra. It is possibly not what you asked for because if still gives you an 'overall clustering' :)

Answer 4

I would also recommend checking out the scikit-maad page - https://scikit-maad.github.io/index.html

They have a very versatile set of tools written in Python for sound inspection, classification and analysis.

Equally noteworthy are the contents posted on GitHub by Emmanuel Dufourq of Stellenbosch University, South Africa.

https://github.com/emmanueldufourq/ISEC2022CEAULML

He provides Python code sets for deep learning analysis of animal sounds, with special focus on birds.

Answer 5

Koe bioacoustics software (koe.io.ac.nz) is free open-source web-based software for classifying animal sounds quickly. You can create interactive ordination plots that allow you to encircle groups of points (acoustic units) and label them in bulk. You can also sort acoustic units by similarity in the unit table and classify them in bulk. Because Koe takes an integrated database approach rather than 'one-file-at-a-time', it has an efficient workflow.

The paper is here: https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13336 and the wiki is here: https://github.com/fzyukio/koe/wiki

Answer 6

Highlighting a few below, but would also definitely recommend checking out Tessa Rhinehart's wonderful list of bioacoustics software for a more comprehensive list - https://github.com/rhine3/bioacoustics-software.

Kaleidoscope Pro (from Wildlife Acoustics) is a desktop app/GUI that has a nice clustering function, but this is not free (though you can get short-term free trials).

Arbimon has a template matching function, which is free & web-based.

In terms of R packages, I second/third the previous mentions of tuneR, monitoR, and seewave. I'd also add gibbonR to that list (uses MFCCs to detect sounds of interest & then has semi-supervised ML functionality as well). As well as ohun, which provides functions to diagnose and optimize detection routines (using energy- or template-based detection). I'd also just note that if you want to go between R & Raven, the Rraven package is helpful.

For Python, Koogu & OpenSoundscape have a number of different ML functions for building detectors.

Answer 7

I am not going into the details of extracting variables. There are several ways to do that. I manually digitised the spectrograms during my PhD as I felt the software accuracy is less and they perform terribly for overlapping calls.

Since you don't know how many call types are in the recording, I would suggest you go for unsupervised clustering. You can follow my methods from (Sadhukhan 2019). Variable extraction > PCA > Clustering > Shilloutte plot for validating number of cluster.

Although I think this method is a little bit primitive as compared to modern advancement, This is a comparatively easy method to follow. I am sure that you will get more advance to classify with ML or ANN.