Improving whistle detection rates in PAMGuard

Question

I'm new to PAMGuard and have been trying to use it to detect whistles (of common & Atlantic white-sided dolphins) and I've been having issues with detection accuracy. I'm running the whistle and moan detector (1024 FFT, 512 hop, Hamming window with 3-24kHz detection freq and other settings at default) and sending the results into a ROCCA module to generate an output .csv file with whistles detected. I'm then comparing this output to manual annotations of the same recordings to calculate rates for detection & false positives.

I can identify 4 potential reasons for a lower than ideal detection accuracy- any insights into solving these issues?

1. Harmonics being counted in our whistle counts- when I lower the upper frequency bound to prevent this, whistle contours that extend above that and come back down get cut off and detected twice.
2. There are a large number of whistle types with discontinuous contours, and each fragment is being counted as a separate whistle.
3. Whistles with low signal-to-noise ratios are not getting picked up.
4. This gets further complicated in some dense sections with >15 whistles/second.

With the size of our dataset, a workflow that does not involve manual boxing of the whistles would be ideal. Any suggestions would be much appreciated. Thanks!

Answer 1

is an accurate whistle count important for your study? If so, the WMD probably isn't the best tool to use (at least on its own). As you have seen, the contours are typically broken into fragments, and harmonics are considered separately. You could try increasing the minimum length and minimum total size parameters, and checking whether this improves correlation with manual counts.

With large datasets, I tend to use the WMD output as an index of whistle activity rather than a whistle count, / unit time. This still involves manual verification of samples to look for common sources of FPs, and sometimes SQL routines to remove WMD contours in narrow bands likely to have been caused by noise. But it can be a useful way to track variation in delphinid activity during baseline surveys.

Answer 2

The problems you have probably can't be solved in PAMGuard. The Whistle and Moan detector is designed to locate sections of recordings that likely contain whistle and therefore minimise the data a human observer has to search through. The detector is based on the following (taken from the PAMGuard Help file):

General Principle of Detection Detection is a multi-stage process, the main steps being

1. Computation of a spectrogram from raw audio data
2. Processing of the spectrogram to remove noise (especially clicks)
3. Thresholding to create a binary map of regions above threshold
4. Connecting regions of the binary map to create sounds
5. Breaking and then rejoining branches of complex regions (for instance, if two whistles cross)

Therefore, harmonics that are high in amplitude (and therefore exceed the detection threshold) will be detected the same as the fundamental frequency. Similarly, if there is a section of the whistle that is low in amplitude, it won't be detected but the section that increases in amplitude will - as there is nothing connecting them (through the detector) they will be counted as two whistles. The detector also sometimes struggles to pick out individual whistles when they overlap as you have said - there isn't anything PAMGuard can do about this (see point 5) and even a human observer would struggle with lots of whistles at once (think about what you're trying to answer - do you need to know the exact number of whistles?). And finally, to solve your low signal-to-noise ratio issue, you'd have to decrease the detection threshold. But, before doing this you should think about why/what you're trying to answer. If you just want to know dolphins are present, picking up the louder whistle should be fine (there'll almost always be ones too quiet to detect, even by a human). If you're wanting to know exactly how many whistles there are, you will need a human observer (but still will probably have some that you're unsure about as they're so quiet). Very quiet whistles could be from very distant animals, so if you're wanting to focus on animals fairly close to your hydrophone, a signal-to-noise threshold might be useful.

So, my best advice it to really think about what you want to know in terms of detecting/counting whistles. There may be a way to solve your problems with Machine Learning, but nothing as readily available and transferable across datasets as PAMGuard. It's a very useful tool and whatever way you analyse data such as this, there will be a need for humans to look at a lot of data to either train an algorithm or verify the findings, and PAMGuard does a very good job at minimising the data a human needs to look at.

Hope this helps!

Answer 3

IMHO, only the first item could be solved by an improved software. I'm not a PAMGuard user, so not sure if that is easy or not and can be solved by parameter settings.

All other observations are to be expected. Especially, low signal-noise-ration and high (overlapping) whistle presence, confuse most algorithms.

I guess, if someone can define exactly what constitutes a whistle and consequently a unique cost function can be defined, then a Machine Learning based approach could be interesting to follow.

Otherwise, welcome to ocean science, where everything is much more complicated than presented in textbook examples.