I have 16 weeks of underwater acoustic recordings (2 recorders, 8 weeks each), and I plan to use an automated detector to detect sperm whale echolocation clicks. I would like to assess the detector performance (precision, recall, false positive rate) but am unsure how much manual checking of the recordings that I need to do to know I have accurately captured the performance.
At the lowest analysis level for this project, I just want to assess presence/absence. But, at a higher level (and thinking back on past work) I'd like to know the answer for this for acoustic density estimation outcomes.
When assessing automated detector performance, how much manual checking of the recordings do I need to do?