Selene, hi. I don't work in marine acoustics, but the amount of manual checking required to assess precision/recall and enumerate your false positives and negatives is probably less than you think. Chambert et al (2018) https://doi.org/10.1111/2041-210X.12910 stated this:

"When false positives occur, estimator accuracy can be improved when a subset of detections produced by the classification algorithm is post-validated by a human observer. We use simulations to investigate the relationship between accuracy and effort spent on post-validation, and found that very accurate occupancy estimates can be obtained with as little as 1% of data being validated".

For my paper on capercaillie https://doi.org/10.1007/s10336-019-01649-8, I checked 4% of the dataset and got useful outputs.

Hope those two leads are of some use. Carlos

Selene, hi. I don't work in marine acoustics, but the amount of manual checking required to assess precision/recall and enumerate your false positives and negatives is probably less than you think. Chambert et al (2018) https://doi.org/10.1111/2041-210X.12910 stated this:

"When false positives occur, estimator accuracy can be improved when a subset of detections produced by the classification algorithm is post-validated by a human observer. We use simulations to investigate the relationship between accuracy and effort spent on post-validation, and found that very accurate occupancy estimates can be obtained with as little as 1% of data being validated".

For my paper on capercaillie https://doi.org/10.1007/s10336-019-01649-8, I checked 4% of the dataset and got useful outputs.