I am finally getting into signal identification and would like to test Convolutional Neural Network (CNN). I am working in R and willing to do Python. What packages are you using in those two softwares? and do you know of any grey litterature ressources (classes/tutorial/examples) about CNN that was helpful to you?
As recommended in a comment here I list the species of interest: crickets, birds but I would be interested in your recommendation even if you do work on other groups.
Ketos is a great Python based framework for training and running acoustic deep learning classifiers. It has fantastic documentation and tutorials to get you started and, once you train a model, it will automatically be compatible with PAMGuard software (www.PAMGuard.org) - (means you can share your and run your model without requiring any coding - see this tutorial for more info).
Other frameworks (all in Python);
koogu - a bit more embryonic and less well documented but same idea as Ketos (can have advantages with compatibility with some annotation software).
AnimalSpot - based on PyTorch instead of Google's Tensorflow. Similar to Ketos and has several publications. Not yet released but when it is, it will also be compatible with PAMGuard.
I am sure there are many others...
OpenSoundscape is an AMAZING Python package from Tessa Rhinehart and the Kitzes lab! You can view/filter/manipulate audio & spectrograms, build training datasets, try out pre-trained CNNs, try your own custom CNNs, etc. I really like their documentation, lots of great beginner-friendly tutorials that are accessible to anyone and really go step-by-step through each bit of code.
In terms of learning Python in general, I started with some DataCamp online courses/modules. You get DataCamp free for a few months if you are a student but there's also free content available & free trials, etc. The modules are helpful because it's a bit of people explaining concepts but mostly going through activities, getting code examples and then having to write your own based on said example, etc. There's a coding GUI built into the system that looks a bit like VS Code. And there are lots of modules on deep learning, ML, etc. with PyTorch and TensorFlow.
And I second Jamie's point on koogu & Ketos. I'd also add Soundclim to that list of Python packages.
CNNs are a type of artificial neural network based on image recognition, so you might divide your time into: (1) signal processing and augmentation, where you are going to decide the best way to visualise your signal (i.e., turn your signal into images), and also to increase your library with specific augmentation techniques for sound such as mixing target sounds with noise; and (2) on the framework you are going to use to build your CNNs. In Python, you can use many packages to turn your annotated sounds into images such as Librosa and IPython. These packages also have tools to be used in augmentation procedures (Stowell et al., 2019). Then, you can think of using PyTorch, TensorFlow, Theano, Keras, etc. to build your models and predict your testing (unseen) data.
If you are already familiarised with signal processing, I would recommend starting with FastAI courses, which are free. This is a very integrated package based on PyTorch (Python), which is easy to build your own models then to understand in practice what is going on inside the machine. Thus, it might be useful for you to build customised CNNs based on your data/research question.
Stowell D, Petrusková T, Šálek M, Linhart P (2019). Automatic acoustic identification of individuals in multiple species: improving identification across recording conditions. Journal of The Royal Society Interface 16, 20180940.
I am certainly not an expert but I found this free 'Deep Learning (for Audio) with Python' really useful to understand the implementation of neural networks in python.
https://www.youtube.com/playlist?list=PL-wATfeyAMNrtbkCNsLcpoAyBBRJZVlnf
