When setting up the system gain (IOW, effective sensitivity) I orient myself on the background noise. Analog gain (i.e. gain before ADC) should be set such that background noise is not dominant but still present. This gives you maximal sensitivity for signal detection.
In case you are in a quiet environment and you need amplitude information for classification and expect clipped sound due to close-by sound sources, then you would decrease gain setting, loosing the capability of detecting faint signals.
Using different gain settings in a stereo configuration could make sense do have both requirements, faint detection and amplitude based classification.
If you do spectral based classification you should further consider the FFT processing gain (noise distributed over the bandwidth), e.g. a broadband noise RMS value of 10 bit gives you after a 1024 point FFT a spectral RMS noise level of 1 bit.
In summary, I set my gain such that during analysis I still 'see' the ambient noise.