How do I estimate the quantization error in recordings based on bit depth?

I've found myself looking this up about once every few years so thought this would be a good place to pose the question for others (to look up every few years). When considering the precision of broadband recordings, the minimum error is driven by quantization error of the least significant bit. In other words pressure is continuous and the digitized signal over and underestimates the true level based on the bit depth. There is a way to relate the bit depth to error (in dB) but I can never remember what that is.

How does one estimate the quantization error in recordings based on the bit depth?

Thanks!

• I edited the title of your question a bit to make it more detailed/specific and better fit the SE Q/A format. I hope to have maintained the essence of your question, but if not, please comment here or edit back. For more info see: How do I write a good title? Sep 12, 2022 at 15:59
• Works for me. :) Sep 14, 2022 at 22:32

• 1 bit corresponds to an amplitude factor of 2.
• an amplitude factor of 2 is equivalent to (roughly) 6 dB
20*log10(2) = 6.0206 dB


The Least significant bit of

• 16-bit ADC is -90 dB (15 * 6 or 20*log10(2^15) = 90.3)
• 24-bit ADC is -138 dB (23 * 6 or 20*log10(2^15) = 138.5)

The bit numbers 15 or 23, instead of 16 and 24 are to exclude the sign bit from the estimation of the bit depth

when talking about bit depth and realistic noise, one should always keep in mind, that for a typical acquisition chain, the dynamic range (max SNR) is always less than the theoretical bit depth.

Electronic systems with a dynamic range of more than 120 dB are really seldom, systems with 102 dB or less, are more realistic, corresponding to a bit depth of an 18-bit ADC

• Thanks so much! Your book is still my favorite reference. I'll have to get it autographed next time we cross paths and buy you a drink of your choice. Sep 14, 2022 at 22:29

Generally you should be within 0.5LSB of the correct value for each sample. The rms of a +/0.5 rectangular distribution is 1/sqrt(12). So the rms error is 1/sqrt(12)=0.29 times 1LSB.

If you've a very small signal, then this matters more than for a larger signal, however it becomes insignificant very quickly. e.g. for signal heights of 10 LSB, the quantisation error is only 2.9% which is .24dB and likely small compared to any errors on sensor calibration, etc. Further, most signals extend over many samples, so unless you're measuring a constant height signal which gets exactly the same error on every sample, then your overall rms error will be a lot less, probably dividing the 1.sqrt(12) by sqrt(N samples in signal). This will likely render it insignificant even for tiny signals.

Note though that the noise floor of many recording systems is considerably higher than 1LSB, so it's that you really need to be looking at.

• Thanks! I can't accept two answers but this is also very helpful. Mar 3, 2023 at 20:43