Understanding Digital Sampling

[MichaelFremer]

I guess I'm not understanding how digital sampling works.

 

Each sample is a measurement of the signal's instantaneous amplitude and assigned the closest value from the quantizing scale, right?

 

 Nyquist says you have to sample at a minimum of twice the rate of your highest frequency, right?

 

So suppose you have a sign wave at a frequency exactly half the sampling rate, which satisfies the Nyquist requirement. That means each wave cycle is sampled twice. It also means every cycle is sampled at the same two points. For example, if the sampler happens to be sampling as the cycle crosses the the X-axis then every sample will be 0 (or whatever value the quantizing scale assigns to that point). So your PCM stream will be 0,0,0,0... That's enough data to specify the frequency, but how do you get the amplitude out of it? A wave at a given frequency passes through 0 twice every cycle, regardless of the amplitude. How does knowing my wave hits 0 at every sample tell me anything about how high the wave peaks in between?

 

[esldude]

Yes you are correct.  A signal that happened to synch up with the zero point at sample times would result in a zero level signal.  While any other timing away from the zero point will let you sample and reconstruct the same signal with correct amplitude.  I've seen it referred to as critical Nyquist frequency.  Nyquist wrote about a bandwidth which can be sampled as sample rate divided by 2.  FS=2B where FS is sampling frequency and B is bandwidth.  Now this assumes perfect band limiting of infinitely long signals.  Later Claude Shannon's papers about what is now called the Shannon-Nyquist theorem specified sample rate had to be greater than the highest frequency sampled.  FS>2B. 

 

So sampled frequencies less than half the sample rate can be reconstructed.  Sampled frequencies greater than half the sample rate will result in aliasing if not filtered out.  Sampled frequencies exactly at the half the sample rate may or may not be reconstructed.  

 

https://courses.engr.illinois.edu/ece110/fa2018/content/courseNotes/files/?samplingAndQuantization#SAQ-NYQ

 

Figures 8 and 9 a bit over halfway down this page deal with the situation you describe. 

 

In real world applications we can't do infinities, but we can get close enough the theorem's still describe a workable system.

The frequencies between full response and frequencies being filtered into the noise floor is called the transition zone.  For 44.1 khz CD that is usually 20 khz for full response transitioning to -96 db or less by 22,050 hz.  So you should be able to fully reconstruct accurately a signal up to 20 khz bandwidth.  

 

You may already know this, but if you had no filters we'd get accurate reconstruction for anything less than 22,050 hz, and for anything over that you get a mirror image of frequencies reflected into the 0 to 22,050 hz band which is called aliasing.  For instance a signal at 26 khz would result in a signal showing up at 18,100 hz while a 30 khz signal would result in a signal at 14,100 hz.  Obviously those are distortions of the band we are trying to record or reproduce.  Which is why an ADC has an anti-alias filter. 

 

[SoundAndMotion]

@esldude I like your answer and the link you provide. Both are clear and informative.

 

I just have one nit to pick. You say (with respect to FS=2B):

6 hours ago, esldude said:

While any other timing away from the zero point will let you sample and reconstruct the same signal with correct amplitude. 

And that is not true. You assume that sin(2πft) and cos(2πft) are the only 2 options. Those are actually rather unlikely, since the actual signal can be any of the infinite number of linear combinations of the two: a*sin(2πft) + b*cos(2πft), with the ratio b/a allowing calculation of the phase (phase=atan(b/a)). But since you don't know the phase, you can't reconstruct the signal at FS=2B ; you know nearly nothing about the amplitude or phase.

 

The only information you get at FS=2B is:
If your values are all zeroes, then either your amplitude is zero (a = b = 0) or the phase is zero (b = b/a = 0)
-otherwise-
Your values are non-zero, which means both amplitude and phase are non-zero (b≠0).

 

Summary: Sample only at FS>2B!

 

[MichaelFremer]

Wow. Thank you so much for those detailed answers! I stumbled into that problem while I was trying to figure out what the sample stream would look like at 0 dBFS. My intuition first told me it would be 32768,-32768,32768,-32768.... A wave nearing 0 dBFS would be 32767,-32767..., etc. Then with more thinking I realized I had that wrong.

 

There could be many, many different sample streams that equal 0 dBFS. Any set of samples that describes a wave that peaks at 32,768 (or whatever the highest value on the quantizing scale is) would qualify even though none of the samples are 32,768 because none of them happen at the instant the wave peaks. So most of the time no human could just glance at a sample stream and, without calculating, see the wave is at 0 dBFS. This also explains how a clipped wave can be described, even though any individual sample word cannot go beyond the peak value. A wave that extends beyond the peak values can still be accurately described by samples taken during the parts of the wave that are within the scale.

 

Do I have that right?

[MichaelFremer]

So I guess a human could look at a sample stream and, seeing many very high numbers, realize immediately that the signal is hot and possibly clipped, but without calculating he could not determine more than that. Right?

[esldude]

13 hours ago, SoundAndMotion said:

@esldude I like your answer and the link you provide. Both are clear and informative.

 

I just have one nit to pick. You say (with respect to FS=2B):

And that is not true. You assume that sin(2πft) and cos(2πft) are the only 2 options. Those are actually rather unlikely, since the actual signal can be any of the infinite number of linear combinations of the two: a*sin(2πft) + b*cos(2πft), with the ratio b/a allowing calculation of the phase (phase=atan(b/a)). But since you don't know the phase, you can't reconstruct the signal at FS=2B ; you know nearly nothing about the amplitude or phase.

 

The only information you get at FS=2B is:
If your values are all zeroes, then either your amplitude is zero (a = b = 0) or the phase is zero (b = b/a = 0)
-otherwise-
Your values are non-zero, which means both amplitude and phase are non-zero (b≠0).

 

Summary: Sample only at FS>2B!

 

Yes you are correct.  I didn't think that one thru. 

 

If we move slightly away from the zero crossing it would reproduce as a small amplitude sine wave.  Move further away and it will reproduce as a larger amplitude sine.  Move 90 degrees away from zero crossing and it would reproduce as the proper amplitude sine.  

 

Due to real world filters we have no choice other than FS>2B anyway.  

 

 

[esldude]

12 hours ago, MichaelFremer said:

So I guess a human could look at a sample stream and, seeing many very high numbers, realize immediately that the signal is hot and possibly clipped, but without calculating he could not determine more than that. Right?

You can have 0 db FS samples that describe a peak above that.  If the following analog stage isn't designed for it you may clip it.  That is called intersample overs.  The peak of the wave is over the peak of the samples.  

 

Benchmark has a paper explaining it. 

 

https://benchmarkmedia.com/blogs/application_notes/intersample-overs-in-cd-recordings

 

 

[MichaelFremer]

Thank you, everyone!

[Speedskater]

Monty over at 'xiph.org' has some excellent videos and blogs on digital subjects.

This link is to just one of several pages:

https://www.xiph.org/video/