DeltaWave null-testing audio comparator (beta)

[pkane2001]

Just in case you have not figured this out yet — Delta Wave is really an experiment in the study of human brain sleep activity:

 

https://en.m.wikipedia.org/wiki/Delta_wave

 

The software doesn’t exist. You are all in a deep sleep state, with your brain generating a 0.8Hz-4Hz high amplitude signal that is, of course, properly phase- and level-matched.

 

When you wake up, you’ll feel refreshed, but realize that the software was just the product of your vivid dreams.

 

Happy dreaming and happy April 1!

[fas42]

Okay, decided to revisit the checking I was doing before, of checking whether DW could detect known variations - gain and offset are as good as I could ask for; but drift still has problems. I altered clock rate by 1 in 352.8k, and the difference was only 60dB down, and still obviously the track. Then, to test if the upsampling in DW was not good enough, I used the 352.8k version of Marley B, with drift, as the comparison - to force upsampling ...

 

And got this,

 

DeltaWave v1.0.22, 2019-04-02T11:46:01.1227656+11:00
Reference:  BM,44.1,orig,16b.wav[L] 1574918 samples 44100Hz 16bits, stereo, MD5=00
Comparison: BM,352.8,orig,1@352k,16b.wav[L] 12599308 samples 352800Hz 16bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain:False    EQ FFT Size:262144, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Upsample:True, Window:Hann
    Spectrum Window:Blackman, Spectrum Size:524288
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Resampled Reference to 352800Hz
Discarding Reference:  Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1.49dB   Comparison: -1.489dB
Initial RMS values Reference: -17.93dB   Comparison: -17.93dB

Null Depth=59.063dB
X-Correlation offset: 1 samples
Residual drift error too large -- couldn't find a solution

Stopped! Match not found!
Signature: 1a328dc0d1b38fcd464da198006fe206

 

Too large?! ...

 

...

 

 

 

 

[STC]

1 hour ago, fas42 said:

X-Correlation offset: 1 samples

 

Frank, I am trying to understand the software and hopefully you could explain what is the meaning of the quote? Does it tell the two files are delayed by one sample? Are you even comparing two files in the above post?

 

thanks. 

[fas42]

41 minutes ago, STC said:

 

Frank, I am trying to understand the software and hopefully you could explain what is the meaning of the quote? Does it tell the two files are delayed by one sample? Are you even comparing two files in the above post?

 

thanks. 

 

The copy and paste of the results panel shows that DW attempted to match two almost identical copies of a single file, The Bob Marley B track - the key difference was in that in Audacity I used resampling, at the SoX standard, which is as good as anything out there, to shift the clock rate by 1 part in 352,800 - something like 2.6ppm - of a duplicate. This is something that happens virtually every time you go through a DAC, ADC loop; the clocks are slightly different, unless they are deliberately set up to use a single clock. So this test is a good examination of what happens using circuitry, rather than software.

 

The two versions are identical volume, identical offset in time - and you can see from the above that by the end of 34 secs they almost perfectly overlap; just a hint of the blue reference, under the pink compared, is visible - but DW misbehaved.

 

No, they are not delayed by one sample from each other - but DW has chosen to think otherwise; an artifact of its upsampling, most likely. Feeding in 44.1kHz versions, DW gets it mostly right, but still can't achieve a good null.

[STC]

21 minutes ago, fas42 said:

This is something that happens virtually every time you go through a DAC, ADC loop; the clocks are slightly different, unless they are deliberately set up to use a single clock.

 

Thanks for the reply. I am trying to measure actual drift of two identical signal but one is delayed by 60 microseconds. I am trying to see if the output is also at 60microsec. Not sure if DW could do that. 

 

Thanks again. 

[esldude]

1 hour ago, STC said:

 

Thanks for the reply. I am trying to measure actual drift of two identical signal but one is delayed by 60 microseconds. I am trying to see if the output is also at 60microsec. Not sure if DW could do that. 

 

Thanks again. 

It probably could handle that just fine.  You might give some more details of what the signal is. 

[STC]

6 hours ago, esldude said:

It probably could handle that just fine.  You might give some more details of what the signal is. 

 

Thanks Dennis. I am taking a shot , no several shots, in the dark to understand the FR in my application. Not even sure I understand what to ask. 

[pkane2001]

10 hours ago, fas42 said:

Okay, decided to revisit the checking I was doing before, of checking whether DW could detect known variations - gain and offset are as good as I could ask for; but drift still has problems. I altered clock rate by 1 in 352.8k, and the difference was only 60dB down, and still obviously the track. Then, to test if the upsampling in DW was not good enough, I used the 352.8k version of Marley B, with drift, as the comparison - to force upsampling ...

 

And got this,

 

DeltaWave v1.0.22, 2019-04-02T11:46:01.1227656+11:00
Reference:  BM,44.1,orig,16b.wav[L] 1574918 samples 44100Hz 16bits, stereo, MD5=00
Comparison: BM,352.8,orig,1@352k,16b.wav[L] 12599308 samples 352800Hz 16bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain:False    EQ FFT Size:262144, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Upsample:True, Window:Hann
    Spectrum Window:Blackman, Spectrum Size:524288
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Resampled Reference to 352800Hz
Discarding Reference:  Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1.49dB   Comparison: -1.489dB
Initial RMS values Reference: -17.93dB   Comparison: -17.93dB

Null Depth=59.063dB
X-Correlation offset: 1 samples
Residual drift error too large -- couldn't find a solution

Stopped! Match not found!
Signature: 1a328dc0d1b38fcd464da198006fe206

 

Too large?! ...

 

...

 

 

Large residual error for DeltaWave is anything above 1 sample offset anywhere in the waveform. So yes, it was unable to reduce the error below 1 sample. Since you upsampled from 44.1k to 352k, one sample is 1/8 the size of the original sample, so the error is actually less than one sample, but DW doesn't take upsampling into account when computing error values.

 

By the way, I am working on tools to let you quantify the phase error/drift a bit better. I'm adding a quality indicator and a drift error plot (still work in progress):

 

[pkane2001]

44 minutes ago, pkane2001 said:

so the error is actually less than one sample, but DW doesn't take upsampling into account when computing error values.

 

That makes me think I that should compute the error value and thresholds in seconds, rather than in samples. A 1 sample error for 44.1kHz sampling rate is 23μs, which might be audible. A 1 sample error in a 352kHz signal is about 3μs, which is below audibility. Thoughts?

[STC]

2 hours ago, pkane2001 said:

 

That makes me think I that should compute the error value and thresholds in seconds, rather than in samples. A 1 sample error for 44.1kHz sampling rate is 23μs, which might be audible. A 1 sample error in a 352kHz signal is about 3μs, which is below audibility. Thoughts?

 

My view from different POV where I use samples for delay. I can confirm It is audible. That’s the reason I upsample everything to 96kHz. Ideally, the higher the better. My understanding is the SPDIF coding itself contains a delay of one sample in one channel. Not sure how this thing work inside the software itself. 

[Jud]

3 hours ago, pkane2001 said:

 

That makes me think I that should compute the error value and thresholds in seconds, rather than in samples. A 1 sample error for 44.1kHz sampling rate is 23μs, which might be audible. A 1 sample error in a 352kHz signal is about 3μs, which is below audibility. Thoughts?

 

Does it make the thing overly complex  if you provide the option? I can think of reasons for both.

[esldude]

3 hours ago, Jud said:

 

Does it make the thing overly complex  if you provide the option? I can think of reasons for both.

I second this.  It would be nice to have both at various times, but don't want to complicate the software by doing so.  

[esldude]

The software is working great for me right now.  I decided to "attack" it from a different direction.  What are the most exact files I can give it to compare and how good can the null be?  I set up a DAC and ADC separate, and used 60 seconds of music put together in a file with 4 copies of this snippet.  Played and recorded one right after the other with  couple seconds silence between.  If the gear were perfect repeatable even if not perfectly distortion free two such files should get me nulls limited only by the noise floor.  I then did the same thing only in a loopback with my recording ADC/DAC.  Same clock so any clocking variations don't really matter.  I would expect it to get very close to being a very deep null limited by noise.  I chopped the files up and labeled them A thru D afterwards to compare. 

 

Well I can happily report the results with separate DAC and ADC are just as good as loopbacks.  So that is impressive on timing alignment.  They do vary more than I expected.  Correlated nulls run 85 db to 105 db (one pair did manage 121 db).  Listening to the difference file portions are just noise.  But there are places here and there where you hear the music embedded into it or sometimes rather clearly (with 80 db extra gain).  My first thought was timing drift or the timing correction drifting.  But I don't think that is the cause. 

 

I think what I'm seeing is gain drift over time.  It is tedious (but amazing we have this capability) to zoom into peaks and see the matched waveform.  I find a level difference of .0001 to .0003 db on such peaks.  The Reference is sometimes high and the compare file is sometimes high.  It seems to drift around over several seconds.  The timing of such peaks looks to usually be dead on as closely as I can look.  I ran into this a decade ago comparing captures of interconnects.  I didn't have this software (and boy would it have been nice to have), but I found the same issue was limiting the ultimate nulls.  

 

So Paul, am I misinterpreting what I'm seeing?  Is there some reasonably doable way to check for gain drift as well as timing drift?  And yes I know .000x db levels are very tiny.  But they seem to be what is limiting better nulls into complete noise.  And when I am lucky I've gotten a couple 60 second clips that are mostly just noise leftover for nearly half the 60 seconds.  This is impressive performance. 

[pkane2001]

 

1 minute ago, esldude said:

So Paul, am I misinterpreting what I'm seeing?  Is there some reasonably doable way to check for gain drift as well as timing drift?  And yes I know .000x db levels are very tiny.  But they seem to be what is limiting better nulls into complete noise.  And when I am lucky I've gotten a couple 60 second clips that are mostly just noise leftover for nearly half the 60 seconds.  This is impressive performance. 

 

That does sound like something is fluctuating during the playback/recording process. Bias feedback loop, supply voltage, or could even be a phase difference caused by clock irregularities or noise. 

 

I can certainly measure a linear change in gain over time, or even fit a polynomial of different degrees to it. But that wouldn't catch a sudden change somewhere in the middle of the track that reverts back after a bit of time, or say, a periodic modulation of gain.

 

Maybe a better way to visualize it would be to plot an average gain difference over time. This would be similar to the delta waveform track, but with a large-scale running average (or rms) over many samples. This might be a way to highlight any abnormal variations in gain between the tracks... 

[fas42]

Sorry, have to run doing other things this morning. Which means I have not digested the recent thread posts - my concern is to be alble to null out the common content, and at the moment DW can't handle a mathematically 'perfect' drift - so how is it going to handle real world instances of such, which will be in the majority?

[pkane2001]

8 minutes ago, fas42 said:

Sorry, have to run doing other things this morning. Which means I have not digested the recent thread posts - my concern is to be alble to null out the common content, and at the moment DW can't handle a mathematically 'perfect' drift - so how is it going to handle real world instances of such, which will be in the majority?

 

Frank, can you please describe the steps you took to generate mathematically perfect drift?

[fas42]

Just back for a moment - to check whether it is reversible, I imported the 16 bit versions of the prepared files, into Audacity; one the original, one with the 1/352.8k drift. Resampled the orig to 352.8k; changed the project rate to 352801 - note the 1 - and resampled the drift track to that rate; then changed the rate of the drift version to 352.8k in the dropdown menu of the track - which the the actual 'undrifting' operation. Changed the project rate back to 352.8k, inverted the drift, and mixed the orig and drift - got a null of about 90dB, which is fine since I started with 16 bit versions; playing that mix was white noise, no hint of the Marley track.

 

Generating the drift track was doing the reverse of the above.

[pkane2001]

 

18 minutes ago, fas42 said:

Just back for a moment - to check whether it is reversible, I imported the 16 bit versions of the prepared files, into Audacity; one the original, one with the 1/352.8k drift. Resampled the orig to 352.8k; changed the project rate to 352801 - note the 1 - and resampled the drift track to that rate; then changed the rate of the drift version to 352.8k in the dropdown menu of the track - which the the actual 'undrifting' operation. Changed the project rate back to 352.8k, inverted the drift, and mixed the orig and drift - got a null of about 90dB, which is fine since I started with 16 bit versions; playing that mix was white noise, no hint of the Marley track.

 

Generating the drift track was doing the reverse of the above.

 

Frank, what did you to do generate 1/352.8k drift? And what does that mean, exactly?

 

Here's my test. Loaded Bob Marley B track into Audacity. Changed speed by extending it by 1 sample (at original 44.1k rate). Saved as 16 bit file. Ran DeltaWave to analyze. Noticed that there was a filter applied at around 20KHz by Audacity, so used a low-pass filter in DeltaWave at 20KHz. Nearly -96dB rms null, about half of the samples are a perfect match at 16 bits.

 

 

Now let's check the drift value reported. Samples increased by 1 from 1377378. So, the drift should be 1/1377378 * 1e6 = 0.726 or 0.73. Guess what DeltaWave computed? 0.73ppm

 

If I save the file from Audacity as 32-bit floating point, I get this result. All null values have improved, and now bit-perfect match at 70%.

 

So, I'm a bit confused as to what you did to get your results. Can you please explain in detail?

[pkane2001]

37 minutes ago, pkane2001 said:

 

 

Frank, what did you to do generate 1/352.8k drift? And what does that mean, exactly?

 

Here's my test. Loaded Bob Marley B track into Audacity. Changed speed by extending it by 1 sample (at original 44.1k rate). Saved as 16 bit file. Ran DeltaWave to analyze. Noticed that there was a filter applied at around 20KHz by Audacity, so used a low-pass filter in DeltaWave at 20KHz. Nearly -96dB rms null, about half of the samples are a perfect match at 16 bits.

 

 

Now let's check the drift value reported. Samples increased by 1 from 1377378. So, the drift should be 1/1377378 * 1e6 = 0.726 or 0.73. Guess what DeltaWave computed? 0.73ppm

 

If I save the file from Audacity as 32-bit floating point, I get this result. All null values have improved, and now bit-perfect match at 70%.

 

So, I'm a bit confused as to what you did to get your results. Can you please explain in detail?

 

@fas42, next test. Converted the original file to 352.8k in Audacity, saved this as 16bit PCM. Slowed it down by 1 sample (still at 352.8k). Saved that file as 16 bit PCM. Used DeltaWave to compare (again, low pass filter at 20KHz):

 

 

Very nice nulls, 75% bit perfect samples. Drift is shown as 0.09ppm. Let's see if that's right: 1/11019024 * 1e6 = 0.0908 or 0.09ppm. Exactly as reported. Again, working as expected.

 

The only thing I can think of is that you are using the resampler in DeltaWave to match sample rates between 44.1k and 352.8k -- that's not recommended. The resampler is to be used only in the case of an emergency, 

 

I don't have any specs or measurements of its quality or even the algorithm used, it was just something provided by the audio library I'm using. At some point I may have to work on creating a resampler that's of a higher quality, but resampling was never part of the plan for DeltaWave, it was just an experiment.

 

EDIT: Just ran the comparison between 44.1k/16 original and 352.8k/16 bit slowed down by 1 sample version. While the RMS null was lower, percent match remained very high at 80%, and drift rate is still exactly right at 0.09ppm.

 

So yes, the resampler introduces some larger distortions, and while most samples match perfectly, the ones that don't, differ by more than without resampling.

[fas42]

1 hour ago, pkane2001 said:

 

So, I'm a bit confused as to what you did to get your results. Can you please explain in detail?

 

No-one has problems with the concept with upsampling and downsampling - 44.1k to 88.2, or 352.8k, or 176.4k to 44.1k. But resamplers can work at any change of rate, going from 352800 to 352801, or the reverse. All I've done is trigger that processing in Audacity - which is identical to a clock drift - if you force the rate of a track to be other than what it was actually sampled at.

 

No speeding up or slowing down - the precision of that is too poor, in Audacity anyway. Only the resampler is invoked - a simple way of doing this is to have a track at a certain rate, change the project rate to anything and generate a new empty track at that rate. Then mix and render to a new track, say; and the result is the same time length, at the different rate.

 

Playing with track rates and project rates is the most precise way of adjusting the sampling in Audacity - it's how I create absurd upsampling numbers, say 10's of Meg Hz.

[pkane2001]

6 minutes ago, fas42 said:

 

No-one has problems with the concept with upsampling and downsampling - 44.1k to 88.2, or 352.8k, or 176.4k to 44.1k. But resamplers can work at any change of rate, going from 352800 to 352801, or the reverse. All I've done is trigger that processing in Audacity - which is identical to a clock drift - if you force the rate of a track to be other than what it was actually sampled at.

 

No speeding up or slowing down - the precision of that is too poor, in Audacity anyway. Only the resampler is invoked - a simple way of doing this is to have a track at a certain rate, change the project rate to anything and generate a new empty track at that rate. Then mix and render to a new track, say; and the result is the same time length, at the different rate.

 

Playing with track rates and project rates is the most precise way of adjusting the sampling in Audacity - it's how I create absurd upsampling numbers, say 10's of Meg Hz.

 

So we are not doing the same test then.  Sorry, don't have the time to look into why Audacity resampler is producing such poor match, but the speed change works precisely and produces the correct result with DW.

[pkane2001]

20 minutes ago, pkane2001 said:

 

So we are not doing the same test then.  Sorry, don't have the time to look into why Audacity resampler is producing such poor match, but the speed change works precisely and produces the correct result with DW.

 

But now I have no choice but to rewrite that resampler in DW!  

[esldude]

1 hour ago, pkane2001 said:

 

@fas42, next test. Converted the original file to 352.8k in Audacity, saved this as 16bit PCM. Slowed it down by 1 sample (still at 352.8k). Saved that file as 16 bit PCM. Used DeltaWave to compare (again, low pass filter at 20KHz):

 

 

Very nice nulls, 75% bit perfect samples. Drift is shown as 0.09ppm. Let's see if that's right: 1/11019024 * 1e6 = 0.0908 or 0.09ppm. Exactly as reported. Again, working as expected.

 

The only thing I can think of is that you are using the resampler in DeltaWave to match sample rates between 44.1k and 352.8k -- that's not recommended. The resampler is to be used only in the case of an emergency, 

 

I don't have any specs or measure;ments of its quality or even the algorithm used, it was just something provided by the audio library I'm using. At some point I may have to work on creating a resampler that's of a higher quality, but resampling was never part of the plan for DeltaWave, it was just an experiment.

 

EDIT: Just ran the comparison between 44.1k/16 original and 352.8k/16 bit slowed down by 1 sample version. While the RMS null was lower, percent match remained very high at 80%, and drift rate is still exactly right at 0.09ppm.

 

So yes, the resampler introduces some larger distortions, and while most samples match perfectly, the ones that don't, differ by more than without resampling.

I'm not seeing the quality box or the clock drift tab.  Are those upcoming improvements not yet out?  Mine says 1.0.22 like yours.

[pkane2001]

5 minutes ago, esldude said:

I'm not seeing the quality box or the clock drift tab.  Is that upcoming improvements not yet out?  Mine says 1.0.22 like yours.

 

Yep. That's new. Still working on it. Seems like it might be useful.

 

[fas42]

1 hour ago, pkane2001 said:

 

So we are not doing the same test then.  Sorry, don't have the time to look into why Audacity resampler is producing such poor match, but the speed change works precisely and produces the correct result with DW.

 

Paul, you've got it back to front. The resampler, which is using the SoX library is extremely accurate - I've done many tests over the time it's been part of Audacity; and it's always delivered the goods, That is, you resample, and then do the reverse operation - and that produces an excellent null. Speed changes by contrast are poor - attempting to reverse the change delivers poor nulling - the routine used is nowhere near as precise.

 

Since the speed change adds artifacts, you get a so so null in DW; exactly what I would expect.