DeltaWave null-testing audio comparator (beta)

[pkane2001]

4 minutes ago, fas42 said:

 

Paul, you've got it back to front. The resampler, which is using the SoX library is extremely accurate - I've done many tests over the time it's been part of Audacity; and it's always delivered the goods, That is, you resample, and then do the reverse operation - and that produces an excellent null. Speed changes by contrast are poor - attempting to reverse the change delivers poor nulling - the routine used is nowhere near as precise.

 

Since the speed change adds artifacts, you get a so so null in DW; exactly what I would expect.

 

Then please show the results from the speed change with the poor null. I posted my tests using speed change, and I can't see getting a much better null than -105dB with 16 bit data. And to your original point: DW drift calculation was exactly spot-on. I'm still struggling to understand why our results are different, and I have to assume I still don't understand what you are doing in Audacity (or in DW?) 

[fas42]

These posts are being done in the middle of other activities; I'll  have to leave it for some hours now, and come back to more carefully go through everything, to answer your questions, Paul.

[esldude]

I took a file, copied it and then applied some .001 db increase and decreases in volume.  In a 60 second file, I had 15 seconds .001 db low and that many high scattered thru it.  What I'm seeing is a gain variation about a third of this amount.  But the results look very similar.  Difference around -85 db and null depths of 100-105 db.  Listening to the difference wave areas I didn't change are nearly gone, but not gone as they should be, and other areas have more prominent differences remaining.  Much like my real results.  

 

So I do think I'm seeing these very small variance in playback levels over several seconds time.  I don't know how you could fix that.  Seems like it would be an issue like linear drift vs non-linear drift.  Of course such an effect at that level has no bearing on any audible concerns, it just lowers the null depth you can manage with DW. 

[Arpiben]

My suggestion would be to generate an audio file with the type of drifts one wants to correct (linear /quadratic / etc...) with maths tools like Matlab or Scipy. Then it will be easier to evaluate how DW performs. Nothing wrong with Audacity just lacking knowledge about library performances....

 

I am still not yet very clear in which domain DW is evaluating and correcting the clock drifts: Time or Frequency ?

 

 I am guessing that the  purpose of clock drift correction is to take into account timing issues between DACs & ADCs, let's say long  term stability rather than short term (noise). In other words correct the oscillators' clock aging or differential aging ...

 

I am not expecting DW to correct frequency drifts sensu stricto (Drift is due to aging plus changes in the environment and other factors external to the oscillator).

 

Hope the following curves will help illustrating my comments without adding further complexity.

Rgds.

 

 

 

 

[pkane2001]

8 hours ago, esldude said:

I took a file, copied it and then applied some .001 db increase and decreases in volume.  In a 60 second file, I had 15 seconds .001 db low and that many high scattered thru it.  What I'm seeing is a gain variation about a third of this amount.  But the results look very similar.  Difference around -85 db and null depths of 100-105 db.  Listening to the difference wave areas I didn't change are nearly gone, but not gone as they should be, and other areas have more prominent differences remaining.  Much like my real results.  

 

So I do think I'm seeing these very small variance in playback levels over several seconds time.  I don't know how you could fix that.  Seems like it would be an issue like linear drift vs non-linear drift.  Of course such an effect at that level has no bearing on any audible concerns, it just lowers the null depth you can manage with DW. 

 

Hi Dennis,

 

While I can certainly add some correction, as well as measurement, of non-linear gain differences (had it before), I think correcting for them is potentially eliminating some non-trivial differences between the two tracks. I guess if you know that the ADC is causing this, you might want to eliminate it.

 

To correct for ADC imperfections, I tested using a calibration file, either loop-back, an impulse response, or an REW-style sweep. The results were not great, although FR was corrected, nulls usually got worse. I suspect the additional noise and uncertainty in the measurement process just adds more noise into the final DW null calculation. For this reason, I'm much more inclined to just show non-linear gain differences on a chart rather than to try to correct them.

[pkane2001]

1 hour ago, Arpiben said:

I am still not yet very clear in which domain DW is evaluating and correcting the clock drifts: Time or Frequency ?

 

 I am guessing that the  purpose of clock drift correction is to take into account timing issues between DACs & ADCs, let's say long  term stability rather than short term (noise). In other words correct the oscillators' clock aging or differential aging ...

 

I am not expecting DW to correct frequency drifts sensu stricto (Drift is due to aging plus changes in the environment and other factors external to the oscillator).

 

Hi Arpiben, 

 

DW calculates and corrects (only linear) clock differences in the frequency domain, but reports the drift in the time-domain units of samples. I did write a time-domain only drift calculation algorithm, but it produced worse precision than the frequency-domain one.

 

The goal of drift correction is not to fix any problems with a clock, not the temperature-related or age-related drift, but the difference in clock speeds between the DAC and the ADC. Unless a master clock is used for both units, some rate difference is guaranteed to be present, and this is not a difference that I'd like to see when nulling two waveforms.

 

Out of curiosity, where are you pulling these charts from?

[Arpiben]

1 hour ago, pkane2001 said:

 

Hi Arpiben, 

 

DW calculates and corrects (only linear) clock differences in the frequency domain, but reports the drift in the time-domain units of samples. I did write a time-domain only drift calculation algorithm, but it produced worse precision than the frequency-domain one.

 

The goal of drift correction is not to fix any problems with a clock, not the temperature-related or age-related drift, but the difference in clock speeds between the DAC and the ADC. Unless a master clock is used for both units, some rate difference is guaranteed to be present, and this is not a difference that I'd like to see when nulling two waveforms.

 

Out of curiosity, where are you pulling these charts from?

 

Thanks for the provided details, clear.

I used to work with synchronization issues and the charts are taken from some generic papers.

I will post or pm them asap.

 

[JanRSmit]

I compared 2 audio files, one 44k/16 and the other MQA, supposedly the same music. This is what i get. Do not know what this means and how to react. What i want is to see if there is a difference in the 2 encodings of the same music(assuming same source is used).

[pkane2001]

4 minutes ago, JanRSmit said:

I compared 2 audio files, one 44k/16 and the other MQA, supposedly the same music. This is what i get. Do not know what this means and how to react. What i want is to see if there is a difference in the 2 encodings of the same music(assuming same source is used).

 

Looks like the DSF file was not read correctly. It shows a -300dB RMS value which means it consists of only silence. You'll get a much better result if you compare FLAC to FLAC at the same sampling rate.

[JanRSmit]

Ah, too bad. Probably a memory limit.

What i want to see is, if and if so, what the differences are between different encodings of the same music. So i did a 44k/16 against a 352k/24 , and now studying what i got as result, and trying to understand it.

Once i start to understand, i will compare it with listening observations.

[pkane2001]

1 hour ago, JanRSmit said:

Ah, too bad. Probably a memory limit.

What i want to see is, if and if so, what the differences are between different encodings of the same music. So i did a 44k/16 against a 352k/24 , and now studying what i got as result, and trying to understand it.

Once i start to understand, i will compare it with listening observations.

 

Yeah, it looks like the DSF file converted contains over a 100 million samples. You'll run out of memory quickly. You can simply tell DeltaWave to trim the file from either end (or both ends) by specifying number of seconds to remove. Try removing a few minutes at a time, but make sure you do the same for both files, otherwise the misalignment between them will be too great.

 

 

[Arpiben]

6 hours ago, Arpiben said:

 

Thanks for the provided details, clear.

I used to work with synchronization issues and the charts are taken from some generic papers.

I will post or pm them asap.

 

 

Charts taken from the following document:

https://apps.dtic.mil/dtic/tr/fulltext/u2/1038555.pdf

(Not sure if the charts are genuine since you retrieve them in various docs)

 

Introducing Oscillator specs:

http://docplayer.net/34332187-Understanding-oscillator-specs-hugo-fruehauf-fei-zyfer-inc-august-2004.html

https://www.testworld.com/wp-content/uploads/understanding-frequency-accuracy-in-crystal-controlled-instruments.pdf

 

Clock error prediction: 

Don't ask me to help 😉. Just an example of how predictions can be done.

http://perso.utinam.cnrs.fr/~vernotte/publis/metrologia01.pdf

[pkane2001]

A lot of changes in DeltaWave version 1.0.23. Some are experimental, like the quality indicator and drift plot. 

• Added Quality of Fit indicator
• Added support for AIFF file format
• Added Clock Drift plot
• Replaced stock library resamplers with my own FFT versions for better accuracy
• Added axis lock/unlock option for each of the plots

 

Quality of Fit

Quality of fit indicates how large the clock difference remains after drift removal. The values shown on the status bar range from Very Poor to Excellent. The indicator will also degrade if the number of samples is low to indicate that the measurement may not be very accurate. This is a heuristic computation and may need to be fine-tuned.

 

Clock Drift Plot

This plot should help see if there are major errors in the drift computation. It shows two lines, one is the drift before the matching process (raw drift) and the other is what remains of the drift after matching. The values on the Y axis are phase offsets, expressed in samples. The X-axis is the time scale of the two tracks. The best possible result is a completely flat horizontal line positioned at Y value of zero.

 

 

 

Axis Lock

The lock icon (in locked position) synchronizes all the similar plots, so that zooming/panning in this one will affect all the others. If you want to zoom/pan a plot without affecting the others, click the lock icon to unlock it. Any changes on this plot will not be reflected in the others. To re-synchronize the plot with the others, click the lock button again.

[fas42]

Again pushed for time ...

 

Short and sharp: opened 44.1/16 Marley B track, less >20k content; set project rate to 352800, resampled to that rate, go into Rate on the dropdown menu for the track, Other .., enter 352801, change project rate to 44.1k, export to 44.1/16 ... output now has clock drift.

 

In DW get for start and end files,

 

DeltaWave v1.0.22, 2019-04-04T08:33:02.1283451+11:00
Reference:  BM,44.1,orig,16b.wav[L] 1574918 samples 44100Hz 16bits, stereo, MD5=00
Comparison: BM,44.1,orig,drift,16b.wav[L] 1574914 samples 44100Hz 16bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain:False    EQ FFT Size:262144, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Upsample:True, Window:Hann
    Spectrum Window:Blackman, Spectrum Size:524288
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Discarding Reference:  Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1.548dB   Comparison: -1.532dB
Initial RMS values Reference: -17.93dB   Comparison: -17.93dB

Null Depth=33.612dB
X-Correlation offset: 1 samples

Final peak values Reference: -1.548dB   Comparison: -1.547dB
Final RMS values Reference: -17.93dB   Comparison: -17.93dB

Gain= 0dB (1x) DC=0 Phase offset=0.000034ms (0.001 samples)
Difference (rms) = -86.04dB [-86.5dBA]
Correlated Null Depth=94.31dB [81.4dBA]
Clock drift: 2.83 ppm

Files are NOT a bit-perfect match (match=40.61%) at 16 bits
Files match @ 50.0284% when reduced to 15.38 bits

RMS of the difference of spectra: -145.538296193969dB
gn=1.00000261750053, dc=0, dr=2.8344E-06, of=0.00149999999999995

DONE!

Signature: 38446e96886baadcc48a86bbf0068a1c

 

 

This is still the Marley track, with white noise background ...

 

Will try again with 23b version later today.

[pkane2001]

40 minutes ago, fas42 said:

Again pushed for time ...

 

Short and sharp: opened 44.1/16 Marley B track, less >20k content; set project rate to 352800, resampled to that rate, go into Rate on the dropdown menu for the track, Other .., enter 352801, change project rate to 44.1k, export to 44.1/16 ... output now has clock drift.

 

In DW get for start and end files,

 

DeltaWave v1.0.22, 2019-04-04T08:33:02.1283451+11:00
Reference:  BM,44.1,orig,16b.wav[L] 1574918 samples 44100Hz 16bits, stereo, MD5=00
Comparison: BM,44.1,orig,drift,16b.wav[L] 1574914 samples 44100Hz 16bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain:False    EQ FFT Size:262144, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Upsample:True, Window:Hann
    Spectrum Window:Blackman, Spectrum Size:524288
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Discarding Reference:  Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1.548dB   Comparison: -1.532dB
Initial RMS values Reference: -17.93dB   Comparison: -17.93dB

Null Depth=33.612dB
X-Correlation offset: 1 samples

Final peak values Reference: -1.548dB   Comparison: -1.547dB
Final RMS values Reference: -17.93dB   Comparison: -17.93dB

Gain= 0dB (1x) DC=0 Phase offset=0.000034ms (0.001 samples)
Difference (rms) = -86.04dB [-86.5dBA]
Correlated Null Depth=94.31dB [81.4dBA]
Clock drift: 2.83 ppm

Files are NOT a bit-perfect match (match=40.61%) at 16 bits
Files match @ 50.0284% when reduced to 15.38 bits

RMS of the difference of spectra: -145.538296193969dB
gn=1.00000261750053, dc=0, dr=2.8344E-06, of=0.00149999999999995

DONE!

Signature: 38446e96886baadcc48a86bbf0068a1c

 

 

This is still the Marley track, with white noise background ...

 

Will try again with 23b version later today.

 

One thing you should try, Frank. When Audacity resamples, it applies a low-pass filter that tramples on high frequencies above 20k. Here's what that 352801Hz file looks like (in white) compared to original 44.1K version (blue). So it's really important that if you filter one of the files, that you apply the same exact filter to the other, otherwise null will suffer:
 

 

Just apply a Low Pass 20k filter in DeltaWave instead of removing it before, apply it to both waveforms, so the filter is the same.

 

Here's what I get with your test if I compare with resampling DOWN in 1.0.23:

 

And here is what I get if I resample UP:

 

 

[fas42]

We've already been here, Paul ... I've mentioned this very issue on this thread a couple of pages ago - and it's the 'fault' of SoX, not Audacity - using the SoX package gives identical behaviour.

 

Which is why I've chopped off all 20kHz content, I say so in the posts. I've confirmed that doing so gives me true white noise residual in the null, in Audacity. All the recent results with DW have been with 20k content brickwalled out.

[pkane2001]

3 minutes ago, fas42 said:

We've already been here, Paul ... I've mentioned this very issue on this thread a couple of pages ago - and it's the 'fault' of SoX, not Audacity - using the SoX package gives identical behaviour.

 

Which is why I've chopped off all 20kHz content, I say so in the posts. I've confirmed that doing so gives me true white noise residual in the null, in Audacity. All the recent results with DW have been with 20k content brickwalled out.

 

Then I don't see the same results, Frank. Here's the delta waveform, and you saw the nulls I posted earlier. If we are doing the same things, why do you get a different result?

 

[fas42]

Which version of Audacity are you using?

[pkane2001]

3 minutes ago, fas42 said:

Which version of Audacity are you using?

 

2.2.2 on a Mac.

 

[TomCapraro]

531/5000

 

 

 

I tried the DW comparing the analog acquisitions of the output stage of a dac (MSB Analog)
The first acquisition with the dac under the same clock circuit, then entering from the s / pdif and connecting the analog output to the ADC controlled by the same master clock.
The second acquisition connecting a PC to the dac via USB, which uses its own clock and is therefore disconnected from that of the master and the ADC.
Now we see the delta waveform without drift correction and then inserting the drift correction.

It seems very reliable to me in this case.

 

[fas42]

Okay, I'll next try 23b, and see what I get ...

[fas42]

And I get the same Delta Waveform, with "Drift computation quality, #1: Excellent (0μs)"

 

Will have to leave it for now; will be back in a few hours ...

[pkane2001]

4 minutes ago, fas42 said:

And I get the same Delta Waveform, with "Drift computation quality, #1: Excellent (0μs)"

 

Will have to leave it for now; will be back in a few hours ...

 

Ok, let me spend some more time with it, Frank. I need to learn more about what Audacity is actually doing under the covers.

[pkane2001]

52 minutes ago, fas42 said:

And I get the same Delta Waveform, with "Drift computation quality, #1: Excellent (0μs)"

 

Will have to leave it for now; will be back in a few hours ...

 

Frank, what do you do in Audacity to reverse the added drift? Resample back to 352800, change rate to 352799, then switch project to 44100 again and export? Just trying to repeat your steps.

 

[esldude]

15 hours ago, pkane2001 said:

 

Hi Dennis,

 

While I can certainly add some correction, as well as measurement, of non-linear gain differences (had it before), I think correcting for them is potentially eliminating some non-trivial differences between the two tracks. I guess if you know that the ADC is causing this, you might want to eliminate it.

 

To correct for ADC imperfections, I tested using a calibration file, either loop-back, an impulse response, or an REW-style sweep. The results were not great, although FR was corrected, nulls usually got worse. I suspect the additional noise and uncertainty in the measurement process just adds more noise into the final DW null calculation. For this reason, I'm much more inclined to just show non-linear gain differences on a chart rather than to try to correct them.

Yes, I was asking as I didn't know, but rather thought it would end up like non-linear drift where it corrupted results.  So if it is reasonably easy showing non-linear gain or non-linear level over time on a chart like you are now doing for drift would be just fine.  This non-linear gain only becomes of enough difference to matter when null depths are 90 db or better.  

 

You can get to a point where it isn't distortion or noise or FR that is preventing a null of just noise, and most of the difference is this level variance over time.  A chart would let you know that is the remaining issue.  It also appears this level variance can very slightly fool DW into making tiny changes in timing drift that aren't from drift.  I'll add a chart to this is a minute. 

 

Here is a loopback of Bela Fleck 60 seconds taken 2 minutes apart.  Notice the very low scale of drift correction.  

 

After the 20 second or so mark with 85 db playback gain you hear mostly just noise with a few notes buried in the noise where you see the little peaks in this difference waveform.  Oh also it appears whenever 0 ppm drift or negative drift is reported the Fit Quality is blank.