DeltaWave null-testing audio comparator (beta)

[pkane2001]

16 hours ago, fas42 said:

Not doing much until the next version appears, but came across this artifact - DW appears to be adding ringing to the start of the matched waveforms,

 

DeltaWave v1.0.23, 2019-04-06T12:47:25.9930668+11:00
Reference:  BM,orig.wav[L] 5898049 samples 176400Hz 32bits, stereo, MD5=00
Comparison: BM,1st.wav[L] 6005308 samples 176400Hz 32bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain:False    EQ FFT Size:262144, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Upsample:True, Window:Hann
    Spectrum Window:Blackman, Spectrum Size:524288
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Discarding Reference:  Start=2.3s, End=0s
Discarding Comparison: Start=3.5s, End=0s

Initial peak values Reference: -1.49dB   Comparison: -1.414dB
Initial RMS values Reference: -17.644dB   Comparison: -17.68dB

Null Depth=12.52dB
X-Correlation offset: 6211 samples
Drift computation quality, #1: Excellent (0.23μs)

Final peak values Reference: -1.783dB   Comparison: -1.783dB
Final RMS values Reference: -22.691dB   Comparison: -22.774dB

Gain= 0.0536dB (1.0062x) DC=0 Phase offset=35.207286ms (6210.565 samples)
Difference (rms) = -63.76dB [-65.56dBA]
Correlated Null Depth=65.61dB [61.32dBA]
Clock drift: 0.63 ppm

Files are NOT a bit-perfect match (match=6.33%) at 16 bits
Files are NOT a bit-perfect match (match=0%) at 32 bits
Files match @ 50.0017% when reduced to 11.1 bits

RMS of the difference of spectra: -170.016220084506dB
gn=0.993850366235055, dc=0, dr=6.349E-07, of=6210.5653

DONE!

Signature: dae5f91427d7bcc8ca454fdd09ab2496
Writing difference file to: D:\DELLTOP\My\Docs\Audio\Sound\AS~CA\BM,orig~1st(176);100,HP.wav

 

 

 

 

 

Just realized that some of the ringing is due to FIR filters being used in DW. Audacity, for example, uses IIR filters, such as Butterworth, which produce no ringing in the transition band, but do affect phase in non-linear ways.

 

So, it's a tradeoff: a filter that causes ringing or one that introduces phase distortions. I may have to add the choice of both in settings. Let me think about it. 

[Jud]

5 hours ago, pkane2001 said:

 

Just realized that some of the ringing is due to FIR filters being used in DW. Audacity, for example, uses IIR filters, such as Butterworth, which produce no ringing in the transition band, but do affect phase in non-linear ways.

 

So, it's a tradeoff: a filter that causes ringing or one that introduces phase distortions. I may have to add the choice of both in settings. Let me think about it. 

 

I have no idea how possible this is, but could you add the capability to use a filter of choice?  Might allow people to see the effect of different filter settings in SoX, for example.

[pkane2001]

3 hours ago, Jud said:

 

I have no idea how possible this is, but could you add the capability to use a filter of choice?  Might allow people to see the effect of different filter settings in SoX, for example.

 

Anything's possible  It's a  bit of a juggling act between trying to keep things simple and as automatic as possible, and yet provide lots of options.

 

[pkane2001]

Version 1.0.24 now available

Changes in 1.0.24b

• Added support for IIR and FIR filters in settings
• Improved ringing performance of LP/HP/Notch filters
• Improved drift calculation precision
• Removed the precision setting (always set to 30 now)
• Added time axis display in samples, in addition to second, and microseconds
• Added data caching option to the Manual Adjustment screen to speed up processing
• Fixed clock drift plot to show last calculated drift instead of next to last

[pkane2001]

@fas42:

Version 1.0.24 improves on the drift calculation/correction a bit. With the test files you generate in Audacity with 1 sample removed at 352k, the result is about a 6-7dB better null:

 

 

Also, a slightly less sharp filter helped reduce the ringing at the start of the file. With IIR or FIR filters, this is what the start looks like now with a 100Hz high-pass filter:

 

You can still get some ringing around sharp transitions if you increase the number of taps for FIR or the order of the Butterworth filter. I recommend leaving it at 8192, which is the default.

 

There's still work left to do on drift correction. I'll spend most of my time on that for the next update.

 

[esldude]

Glad to have another update.  I've been recording some files on multiple devices this afternoon.  Typically a one generation copy gets me something like a 55-60 db null depth. Still have to do some evaluation of the latest recordings.  So what is keeping it from being more?  Doesn't seem to be FR.  I wouldn't think distortion is the reason.  So is it not good enough drift correction?  Not good enough level matching?  Phase differences? 

 

Is there a good way to figure this out?   Test signals could do that, but DW doesn't line up well with most test signals.  Best test signal I've come up with, but not written about is a low to high sawtooth sweep at the same time as I'm doing a high to low sawtooth sweep.  

[pkane2001]

5 minutes ago, esldude said:

Glad to have another update.  I've been recording some files on multiple devices this afternoon.  Typically a one generation copy gets me something like a 55-60 db null depth. Still have to do some evaluation of the latest recordings.  So what is keeping it from being more?  Doesn't seem to be FR.  I wouldn't think distortion is the reason.  So is it not good enough drift correction?  Not good enough level matching?  Phase differences? 

 

Is there a good way to figure this out?   Test signals could do that, but DW doesn't line up well with most test signals.  Best test signal I've come up with, but not written about is a low to high sawtooth sweep at the same time as I'm doing a high to low sawtooth sweep.  

 

55-60dB RMS null is actually good for an analog recording compared to digital source. If you check the DAC/ADC loopback archives on DIYaudio, you'll see that most fall below that level. The better DAC/ADC loops I've seen there came to about a 70-75dB null (and I've been checking their numbers with DW  -- DW usually reports numbers that are at least as good, often better than what's reported there).

 

The way to figure out what's happening is to check the various plots and results and to go from there:

• Check that the start/end of the two tracks are not causing a large error -- often ADCs take time to synchronize, so trim a second or two at least from the start of each track
• Start by looking at spectral comparison plots. If you are comparing the original with the first generation, there will be some sort of filter that will cause the null to be worse. Apply a low-pass filter in DW just below where the two spectral plots diverge and see if the null improves
• Check if A-weighted null values are better -- if yes, there may be differences in very low frequencies or high on the audio spectrum
• If RMS null is significantly lower than correlated null, this indicates problems with level matching -- phase alignment is good, but level may be non-linearly related between the two tracks. I'll add a chart to show average gain differences over time which hopefully will illustrate this in more detail
• Check the delta of spectra plot. Does it look linear? Are the areas where spectra diverge significantly? This could point to frequency-related issues. If yes, try engaging non-linear EQ.
• Check the clock drift plot. Zoom-in to the level where the remaining drift is no longer a single line. Does it look mostly linear, or oscillating? What's the magnitude? There may be some clock modulation by noise or signal that causes clock variations between DAC and ADC. The remaining drift will tell you that
• Zoom-in to the same level at the beginning, middle and end of the aligned track waveforms. Do they look equally well-aligned in time? Level? 
• Check phase difference plot. Is it chaotic and with no patterns? Does the error increase towards high frequencies? Or does it look like the plot is bent, like on a curve? This could be indications of non-linear clock differences

Any kind of sweep that changes in frequency with time should be a good one to use with DW. It is a repeating wave at the same frequency over a period of time that will give it trouble. This is only because the drift measurement calculation relies on only one point at each part of the waveform to align best with the other. If there are multiple such points that cannot be distinguished from each other, the drift calculation will fail.

 

 

 

 

[esldude]

5 minutes ago, pkane2001 said:

 

55-60dB RMS null is actually good for an analog recording compared to digital source. If you check the DAC/ADC loopback archives on DIYaudio, you'll see that most fall below that level. The better DAC/ADC loops I've seen there came to about a 70-75dB null (and I've been checking their numbers with DW  -- DW usually reports numbers that are at least as good, often better than what's reported there).

 

The way to figure out what's happening is to check the various plots and results and to go from there:

• Check that the start/end of the two tracks are not causing a large error -- often ADCs take time to synchronize, so trim a second or two at least from the start of each track
• Start by looking at spectral comparison plots. If you are comparing the original with the first generation, there will be some sort of filter that will cause the null to be worse. Apply a low-pass filter in DW just below where the two spectral plots diverge and see if the null improves
• Check if A-weighted null values are better -- if yes, there may be differences in very low frequencies or high on the audio spectrum
• If RMS null is significantly lower than correlated null, this indicates problems with level matching -- phase alignment is good, but level may be non-linearly related between the two tracks. I'll add a chart to show average gain differences over time which hopefully will illustrate this in more detail
• Check the delta of spectra plot. Does it look linear? Are the areas where spectra diverge significantly? This could point to frequency-related issues. If yes, try engaging non-linear EQ.
• Check the clock drift plot. Zoom-in to the level where the remaining drift is no longer a single line. Does it look mostly linear, or oscillating? What's the magnitude? There may be some clock modulation by noise or signal that causes clock variations between DAC and ADC. The remaining drift will tell you that
• Zoom-in to the same level at the beginning, middle and end of the aligned track waveforms. Do they look equally well-aligned in time? Level? 
• Check phase difference plot. Is it chaotic and with no patterns? Does the error increase towards high frequencies? Or does it look like the plot is bent, like on a curve? This could be indications of non-linear clock differences

Any kind of sweep that changes in frequency with time should be a good one to use with DW. It is a repeating wave at the same frequency over a period of time that will give it trouble. This is only because the drift measurement calculation relies on only one point at each part of the waveform to align best with the other. If there are multiple such points that cannot be distinguished from each other, the drift calculation will fail.

 

 

 

 

Yes all good suggestions.  Also why I'm making some recordings with a few different devices to see what the relative strength and differences are.  About the phase plot, what is a reasonable lower limit to place on it.  I can make it so high phase is always a straight line or so low it is chaotic.  How do you know where a good point to choose for that lies?

[pkane2001]

4 minutes ago, esldude said:

Yes all good suggestions.  Also why I'm making some recordings with a few different devices to see what the relative strength and differences are.  About the phase plot, what is a reasonable lower limit to place on it.  I can make it so high phase is always a straight line or so low it is chaotic.  How do you know where a good point to choose for that lies?

 

If you mean the lower phase limit setting, that’s determined by the noise floor of the signal. Set it to be a bit above where you think the noise floor is, as that will eliminate low-level, noise dominated frequencies from appearing on the plot.

[pkane2001]

16 minutes ago, esldude said:

Yes all good suggestions.  Also why I'm making some recordings with a few different devices to see what the relative strength and differences are.  About the phase plot, what is a reasonable lower limit to place on it.  I can make it so high phase is always a straight line or so low it is chaotic.  How do you know where a good point to choose for that lies?

 

One other point: a single null value is really not the best way to judge accuracy. RMS value is designed to be dominated by the larger errors, so a few large spikes in otherwise perfectly matching  waveforms can cause a lower null than you might expect.

[fas42]

Whoops! ... Just started having a look, and on the first match happened to click Refresh all charts - and ended up with Original, X-Correlated and Matched being identical plots ... .

[pkane2001]

3 minutes ago, fas42 said:

Whoops! ... Just started having a look, and on the first match happened to click Refresh all charts - and ended up with Original, X-Correlated and Matched being identical plots ... .

 

That's always been the case. Only the final result is stored in memory, original data and intermediate data are not kept as it would take up too much room. You'll have to re-run the match to get these back.

 

[esldude]

In changing the level of phase in settings, 60, 78 and 96 aren't retained.  It looks blank after resetting.  It does appear the setting is held and used in the software for the next match, but it doesn't show up.  Lower values do show in the box if you reset them. 

[pkane2001]

11 minutes ago, esldude said:

In changing the level of phase in settings, 60, 78 and 96 aren't retained.  It looks blank after resetting.  It does appear the setting is held and used in the software for the next match, but it doesn't show up.  Lower values do show in the box if you reset them. 

 

Hmm. Strange, but should be easy to fix.

 

[esldude]

Alright some worthwhile finding using multiple devices already. 

 

Here is a loopback of my Focusrite 18i20 vs itself a couple minutes later.  As you would expect when you listen to it with 85 db gain, there is nothing except noise.  No music at all.  There is a low howling tone, which is 120 hz and 180 hz hum coming thru.  Hard to hear even it is so low.  The Focusrite Forte gets a similar result without the hum related stuff. 

 

Clock drift looks very, very good too. 

 

The delta spectrum is good and if I switch it to log scale you clearly see the low end peaks are hum related. Though I don't show that here. 

 

But loopbacks with my Zen Tour are good, but not this good compared to itself.  It appears to have a little timing drift.  I don't know if they use separate clocks for ADC or DAC or if the way they are connected causes some issue.  The delta wave listened to is still pretty good, but music is heard drifting in and out in level sometimes almost gone other times not. 

 

 

So is the Antelope device which makes their name with their fantastic clocks not well clocked?  There is some evidence what really happens is level shifting over time.  The difference RMS being lower than the correlated nulls.  

 

I'll need to experiment using the clock from the Antelope as an external clock on one of my other devices and see if it causes the same issue.  That should tell me if it is the clock or level drift.  Sure would be something if the company that hangs its hat on its great clocking has a drifty so so clock. 

 

 

 

 

 

 

[pkane2001]

7 hours ago, esldude said:

Alright some worthwhile finding using multiple devices already. 

 

Here is a loopback of my Focusrite 18i20 vs itself a couple minutes later.  As you would expect when you listen to it with 85 db gain, there is nothing except noise.  No music at all.  There is a low howling tone, which is 120 hz and 180 hz hum coming thru.  Hard to hear even it is so low.  The Focusrite Forte gets a similar result without the hum related stuff. 

 

Clock drift looks very, very good too. 

 

The delta spectrum is good and if I switch it to log scale you clearly see the low end peaks are hum related. Though I don't show that here. 

 

But loopbacks with my Zen Tour are good, but not this good compared to itself.  It appears to have a little timing drift.  I don't know if they use separate clocks for ADC or DAC or if the way they are connected causes some issue.  The delta wave listened to is still pretty good, but music is heard drifting in and out in level sometimes almost gone other times not. 

 

 

So is the Antelope device which makes their name with their fantastic clocks not well clocked?  There is some evidence what really happens is level shifting over time.  The difference RMS being lower than the correlated nulls.  

 

I'll need to experiment using the clock from the Antelope as an external clock on one of my other devices and see if it causes the same issue.  That should tell me if it is the clock or level drift.  Sure would be something if the company that hangs its hat on its great clocking has a drifty so so clock. 

 

 

Check the clock drift plot. If the clocks are not running at the same rate, there will at least be a sloped line in the raw clock drift. Looks like a very small drift is present, and DW doesn’t seem to correct it because it’s so small.

 

Then check what the remaining drift looks like after correction by zooming in on the Y axis. Is there still a line with a slope? Is it more like a wave? A remaining sloped line would indicate that not all the drift was corrected. A wave or other curve would indicate a non-linear relationship between the two clocks — something that DW currently does not attempt to correct.

[pkane2001]

I'm thinking of adding some new metrics to aid in the error analysis... Not sure if it'll confuse more than enlighten, but, here's one. This one shows the number of samples (in %) that are within a certain error band. For example, more than 50% of samples were below 0.0064 (fractional) error.  I can also express the error in dB. Any thoughts/concerns on this particular chart, or how this can be improved? 

 

 

I'm trying to come up with better metrics than just RMS null... Just thinking aloud

[fas42]

Tried a variation of introduced clock drift - simple speed adjustment via Audacity. Not as precise as resampling tricks, but just tried slowing down by 0.001%, and then speeding up the output by the same percentage - in Audacity this didn't create a good null, the amplitude steadily increased, from nothing to about 80dB down at the end of 34 secs.

 

What about DW? 

 

DeltaWave v1.0.24, 2019-04-09T08:19:02.8587766+10:00
Reference:  BM,orig.wav[L] 6299672 samples 176400Hz 32bits, stereo, MD5=00
Comparison: BM,orig,slowed.wav[L] 6299735 samples 176400Hz 32bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain:False    EQ FFT Size:262144, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Upsample:True, Window:Hann
    Spectrum Window:Blackman, Spectrum Size:524288
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Discarding Reference:  Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1.49dB   Comparison: -1.49dB
Initial RMS values Reference: -17.93dB   Comparison: -17.93dB

Null Depth=57.29dB
X-Correlation offset: -5 samples
Drift computation quality, #1: Excellent (0.04μs)

Final peak values Reference: -1.49dB   Comparison: -1.486dB
Final RMS values Reference: -17.93dB   Comparison: -17.93dB

Gain= -0.0041dB (0.9995x) DC=0 Phase offset=0ms (0 samples)
Difference (rms) = -67.16dB [-67.6dBA]
Correlated Null Depth=73.5dB [77.3dBA]
Clock drift: -10 ppm

Files are NOT a bit-perfect match (match=30.62%) at 16 bits
Files are NOT a bit-perfect match (match=0%) at 32 bits
Files match @ 49.9981% when reduced to 14.25 bits

RMS of the difference of spectra: -131.824042551851dB
gn=1.00047575910289, dc=9.10824627407456E-09, dr=-1.00001E-05, of=0

DONE!

Signature: 660fec8ab99c63e896b2fda9520a4fdc

 

Ummm,

 

 

This sounds very nasty ... electronic zzzz'ng, robot music ....

[fas42]

Very telling ... tried manual adjustment of the above, by giving it clean Gain, DC, and Drift numbers: 1.000, 0, and 10 - the Corr Null went from 73 to 102dB.

[fas42]

Hmmm, the end of the Delta of the above, zoomed in,

 

[pkane2001]

25 minutes ago, fas42 said:

Very telling ... tried manual adjustment of the above, by giving it clean Gain, DC, and Drift numbers: 1.000, 0, and 10 - the Corr Null went from 73 to 102dB.

 

Interesting. This implies that the tiny change in drift correction from 10.0001ppm  to 10.0000ppm made a big jump in correlated null. That's about 1 sample delay in 10 billion! I can't imagine this caused such a huge jump. Maybe it's one of the other computed values.

 

Try leaving the drift at whatever fraction DW calculated, but clean-up the fractional part for each of the other parameters, one at a time. This may help pinpoint which part of the matching process is causing such a large jump in the correlated null value. Maybe I need to focus on the precision of the gain/DC calculation, instead.

[fas42]

Done ...

 

[pkane2001]

5 minutes ago, fas42 said:

Done ...

 

 

Thanks, Frank! Am I wrong or does it appear that the fraction in the gain factor makes the largest difference?

 

I think I may know what's causing that: DW is measuring the first quarter or so of the track to determine proper gain match coefficients. Since the delta wave for this particular track shows a growing amplitude error towards the end, DW is not using that part in the computation. This was an optimization to speed up processing, but perhaps I need to rethink that! Probably not an issue with most tracks that have a simple constant amplitude error

 

[fas42]

21 minutes ago, pkane2001 said:

 

Thanks, Frank! Am I wrong or does it appear that the fraction in the gain factor makes the largest difference?

 

Percisely so, .

 

21 minutes ago, pkane2001 said:

 

I think I may know what's causing that: DW is measuring the first quarter or so of the track to determine proper gain match coefficients. Since the delta wave for this particular track shows a growing amplitude error towards the end, DW is not using that part in the computation. This was an optimization to speed up processing, but perhaps I need to rethink that! Probably not an issue with most tracks that have a simple constant amplitude error

 

 

I like your thinkin' ...

[esldude]

17 hours ago, pkane2001 said:

 

Thanks, Frank! Am I wrong or does it appear that the fraction in the gain factor makes the largest difference?

 

I think I may know what's causing that: DW is measuring the first quarter or so of the track to determine proper gain match coefficients. Since the delta wave for this particular track shows a growing amplitude error towards the end, DW is not using that part in the computation. This was an optimization to speed up processing, but perhaps I need to rethink that! Probably not an issue with most tracks that have a simple constant amplitude error

 

I too think the gain matching needs improvement, and should use the whole file.  I've also noticed some tracks have error concentrated either at both ends or toward the middle depending upon the track.  And that stays consistent with different gear in use.  I've one that seems to usually produce a great null at the beginning and then get worse gradually toward the end across multiple pieces of gear.  I need to sit down and go over this again to make sure that is consistent.  It does seem related to gear having a drift in gain levels.  But you only see it once the nulls are pretty good to begin with. 

 

@fas42 you might want to get an earlier version of Audacity.  I forget when it changed.  It was not too long after 2.0 status of Audacity.  2.0.3 maybe? These earlier versions allowed speed changes down to .3 parts per billion and seemed smoother.  These versions also allowed gain changes to something like 11 decimal places.  I need to fire up an old, old laptop I keep such a version on and tell you which one to look for. 

 

Good news is that DW seems to consistently show differences in clocking.  I spent some time yesterday using external clocking.  Some other aspects are puzzling at first glance.  One clock seems to bump the nulls of my Zen tour by over 10 db (some of those are differences of 100 db or lower and null depths of 110 db), and my 18i20 by 6 or so db.  This is comparing consecutive runs to themselves.  But it has an occasional glitch somehow.  So you have to run it a few times and get lucky.  Though you see the glitch when it happens in the difference file and it is very low except when a glitch happens.  Using this clock with the Zen Tour consecutive runs compared leave faint noise, but you can still hear very faintly the music.  You must use about 95 db of gain to hear it however.  My other interfaces you hear only noise, but the noise is higher in level so the same residual of music would be obscured. 

 

While the correlated null depth as well as difference RMS is useful info (and the dbA versions too) I find the best indicator between devices to be difference RMS minus final RMS for the track.  I know the read out has gotten full, but maybe this is worth putting on the results line at the bottom or at least make it an option.