Jump to content
IGNORED

DeltaWave null-testing audio comparator (beta)


Recommended Posts

37 minutes ago, Arpiben said:

Hi Paul,

 

Please note that when dealing with pure tones results are varying depending on settings.

 

For pure tones with offset only ( same level) the best accuracy is achieved with:

 

For pure tones with constant Time Interval Error, MATCH will crash DW 1.045.

 

Example:  sin((2*.pi*f0*t)-TIE)  fo=300Hz  TIE=10E-2

 

image.thumb.png.d5b19c86f66b4452d250dc2e619f4608.png

 

At the end for such signals I still prefer using manual corrections in DW 😉

 

 

I tried exactly your example, 300Hz, TIE=10^-2. I don't see a crash. Do you have any other settings enabled, like non-linear EQ? A log will help me to get to the bottom of this.

 

Here's the result when only the Measure Simple Waveforms option is selected. You can see the time offset is still there:

image.thumb.png.7f42e7e948fc305180aa565f0d9951c1.png

 

 

Now, the same with Correct Drift and Gain enabled, as well:

image.thumb.png.f429658c5852a58568d44b2d16851abf.png

 

And phase difference looks like this, with all three options turned on:image.thumb.png.d568bdb51a56f22dc296466d80e875e3.png

 

May need more details to try to reproduce the error you're seeing.

Link to comment
4 minutes ago, TomCapraro said:

Paul, you say you don't use simple alignment with complex signals ... instead I get almost theoretical null results with the use of simple alignment on a pink noise loopback then comparing the digital file with the one acquired in analog.
I believe that we must always try as a valid alternative.

 

It might work, or it might not :) Depends on the complexity of the differences between the two captures. Certainly anything with clock drift errors will not be corrected by the simple waveform method.

Link to comment
33 minutes ago, Arpiben said:

 

Check it for example with @TomCapraro 10kHz  7 samples  file:

 

-> Correct Phase Drift & Measure Simple Waveforms leads to a fake drift and a poor spectrum delta

image.thumb.png.1395d3c3de3b81568ce533018c82984f.png 

 -> ONLY Measure simple Waveforms leads to:

image.thumb.png.d1676c8e1deb178b9102467a67617098.png

 

Crash logs when dealing with TIE:

2019-09-30 00:05:33.0765|DEBUG|Wave.WaveForm|Settings: 
    Gain:False, Remove DC:False
    Non-linear Gain EQ:False    Non-linear Phase EQ: False
    EQ FFT Size:524288, EQ Frequency Cut: 0Hz - 384000Hz, EQ Threshold: -300dB
    Correct Drift:False, Precision:30
    Non-Linear drift Correction:False
    Upsample:False, Window:Hann
    Spectrum Window:Kaiser, Spectrum Size:524288
    Spectrogram Window:Kaiser, Spectrogram Size:65536, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False
    Enable Simple Waveform Measurement: True

2019-09-30 00:05:33.0961|INFO|Wave.WaveForm|Discarding Reference:  Start=0s, End=0s
2019-09-30 00:05:33.0961|INFO|Wave.WaveForm|Discarding Comparison: Start=0s, End=0s
2019-09-30 00:05:33.1579|INFO|Wave.WaveForm|
Initial peak values Reference: 0dB   Comparison: 0dB
2019-09-30 00:05:33.1579|INFO|Wave.WaveForm|Initial RMS values Reference: -3,01dB   Comparison: -3,01dB

2019-09-30 00:05:33.1689|INFO|Wave.WaveForm|Null Depth=62,863dB
2019-09-30 00:05:33.1689|DEBUG|Wave.WaveForm|Progress Updating Charts, , 20%
2019-09-30 00:05:33.2477|DEBUG|Wave.WaveForm|Progress Cross-correlation, , 35%
2019-09-30 00:10:01.9045|INFO|Wave.WaveForm|DeltaWave 1.0.45.0 starting up
2019-09-30 00:10:02.1076|INFO|Wave.WaveForm|Adding driver: [WASAPI]{0.0.0.00000000}.{f8793d52-f284-4752-bb2d-81282ff74d5b} | [WASAPI] Speaker/HP (Realtek High Definition Audio) 48000/32
2019-09-30 00:10:02.1232|INFO|Wave.WaveForm|Current driver: [WASAPI]{0.0.0.00000000}.{f8793d52-f284-4752-bb2d-81282ff74d5b}
2019-09-30 00:10:02.2794|INFO|Wave.WaveForm|Adding driver: [WASAPI]{0.0.0.00000000}.{f8793d52-f284-4752-bb2d-81282ff74d5b} | [WASAPI] Speaker/HP (Realtek High Definition Audio) 48000/32
2019-09-30 00:10:02.2794|INFO|Wave.WaveForm|Current driver: [WASAPI]{0.0.0.00000000}.{f8793d52-f284-4752-bb2d-81282ff74d5b}
2019-09-30 00:10:02.3106|DEBUG|Wave.WaveForm|TaskStart: Interactive=False
2019-09-30 00:10:02.3575|DEBUG|Wave.WaveForm|Settings: 

 

 

 

Interestingly, I don't see any errors and the last entry shows that the program was just starting, not running a comparison? Is that when it crashes? It may be related to some corrupted data from previous runs, if that's the case.

 

EDIT: Just tried Tom's 10KHz file. The main problem is that you don't have the 'Trim' option selected, so the 7 samples  that are different remain in the calculation and cause a large error in RMS calculation. That's normal. Just select the Trim option, and you'll get this:

 

image.thumb.png.d120a212fed2b3237911380231668cf6.png

Link to comment
15 minutes ago, pkane2001 said:

 

Interestingly, I don't see any errors and the last entry shows that the program was just starting, not running a comparison? Is that when it crashes? It may be related to some corrupted data from previous runs, if that's the case.

 

 

It gets stuck at 35% cross correlation and always crashes with or without warning .

i tried several times to retrieve more information from logs but always get what I pasted.

 

Tom''s files:

https://audiophilestyle.com/applications/core/interface/file/attachment.php?id=60724

 

TIE files:

A.wavB.wav

 

 

 

Link to comment
4 minutes ago, Arpiben said:

 

It gets stuck at 35% cross correlation and always crashes with or without warning .

i tried several times to retrieve more information from logs but always get what I pasted.

 

Tom''s files:

https://audiophilestyle.com/applications/core/interface/file/attachment.php?id=60724 29.3 MB · 5 downloads

 

TIE files:

A.wav 4 MB · 1 download B.wav 4 MB · 0 downloads

 

 

 

 

Thanks, Arpiben! I see the same with these two files. What's curious is that they are at 9600Hz sample rate, I think that might have something to do with it. I'll fix it.

Link to comment
58 minutes ago, pkane2001 said:

 

Thanks, Arpiben! I see the same with these two files. What's curious is that they are at 9600Hz sample rate, I think that might have something to do with it. I'll fix it.

 

 

@Arpiben, I updated v1.0.45 to properly handle smaller-sized files. The problem was with the number of samples in the file and the FFT size being the same, causing the FFT hop size to be zero, which in turn, caused an infinite loop. It will now compute the step size correctly.

 

You'll need to uninstall the previous DW version from Windows before installing the new one. The result of Tom's A/B comparison is below:

image.thumb.png.3b85e83b6b6eff29d261d459af99f13e.png

 

And delta Phase:

image.thumb.png.cd9452e3459ae6afe43aab3e832a5922.png

 

Thanks for testing!

 

(By the way, that 'Poor' fit quality indicator above is not because of poor fit but because there are too few samples to compute the proper drift value).

Link to comment
8 hours ago, pkane2001 said:

 

 

@Arpiben, I updated v1.0.45 to properly handle smaller-sized files. The problem was with the number of samples in the file and the FFT size being the same, causing the FFT hop size to be zero, which in turn, caused an infinite loop. It will now compute the step size correctly.

 

You'll need to uninstall the previous DW version from Windows before installing the new one. The result of Tom's A/B comparison is below:

image.thumb.png.3b85e83b6b6eff29d261d459af99f13e.png

 

And delta Phase:

image.thumb.png.cd9452e3459ae6afe43aab3e832a5922.png

 

Thanks for testing!

 

(By the way, that 'Poor' fit quality indicator above is not because of poor fit but because there are too few samples to compute the proper drift value).

 

Thanks Paul for the corrected release 1.0.45.

 

1. FFT size = Number of samples (whatever sampling frequency) corrected issue acknowledged.

 BTW A&B files are from Arpiben & 10 kHz 7 samples (Left vs Right) files are from TomCapraro 😉 

 

2. Please note that I am not expecting DW to find the best automatic strategy for every cases simply because it is not possible. That is the reason I am quite happy with Manual Adjustments when dealing with extreme specific cases.

In my non Audio test cases , I had been testing pure tones with known TIE characteristics -> easy to apply the proper correction.

(ex. parabolic phase curve -> constant frequency drift)

With unknown defects ( majority of cases) it is a compromise and I guess you found the proper one for audio files.

 

I am glad and very grateful that DW can also be used for non audio comparisons.

Sincerely, congratulations Paul for the job you have been doing.

Rgds.

 

Link to comment

Hi Paul,

 

Please do you mind explaining how is Jitter estimated in DW ?

As far as I remember, it is calculated in Time Domain.

 

Now, again dealing with tones I have consistency issues regarding the Jitter values.

Pressing several times Show leads to at least three different values.

You can retrieve the behaviour with my previous A & B files. 

 

Ex:

Jitter values: 37 ns / 58.6 ns / 151.2 ns

All with same full bandwidth phase 0,00214601355966744° reported.

 

Rgds.

 

 

Link to comment
8 hours ago, Arpiben said:

Hi Paul,

 

Please do you mind explaining how is Jitter estimated in DW ?

As far as I remember, it is calculated in Time Domain.

 

Now, again dealing with tones I have consistency issues regarding the Jitter values.

Pressing several times Show leads to at least three different values.

You can retrieve the behaviour with my previous A & B files. 

 

Ex:

Jitter values: 37 ns / 58.6 ns / 151.2 ns

All with same full bandwidth phase 0,00214601355966744° reported.

 

Rgds.

 

 

 

Let me check. Maybe there’s still a place where the memory allocation needs to be corrected that’s used in the jitter calculation.

Link to comment

What do you think a delta/null value is between the original source file and one captured from the output of a DAC? ;)

 

If you correct for amplitude and phase errors before nulling, as you can with DeltaWave, you get the more complex SINAD value for a musical recording. That is, RMS value of all the noise and harmonic distortions with all the fundamental frequencies ‘notched out’.

 

The above from Paul I wanted to continue discussing here. 

 

Also this:

 

Right. The software is not perfect, but a perfect ADC is not needed for a comparison, as long as the same one is used. There’s also a way to eliminate the influence of the ADC in measurements, but let’s continue this in the DeltaWave thread...

 

What do you have in mind in regards to eliminate the influence of the ADC in measurements?

 

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
8 minutes ago, esldude said:

What do you think a delta/null value is between the original source file and one captured from the output of a DAC? ;)

 

If you correct for amplitude and phase errors before nulling, as you can with DeltaWave, you get the more complex SINAD value for a musical recording. That is, RMS value of all the noise and harmonic distortions with all the fundamental frequencies ‘notched out’.

 

The above from Paul I wanted to continue discussing here. 

 

Also this:

 

Right. The software is not perfect, but a perfect ADC is not needed for a comparison, as long as the same one is used. There’s also a way to eliminate the influence of the ADC in measurements, but let’s continue this in the DeltaWave thread...

 

What do you have in mind in regards to eliminate the influence of the ADC in measurements?

 

 

The simplest way to eliminate the influence of an ADC on frequency response is to record the source twice, inverting ADC phase between the captures. Then add the two captures and divide by two. This does have the unfortunate effect of reducing noise by √2, but harmonic distortions should stay constant.

Link to comment

Some observations about good results.  

 

Looking at the Diffmaker loop results at Gearslutz, my Zen Tour is currently 12th in a very long list of gear.  Yet it has a difference null of a bit more than -56 db.  

 

Unsynchronized  DAC and ADC, the March Audio Dac and my Zen Tour sit currently 3rd in a large list at about the same number.  This is a list nearly 9 years in the making. 

 

The best numbers of all are all good DACs feeding good ADCs where the DAC also provides the master clock. Top of the list is a Forsell MDAC feeding a Focusrite Blue Mastering 245 ADC.  This Focusrite is not your regular little interface it is a pro quality device.  The best of those get -81 db for the difference null.  Right behind it fractionally is an SPL Madison DAC feeding an evaluation board TI ADC.  

 

If I invoke Level EQ and Phase EQ those top two only improve to about -90 db.  If I do the same with my Zen Tour loopback and my March-Zen Tour they improve much more to the -86 db range.  So it would appear phase and FR are the main differences.  It looks like viewing the graphs in Deltawave the very low end is the biggest difference.  The top ones are flatter down to a 1hz or less, while others are starting to droop around 3 or 5 hz.  

 

Surprisingly the RME ADI is a couple db lower in results than my Zen Tour despite boasting better individual measurements of the ADC and DAC. It too improves to around -84 db difference if Level EQ and Phase EQ are used.  So is my Zen Tour really better than the RME all things considered?  Yeah, I know a couple db isn't much.

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
1 hour ago, esldude said:

Some observations about good results.  

 

Looking at the Diffmaker loop results at Gearslutz, my Zen Tour is currently 12th in a very long list of gear.  Yet it has a difference null of a bit more than -56 db.  

 

Unsynchronized  DAC and ADC, the March Audio Dac and my Zen Tour sit currently 3rd in a large list at about the same number.  This is a list nearly 9 years in the making. 

 

The best numbers of all are all good DACs feeding good ADCs where the DAC also provides the master clock. Top of the list is a Forsell MDAC feeding a Focusrite Blue Mastering 245 ADC.  This Focusrite is not your regular little interface it is a pro quality device.  The best of those get -81 db for the difference null.  Right behind it fractionally is an SPL Madison DAC feeding an evaluation board TI ADC.  

 

If I invoke Level EQ and Phase EQ those top two only improve to about -90 db.  If I do the same with my Zen Tour loopback and my March-Zen Tour they improve much more to the -86 db range.  So it would appear phase and FR are the main differences.  It looks like viewing the graphs in Deltawave the very low end is the biggest difference.  The top ones are flatter down to a 1hz or less, while others are starting to droop around 3 or 5 hz.  

 

Surprisingly the RME ADI is a couple db lower in results than my Zen Tour despite boasting better individual measurements of the ADC and DAC. It too improves to around -84 db difference if Level EQ and Phase EQ are used.  So is my Zen Tour really better than the RME all things considered?  Yeah, I know a couple db isn't much.

 

How are the DAC-ADC loop(s) performed ( internal/external cable/etc) ?

 

Link to comment
6 hours ago, esldude said:

Some observations about good results.  

 

Looking at the Diffmaker loop results at Gearslutz, my Zen Tour is currently 12th in a very long list of gear.  Yet it has a difference null of a bit more than -56 db.  

 

Unsynchronized  DAC and ADC, the March Audio Dac and my Zen Tour sit currently 3rd in a large list at about the same number.  This is a list nearly 9 years in the making. 

 

The best numbers of all are all good DACs feeding good ADCs where the DAC also provides the master clock. Top of the list is a Forsell MDAC feeding a Focusrite Blue Mastering 245 ADC.  This Focusrite is not your regular little interface it is a pro quality device.  The best of those get -81 db for the difference null.  Right behind it fractionally is an SPL Madison DAC feeding an evaluation board TI ADC.  

 

If I invoke Level EQ and Phase EQ those top two only improve to about -90 db.  If I do the same with my Zen Tour loopback and my March-Zen Tour they improve much more to the -86 db range.  So it would appear phase and FR are the main differences.  It looks like viewing the graphs in Deltawave the very low end is the biggest difference.  The top ones are flatter down to a 1hz or less, while others are starting to droop around 3 or 5 hz.  

 

Surprisingly the RME ADI is a couple db lower in results than my Zen Tour despite boasting better individual measurements of the ADC and DAC. It too improves to around -84 db difference if Level EQ and Phase EQ are used.  So is my Zen Tour really better than the RME all things considered?  Yeah, I know a couple db isn't much.

 

The differences shown in the Diffmaker list are inclusive of phase and amplitude filter errors. DeltaWave removes these with nonlinear-EQ engaged, so you get a cleaner looking result. It doesn't mean that the phase and filter errors are not important or even audible, it just means that DiffMaker clumps all the errors together in one number, while DeltaWave allows you to separate them out. If you subtract the RMS error with and without nonlinear EQ in DW, you'll get the degree of influence of the filter phase/amplitude errors on the result.

 

In effect, the -84dB difference compared to a -90dB difference with nonlinear EQ means that the SINAD of one device is better by 4dB when measured with a complex music stimulus, excluding non-linear phase and amplitude effects of the device. Again, that's just one measure that's better, not including the effects of digital/analog filters, which may make it measure and/or sound worse than the other device.

 

Link to comment
8 hours ago, pkane2001 said:

 

The differences shown in the Diffmaker list are inclusive of phase and amplitude filter errors. DeltaWave removes these with nonlinear-EQ engaged, so you get a cleaner looking result. It doesn't mean that the phase and filter errors are not important or even audible, it just means that DiffMaker clumps all the errors together in one number, while DeltaWave allows you to separate them out. If you subtract the RMS error with and without nonlinear EQ in DW, you'll get the degree of influence of the filter phase/amplitude errors on the result.

 

In effect, the -84dB difference compared to a -90dB difference with nonlinear EQ means that the SINAD of one device is better by 4dB when measured with a complex music stimulus, excluding non-linear phase and amplitude effects of the device. Again, that's just one measure that's better, not including the effects of digital/analog filters, which may make it measure and/or sound worse than the other device.

 

I think I understood as you intended.  Once you've shown a difference the next question is what is causing the difference.  Your other choices for EQ, Phase and non-linear timing let us see where some differences are.  It appears most of them are FR and phase. Which isn't surprising.  Though still nice to confirm with some rigor. 

 

One interesting thing your software made easy to see.  Comparing a recording done now and a few minutes from now with nothing changed the nulls are of course very good.  Most of the remaining difference is 60 hz hum.  Everything in the process is the same, but I can't synchonize the 60 hz hum (even if below 100 dbFS) from one run to the next.  So sometimes it is closer in phase on 60 hz than other runs.  Which is visible in the Difference spectrum plot, and alters the already low numbers a few db.  Then you can of course notch out 60 hz.  

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
22 hours ago, pkane2001 said:

 

The simplest way to eliminate the influence of an ADC on frequency response is to record the source twice, inverting ADC phase between the captures. Then add the two captures and divide by two. This does have the unfortunate effect of reducing noise by √2, but harmonic distortions should stay constant.

Hi Paul, can you explain better what you mean by this process? maybe it's something I do as well but ... when you talk about inverting the ADC add and then split I can't translate it

Link to comment
2 hours ago, TomCapraro said:

Hi Paul, can you explain better what you mean by this process? maybe it's something I do as well but ... when you talk about inverting the ADC add and then split I can't translate it

Yes, I meant to ask about this too, but sidetracked myself. 

 

Actually wouldn't these signals council out?  I've got a few of those where I recorded everything twice.  Wouldn't this be the same as inverting the second one and mixing together or am I missing something.  I also am not sure this separates the effects of DAC and ADC if you do this. 

 

I did think about taking the difference signal, inverting it and adding to the original.  Run the original plus inverted difference back thru, and compare this one to the unaltered original.  But if that worked perfectly, I still just get the original file.  

 

 

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
3 hours ago, TomCapraro said:

Hi Paul, can you explain better what you mean by this process? maybe it's something I do as well but ... when you talk about inverting the ADC add and then split I can't translate it

 

The idea was to capture the same DAC signal twice, but invert the polarity (absolute phase) of the the input to ADC for one of the recordings. Then, adding the two captures together while flipping the polarity of one of them will produce twice the signal from the DAC and a cancelled signal from the ADC. Thinking it through, doesn't seem like this will work O.o

Link to comment
8 hours ago, esldude said:

I think I understood as you intended.  Once you've shown a difference the next question is what is causing the difference.  Your other choices for EQ, Phase and non-linear timing let us see where some differences are.  It appears most of them are FR and phase. Which isn't surprising.  Though still nice to confirm with some rigor. 

 

One interesting thing your software made easy to see.  Comparing a recording done now and a few minutes from now with nothing changed the nulls are of course very good.  Most of the remaining difference is 60 hz hum.  Everything in the process is the same, but I can't synchonize the 60 hz hum (even if below 100 dbFS) from one run to the next.  So sometimes it is closer in phase on 60 hz than other runs.  Which is visible in the Difference spectrum plot, and alters the already low numbers a few db.  Then you can of course notch out 60 hz.  

 

You may simultaneously record the AC hum and then substract it after alignment.

Well I didn't take yet my first coffee cup....😉

 

Link to comment
On 10/1/2019 at 8:37 AM, Arpiben said:

Hi Paul,

 

Please do you mind explaining how is Jitter estimated in DW ?

As far as I remember, it is calculated in Time Domain.

 

Now, again dealing with tones I have consistency issues regarding the Jitter values.

Pressing several times Show leads to at least three different values.

You can retrieve the behaviour with my previous A & B files. 

 

Ex:

Jitter values: 37 ns / 58.6 ns / 151.2 ns

All with same full bandwidth phase 0,00214601355966744° reported.

 

Rgds.

 

 

 

Hi Arpiben, 

 

I'm trying to reproduce this problem. Did you use Tom's A/B files or something else? I've tried with multiple files, and the jitter value appears to be constant on all Show button presses. 

Link to comment
6 hours ago, pkane2001 said:

 

Hi Arpiben, 

 

I'm trying to reproduce this problem. Did you use Tom's A/B files or something else? I've tried with multiple files, and the jitter value appears to be constant on all Show button presses. 

 

Hi Paul,

 

I did not retrieve the reported behaviour with previous cases: pure tones or audio files.😟

But I have it with the following files. TIE values obtained with several shows: 3.5 ms / 4.4 ms / 5.6 ms / 7.8 ms 

 

A.wav

B.wav

 

Please do not play them !

A :1000 kHz 0 dBFs 

B: 1000 kHz 0 dBFs + TIE 100 ns sinus 0.1 Hz

 

DW Jitter estimation seems to be tricked with pure tones. Reported values are not correct for such cases.

With TIE= 1ms instead of 100 ns reported jitter values are: 83400 s / 39560 s

 

Rgds 

 

image.png.2a637ac15b4af535fed24057f3940d74.png

 

image.thumb.png.a7248ac87e9e94d4897155cfac490cc9.png

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...