DeltaWave null-testing audio comparator (beta)

[pkane2001]

3 minutes ago, esldude said:

I was about to ask about that.  As I tried cutting frequencies at both ends, and came to the conclusion the filtering must be done after alignment.  Now I know this might complicate matters, do you think having an option to filter prior to alignment might be worthwhile.  Or maybe that filtering prior to alignment when filtering is selected is simply a better way to go about it?

 

The reason I left it till the end is not to let things like ringing and phase distortions interfere with the alignment algorithm. Adding another option to pre-filter is not a problem, except for the already very busy main window.

[esldude]

Another pointer to it somehow being in the recorded file properties,  I used the original file with the Anna Caram snippet twice.   And looked at that versus a loopback.  You see the pattern repeat.  Notice the lower point occurs at 34 seconds in the single file.  The single file was 1:07.  Add 34 seconds and you'd expect it to happen again at 1:41.  Which it does. 

[fas42]

26 minutes ago, pkane2001 said:

 

Probably not in detail, but the algorithm is to measure offsets of samples at multiple locations in the waveforms.  Then, a linear regression is used to fit a line to the offsets, the slope of the line is the drift, the  intercept is the offset.

 

The measurement at a particular location is per a single sample, a sequence of samples, or other grouping; any sort of averaging of multiple values at a particular location done?

[pkane2001]

2 minutes ago, fas42 said:

 

The measurement at a particular location is per a single sample, a sequence of samples, or other grouping; any sort of averaging of multiple values at a particular location done?

 

It’s done in the frequency domain, so over many samples.

[esldude]

3 hours ago, pkane2001 said:

 

Dennis, you were on to something when you thought of phase. Except it’s the phase of the lower frequencies that is the problem here, not the higher ones. High pass filter of 800Hz or so eliminates the pattern completely. Needs to be applied in Audacity since DW applies filters at the end of processing, and not at the beginning.

 

What causes these errors remains to be determined, but we are getting closer! 

I don't know definitively, but am pretty sure none of my interfaces are direct coupled.  So they likely go thru a cap.  They likely roll a bit early as in recordings low frequency garbage is a problem.  So they might have some phase anomalies an octave or three above where they start getting close to flat response.   So the next question would be how does this interact with the software or otherwise to cause a genuine drift pattern.  Is it a residual of what Deltawave does, or is it a real drift difference actually happening in the signal itself?

[pkane2001]

17 minutes ago, esldude said:

I don't know definitively, but am pretty sure none of my interfaces are direct coupled.  So they likely go thru a cap.  They likely roll a bit early as in recordings low frequency garbage is a problem.  So they might have some phase anomalies an octave or three above where they start getting close to flat response.   So the next question would be how does this interact with the software or otherwise to cause a genuine drift pattern.  Is it a residual of what Deltawave does, or is it a real drift difference actually happening in the signal itself?

 

No definitive answer yet, but we know that something in the lower frequencies, introduced by a recording loop, is causing the distinctive drift pattern. The same pattern doesn't appear if the drift is added in software.

[esldude]

I just checked the Zen Tour and was surprised to find it is fully flat down to 10 hz and only .8 db down at 2 hz.  

[esldude]

The 18i20 is only flat down to 300 hz.  The drop is very slow reaching -1 db at 20 hz, but then 18 db down at 2 hz.  Maybe if the roll off is effecting phase that is why the pattern was at a higher magnitude when it was involved. 

[blue2]

Could you try to summarise the problem in precise terms? Here's my understanding - please correct.

 

The problem: The clock drift analysis shows a variation in a smooth sinusoid like pattern amplitude ~0.02ppm but expected is no variable drift as seen in software generated examples - how many samples does this equate to?

Test scenario: loopback recording only, similar behaviour across various computers as source and various DAC/ADC

 

The algorithm:

… the algorithm is to measure offsets of samples at multiple locations in the waveforms. Then, a linear regression is used to fit a line to the offsets, the slope of the line is the drift, the intercept is the offset.
From the charts 20 samples per file are used regardless of run time of 1-3mins in examples.
You mentioned the comparison is in the frequency domain? Please explain how this is done and what values are compared. I'd have thought you would just pattern match the raw data samples (which means both files have to be at the same rate)?

[pkane2001]

2 hours ago, blue2 said:

Could you try to summarise the problem in precise terms? Here's my understanding - please correct.

 

The problem: The clock drift analysis shows a variation in a smooth sinusoid like pattern amplitude ~0.02ppm but expected is no variable drift as seen in software generated examples - how many samples does this equate to?

Test scenario: loopback recording only, similar behaviour across various computers as source and various DAC/ADC

 

The problem is not just a sinusoid drift pattern, that was just one of the tracks that caused that. For some source material, the pattern appears to have no sine-wave component. The magnitude of the residual drift also varies with the number of passes through the DAC/ADC loop, although the general pattern remains similar:

 

2 hours ago, blue2 said:

You mentioned the comparison is in the frequency domain? Please explain how this is done and what values are compared. I'd have thought you would just pattern match the raw data samples (which means both files have to be at the same rate)?

 

There's no comparison, just a simple phase calculation of an offset. Calculating precise sub-sample offset in the time domain is slow and produces an inferior precision. I do have a time-domain offset calculation buried somewhere, probably worth it to dig it up to see if the problem goes away.

 

Both files are sampled at the same rate in all of these examples, but DeltaWave will resample if needed.

 

By the way, since I now have a way to correct this by high-pass filtering, this residual drift has very little effect on the final null value calculation. In fact, in all my tests it was producing less than 1dB difference in RMS null. So, while this may be a curious effect, it doesn't seem to make much of a difference in the final result. Perhaps because of the low magnitude of the error (0.01ppm represents a 1 sample delay in 100 million samples, or in 38 minutes of music at 44.1kHz).

[pkane2001]

7 hours ago, esldude said:

The 18i20 is only flat down to 300 hz.  The drop is very slow reaching -1 db at 20 hz, but then 18 db down at 2 hz.  Maybe if the roll off is effecting phase that is why the pattern was at a higher magnitude when it was involved. 

 

Could be. This brings up a different problem that I've been thinking about: how to see where the main differences are in a specific frequency range, at a specific place in the file. Not averaged over the whole track. Spectrogram is meant to help with that, but its resolution and color-coding makes it very hard to see minute details. You can't effectively zoom-in on it to see more. I may need to build more analysis tools

 

[esldude]

Okay now for reference here is the pattern of drift comparing a loopback of the Anna Caram recording. 

 

Now here is the pattern if I boost everything below 200 hz.  It actually was EQ that was up 3 db/octave from 200 hz down to 20 hz.  I had three of these all so similar there is no point in showing more than one.  The pattern is the same, but notice the amplitude of the drift is doubled. 

 

 

In this one I added a 70 hz steady tone at -26 dbFS in a file that is averaging an RMS level of 19 dbFS.  It is slightly different being twice the amplitude of the original comparison, and the flattened area between 14 and 23 seconds is the same in all three loopbacks of this.  So that is a consistent change.  Worth noting I did the same thing with a 200 hz tone at -26 dbFS which effectively made no difference.  So I think lower frequencies are where this is happening. 

[Miska]

10 hours ago, pkane2001 said:

Could be. This brings up a different problem that I've been thinking about: how to see where the main differences are in a specific frequency range, at a specific place in the file. Not averaged over the whole track. Spectrogram is meant to help with that, but its resolution and color-coding makes it very hard to see minute details. You can't effectively zoom-in on it to see more. I may need to build more analysis tools

 

That is essentially the tricky part and where wavelet analysis comes to help.... WVD is one of the good ways, but still needs some amount of correct interpretation from the reader.

 

[Miska]

17 hours ago, esldude said:

The 18i20 is only flat down to 300 hz.  The drop is very slow reaching -1 db at 20 hz, but then 18 db down at 2 hz.  Maybe if the roll off is effecting phase that is why the pattern was at a higher magnitude when it was involved. 

 

Any analog filter will have phase impact far beyond the actual "cut-off" points. For example low-pass filter with -3 dB fc at 100 kHz still tends to have some degrees of phase error at 20 kHz. Also DC-block high-pass filters have the same effect. I've personally have been using 0.1 Hz -3 dB fc point for DC-blocks to avoid excessive phase shifts at 20 Hz.

 

[esldude]

Here is the pattern for an added 25 hz at -26 dbFS.  Pattern is slightly altered, but the magnitude of reported drift is now 3 times greater. 

 

 

[fas42]

Just exploring the speed change behaviour of that 2.0.5 version of Audacity - tried a drift of 0.75ppm, because from eyeballing the waveforms, that gave good clock alignment of the original and 1st gen Bob Marley. The clock drift pattern between the now aligned orig and 1st showed excellent drift correction. but mediocre nulling, in DW.

 

Okay, what about comparing orig, with sped up by 0.75ppm orig, in DW - as compared to doing the equivalent, manual procedure in 2.0.5 Audacity - slowing down by 0.75ppm ... the good news is that the waveform envelopes of the deltas were remarkably similar, between DW and Audacity; the not so good news is that Audacity was easily 40dB or so better in suppressing the musical content.

 

 

[esldude]

10 minutes ago, Miska said:

 

Any analog filter will have phase impact far beyond the actual "cut-off" points. For example low-pass filter with -3 dB fc at 100 kHz still tends to have some degrees of phase error at 20 kHz. Also DC-block high-pass filters have the same effect. I've personally have been using 0.1 Hz -3 dB fc point for DC-blocks to avoid excessive phase shifts at 20 Hz.

 

Yes I was aware of that which is why I thought high pass filters might be causing these effects.  I would think the drift difference isn't in the signal itself, but in the way it effects drift calculation in software.  

[esldude]

Here is the same file showing a similar pattern with -20dbFS 5 hz tone added.  The amplitude of the drift is now much much higher though showing the similar pattern.  So it looks like phase issues at lower frequencies can trip up Deltawave on drift calculation even enough to lower the null depths.  Well actually when gear messes up phase it actually is a less well nulled result.  But in a perfect world you'd get a lowered null without mistaking low frequency phase for increased drift.  

 

[fas42]

Wavelet analysis has been mentioned by Miska ... tracking down what it's about, came across this good intro explanation, https://georgemdallas.wordpress.com/2014/05/14/wavelets-4-dummies-signal-processing-fourier-transforms-and-heisenberg/

[fas42]

I had a go looking at phase variations; my way - noted peaks in the Marley tracks at 60 and 5161Hz, so notched those out in Audacity, for the orig and 1st gen, in 2 separate copies each, four tracks now. Looked for where zero crossings were close for the 60 and 5161 frequencies, and this showed that the 1st gen had its 5161 frequency a 1/4 wavelength delayed, compared to orig. This was where the 5161 energy was at a level of 0.2, so would have sizeable impact.

[esldude]

19 minutes ago, fas42 said:

I had a go looking at phase variations; my way - noted peaks in the Marley tracks at 60 and 5161Hz, so notched those out in Audacity, for the orig and 1st gen, in 2 separate copies each, four tracks now. Looked for where zero crossings were close for the 60 and 5161 frequencies, and this showed that the 1st gen had its 5161 frequency a 1/4 wavelength delayed, compared to orig. This was where the 5161 energy was at a level of 0.2, so would have sizeable impact.

I'm pretty sure the reason some of these nulls aren't better is down to phase.  If things aren't DC coupled or like Miska mentioned made to work to a very low response of having the low corner at 2 hz or less they'll have some phase shift on the low end.  You'll have some phase shift on the high end from digital filtering at 44 khz rates.  You can have low noise, low distortion, and still with a little phase shift nulls will be decreased.  That is why when I compare a pair of loopbacks that were taken at different times they will null better than 100 db, but from one generation to another you are doing good to get near 70 db.  The loopbacks compared to each other have both seen the same amount of phase shift at both ends of the spectrum.  

 

Now having said all that I'm not convinced these phase shifts make much of a difference we can hear.  I prefer it not be there however. 

 

 

[pkane2001]

15 hours ago, Miska said:

 

That is essentially the tricky part and where wavelet analysis comes to help.... WVD is one of the good ways, but still needs some amount of correct interpretation from the reader.

 

 

Wavelets is what I was thinking. Proper presentation is going to be crucial to aid in understanding. So far, I've not seen any good examples from others, so it seems I'll need to come up with my own.

[pkane2001]

15 hours ago, fas42 said:

Just exploring the speed change behaviour of that 2.0.5 version of Audacity - tried a drift of 0.75ppm, because from eyeballing the waveforms, that gave good clock alignment of the original and 1st gen Bob Marley. The clock drift pattern between the now aligned orig and 1st showed excellent drift correction. but mediocre nulling, in DW.

 

Okay, what about comparing orig, with sped up by 0.75ppm orig, in DW - as compared to doing the equivalent, manual procedure in 2.0.5 Audacity - slowing down by 0.75ppm ... the good news is that the waveform envelopes of the deltas were remarkably similar, between DW and Audacity; the not so good news is that Audacity was easily 40dB or so better in suppressing the musical content.

 

 

 

Frank, try to speed up using rate change (resampling) in Audacity and then slow it down using speed change. These two should result in the same good null. Do they? After all these two should be equivalent operations, right?

[pkane2001]

15 hours ago, esldude said:

Yes I was aware of that which is why I thought high pass filters might be causing these effects.  I would think the drift difference isn't in the signal itself, but in the way it effects drift calculation in software.  

 

Considering the algorithm is using phase to compute drift, that can certainly be possible.

 

[fas42]

8 hours ago, pkane2001 said:

 

Frank, try to speed up using rate change (resampling) in Audacity and then slow it down using speed change. These two should result in the same good null. Do they? After all these two should be equivalent operations, right?

 

As mentioned before, the speed change algorithm in Audacity is different from, and far less precise, than the resampling trick; can easily get a 60dB better null, in Audacity, using rate change as compared to speed %'s. Which is why I use the "funny" way to adjust the clock.