DeltaWave null-testing audio comparator (beta)

[pkane2001]

New version of DeltaWave has just been posted. This version includes the following new features and bug fixes:

Changes in 1.0.25b

• Added error % distribution chart
• Added gain error over time chart
• Improved sub-sample offset calculation to reduce residual drift error
• Added tab selection menu item
• Added option to apply filters before or after alignment calculations
• Changed clock drift error raw plot to show the actual error instead of the linear interpolation
• Changed sample trimming to remove any data from time that doesn’t have corresponding data in the other waveform.

@esldude: please take a look at the residual drift plot for your loopback analysis with this version. I think it should show some improvement.

 

Error distribution chart shows the number of samples (in percent) that are at that error value. Horizontal axis is the amplitude of the error, the vertical is the % of total samples at that error value:

 

Clock drift plot now shows the actual measured raw error rather than the linear interpolation:

 

Gain error plot shows the average residual amplitude error at a number of points over the duration of the track to help pinpoint where there are large errors in alignment:

 

As always, all comments, suggestions and bug reports are more than welcome!

[esldude]

So can you fill in details about the HP and LP start, end and start/end filtering?

[pkane2001]

8 minutes ago, esldude said:

So can you fill in details about the HP and LP start, end and start/end filtering?

 

Basically the same filtering algorithm as before (IIR or FIR filters, as in settings). Previously, all phase and gain alignment was performed first and the filters were applied at the end, just before computing the null values.

 

The new version allows the filters to be applied before any alignment, after the alignment, or both, before and after. Shouldn't make a big difference in the result, but if there are frequencies or phase errors that are affecting phase/drift computation then pre-filtering may help detect this condition. I'm still not sure if this is really useful, but I'll leave it in as an option for now

 

[esldude]

Okay just a quick look.  For files with no options turned on it looks like difference results are regularly about 1 db less good.  Null depths are a few db less good.  

 

As for doing filtering before vs after I'm not seeing much if any difference just as you said. 

 

The timing drift does seem to show more variable results on loopbacks vs original.

 

Results do seem several db better on loopbacks vs loopbacks where result are -100 db or better for difference and some really impressive null depths.  The gain error seems very useful for this.  It shows where there is more error vs others.  So you can snip it off and improve the results.  I wonder about this correlated null depth. 

 

 

 

 

 

 

 

[pkane2001]

16 minutes ago, esldude said:

Okay just a quick look.  For files with no options turned on it looks like difference results are regularly about 1 db less good.  Null depths are a few db less good.  

 

As for doing filtering before vs after I'm not seeing much if any difference just as you said. 

 

The timing drift does seem to show more variable results on loopbacks vs original.

 

Results do seem several db better on loopbacks vs loopbacks where result are -100 db or better for difference and some really impressive null depths.  The gain error seems very useful for this.  It shows where there is more error vs others.  So you can snip it off and improve the results.  I wonder about this correlated null depth. 

 

 

 

 

 

 

 

 

So not as much of a repeatable pattern in the clock drift now? Do you see any reduction in the magnitude of the residual clock drift?

 

The change I made was to use an estimate of the noise floor to cut off phase values with the magnitude below the floor, so they don’t influence the calculation. Previously, all frequencies were included in the calculation, just weighted proportional to their magnitude. 

[esldude]

40 minutes ago, pkane2001 said:

 

So not as much of a repeatable pattern in the clock drift now? Do you see any reduction in the magnitude of the residual clock drift?

 

The change I made was to use an estimate of the noise floor to cut off phase values with the magnitude below the floor, so they don’t influence the calculation. Previously, all frequencies were included in the calculation, just weighted proportional to their magnitude. 

Give a bit to look more closely.  It looks like no pattern at a glance. 

 

Also on loopback vs loopback though values were good before you could hear music clearly in the noise previously.  It is gone now.  Makes me think the matching is better as the noise level is just as low, but music isn't heard even with 95 db of gain.  Well this is if I use a 20 hz high pass filter. 

[esldude]

Yes, no pattern seems to repeat now for drift that are related to the musical content. 

 

I also notice when I HP at 20 hz there is no music only noise left when comparing loopbacks taken a few minutes apart.  While also getting low numbers loopbacks done a week apart do have a little music left among the noise.  So apparently a very small amount of drift difference done days apart. 

 

If I leave off the HP filters there is a bit of residual music at low levels in the noise. 

 

So all in all I think the change to leave off phase values in or below the noise floor was a positive improvement. 

[esldude]

I like the drift being shown this way before correction better than the linear drift line used previously even though you are still using the overall linear drift in the function of the program. 

 

The error distribution looks nice, but I'm at a loss as to how to make use of it or what it shows that would alert me to some issue.  I suppose if it were badly messed up it might alert one to be suspicious of the results. 

[pkane2001]

26 minutes ago, esldude said:

I like the drift being shown this way before correction better than the linear drift line used previously even though you are still using the overall linear drift in the function of the program. 

 

The error distribution looks nice, but I'm at a loss as to how to make use of it or what it shows that would alert me to some issue.  I suppose if it were badly messed up it might alert one to be suspicious of the results. 

 

Agree on all points. While I can see some patterns in the error distribution plot, it doesn’t seem obvious how to interpret it. Maybe with a little experience it’ll become more obvious. If not, I’ll remove it.

[fas42]

The latest version looks very good on a number of fronts, Paul! However, what I'm caught up in at the moment is trying the sped up in Audacity against original comparison, trying to get an audible null. And am getting there! The automatic matching is not doing it, but I'm now juggling the combo of offset, and drift, and listening to the delta - this gives an excellent audible null at one point: starts off loud, slowly fades down to almost inaudible; then fades up again. Now fine tuning both the offset and drift, to see if I can crack the jackpot!

 

A key giveaway is the Corr Null (dbA) - this jumps in improvement by nearly 30dB with the tiniest alteration, which you then lose by going too far.

[pkane2001]

34 minutes ago, fas42 said:

The latest version looks very good on a number of fronts, Paul! However, what I'm caught up in at the moment is trying the sped up in Audacity against original comparison, trying to get an audible null. And am getting there! The automatic matching is not doing it, but I'm now juggling the combo of offset, and drift, and listening to the delta - this gives an excellent audible null at one point: starts off loud, slowly fades down to almost inaudible; then fades up again. Now fine tuning both the offset and drift, to see if I can crack the jackpot!

 

A key giveaway is the Corr Null (dbA) - this jumps in improvement by nearly 30dB with the tiniest alteration, which you then lose by going too far.

 

Looking forward to seeing your results, Frank. This is what I saw previously: a tiny change in the drift value, for example, could cause a large jump in the correlated null. That's not necessarily a bad thing, but it does mean that the correlated null is not the best indicator of a good match. As I said before, I'm not sure that any single value is. But I'll certainly continue looking and thinking about better ways to measure and report how close the two files are.

 

So far, what I've been using is a combination of RMS null, correlated null, and % bit perfect match. As long as I see similar or better values in all three of these indicators, I can be sure I found a better match. But nothing beats checking the spectra alignment, the drift error plot, and the delta waveform. All of these add to the story.

[esldude]

Okay, so I've recorded some loopbacks and done them a minute or two apart.  Clocks are the same, gear the same etc etc. you'd expect nothing except noise if the two are properly aligned and nulled with each other.  So I gave it a try in Paul's newest version of Deltawave. 

 

Okay here is the delta or difference between two such files.  Did not leave me much to look at here. This is what it looks like in Audacity. 

 

Then there is this pathetic graph of the error distribution.  It clusters around something like -130db.  What is that going to tell you? Ha!

 

So in this loopback mode with these gain settings the effective silent noise floor is maybe around -108 or -110 db A-wtd.  I suppose a null result of -103.8 db A-wtd isn't too far off the mark.  

 

It isn't perfect though.  There are hints of the music and the vocal now and again.  Well if you apply 90 db of gain there is.  Here is a link to the Delta wave of this comparison so you can listen for yourself.  I would suggest 90 db of gain otherwise it will be kind of a null listening experience. 

https://www.dropbox.com/s/ims9mos2798xviq/Caram zt loop3 delta.wav?dl=0

 

So I suppose the current performance of Deltawave is such that when two bits of sound shot thru the analog world should be about as close as the noise varies over time, that Deltawave can line them up and show such is the case.  Or almost.  

 

 

 

 

[pkane2001]

7 hours ago, esldude said:

Okay, so I've recorded some loopbacks and done them a minute or two apart.  Clocks are the same, gear the same etc etc. you'd expect nothing except noise if the two are properly aligned and nulled with each other.  So I gave it a try in Paul's newest version of Deltawave. 

 

Okay here is the delta or difference between two such files.  Did not leave me much to look at here. This is what it looks like in Audacity. 

 

Then there is this pathetic graph of the error distribution.  It clusters around something like -130db.  What is that going to tell you? Ha!

 

So in this loopback mode with these gain settings the effective silent noise floor is maybe around -108 or -110 db A-wtd.  I suppose a null result of -103.8 db A-wtd isn't too far off the mark.  

 

It isn't perfect though.  There are hints of the music and the vocal now and again.  Well if you apply 90 db of gain there is.  Here is a link to the Delta wave of this comparison so you can listen for yourself.  I would suggest 90 db of gain otherwise it will be kind of a null listening experience. 

https://www.dropbox.com/s/ims9mos2798xviq/Caram zt loop3 delta.wav?dl=0

 

So I suppose the current performance of Deltawave is such that when two bits of sound shot thru the analog world should be about as close as the noise varies over time, that Deltawave can line them up and show such is the case.  Or almost.  

 

Another chart that may aid in understanding the quality of alignment is correlated null over time. Rather than reporting a single number, it may be interesting to see how the correlated null value varies.

 

For example, this is reported as a 52dB correlated null, but obviously varies over time. Seems to get better towards the end of the track:

 

 

 

[fas42]

Just to tantalise people with what's possible, if you get the numbers exactly right, here's a progression of manual adjustment of the initial matching of the files mentioned just before - note a problem with the table display; we need far greater precision in the columns, to more easily see where we're heading ...

 

 

The result always looks the same in the Delta waveform; the telling factor is playback of it - the last entry above has a good portion of it becoming buried in noise.

[pkane2001]

10 minutes ago, fas42 said:

Just to tantalise people with what's possible, if you get the numbers exactly right, here's a progression of manual adjustment of the initial matching of the files mentioned just before - note a problem with the table display; we need far greater precision in the columns, to more easily see where we're heading ...

 

 

The result always looks the same in the Delta waveform; the telling factor is playback of it - the last entry above has a good portion of it becoming buried in noise.

 

I knew that the manual adjustment tool would appeal to you, Frank!  

 

When you click on any of the rows, you'll see the full-precision values displayed above. But I do see your point about displaying more precision in the table.

 

[esldude]

3 hours ago, pkane2001 said:

 

Another chart that may aid in understanding the quality of alignment is correlated null over time. Rather than reporting a single number, it may be interesting to see how the correlated null value varies.

 

For example, this is reported as a 52dB correlated null, but obviously varies over time. Seems to get better towards the end of the track:

 

 

 

Yes this null over time graphic would be very useful.  Oh, my previous post was meant as a joke. The joke being to complain when currently the results are so good. You've made plenty of improvement in a short time. Deltawave is currently very impressive software.

[TomCapraro]

I compared a loopback between the original file (pink noise) and the analog signal passing through a M2tech Young DAC. Of all the comparisons I made the DW 1.0.25 version turns out to be, not only the best, but it has surpassed even the best things that Audio Diff Maker produced with some signals.

 

 

[pkane2001]

14 minutes ago, TomCapraro said:

I compared a loopback between the original file (pink noise) and the analog signal passing through a M2tech Young DAC. Of all the comparisons I made the DW 1.0.25 version turns out to be, not only the best, but it has surpassed even the best things that Audio Diff Maker produced with some signals.

 

 

 

I'm glad you were able to run the test with AudioDiffMaker, Tom! I have had no luck getting it to work right. That's one of the main reasons I developed DeltaWave

[fas42]

8 hours ago, pkane2001 said:

 

I knew that the manual adjustment tool would appeal to you, Frank!  

 

 

 

The point of course is that I could align the two waveforms more accurately just by watching what the numbers were doing, and noting how the delta signal altered through the clip, from the beginning to the end. Which is something that software can also do, .

[pkane2001]

Speaking of the new error distribution plot... Batman or something more sinister?  

 

An example of a very poor alignment between two unrelated white noise files.

 

[esldude]

Just so you won't feel the work on filtering at start or end was wasted effort.  I have a signal showing filtering at the start can be beneficial. The signal was a 20khz to 20 hz sweep.  Combined with various other short partial sweeps and silence so the waveform might give the software something to sink its teeth in.  Reference was compared to the same file slowed by 10 ppm in Audacity. 

 

First up is with no filters.  Results aren't so good with only a very good fit quality. 

 

Next is filtering LP at 20 khz at the end.  Really no help at all. 

 

Next is LP set to 20 khz with filtering at the start.  You see fit become excellent, difference improves more than 20 db as do null depths.  The clock drift now is very close to the known amount of 10 ppm. 

 

 

[esldude]

The above is also a case where doing L+R is better than doing L or doing R alone. 

In the left channel I had the same sweeps as the right channel, but with the file reversed.  So it was a 20hz to 20 khz sweep in that channel.  As you can see even with no filtering this one is improved by 20 db over either channel alone compared.  Fit is now excellent, and clock drift is closer to correct. 

 

[fas42]

Still refining a process of doing manual adjustments to achieve a true null, when this is possible. Gone back to drift altered by rate change combo, which have higher precision. DW still leaves in spikes only 40dB down, and the music is clearly heard well above the noise in the delta - by careful fine tuning of offset and drift so far I have managed to get complete burial in the noise over about 2/3rd of the clip; implying it should be possible to make it inaudible completely.

 

So, DW should be able to achieve excellent nulls by more exact measuring of what's going on as those settings are played with; mimicking the manual trialing.

[pkane2001]

4 hours ago, fas42 said:

Still refining a process of doing manual adjustments to achieve a true null, when this is possible. Gone back to drift altered by rate change combo, which have higher precision. DW still leaves in spikes only 40dB down, and the music is clearly heard well above the noise in the delta - by careful fine tuning of offset and drift so far I have managed to get complete burial in the noise over about 2/3rd of the clip; implying it should be possible to make it inaudible completely.

 

So, DW should be able to achieve excellent nulls by more exact measuring of what's going on as those settings are played with; mimicking the manual trialing.

 

Sure, Frank. Once you have some test results to share, I can take a look at which part of the process needs to be fine tuned. 

[fas42]

Too much going on at the moment - will aim to to give some detail later today ...