DeltaWave null-testing audio comparator (beta)

[pkane2001]

13 minutes ago, fas42 said:

It's worse than that, Paul - the matching is completely failing in the run with 10 secs trimmed, visually in the Matched panel the correlation is terrible.

 

Nevertheless, the only difference I can see is in the gain adjustment factor. Drift, and offset values are calculated the same in both cases.

[pkane2001]

19 minutes ago, fas42 said:

It's worse than that, Paul - the matching is completely failing in the run with 10 secs trimmed, visually in the Matched panel the correlation is terrible.

 

Here’s one possibility... can you please try to set the same FFT size for non-linear EQ and for Spectrum in settings? I think that may be the cause when non-linear EQ is used.

[fas42]

Will do shortly ...

[fas42]

26 minutes ago, fas42 said:

Will do shortly ...

 

Nope, didn't make any difference, unfortunately ...

[pkane2001]

8 minutes ago, fas42 said:

 

Nope, didn't make any difference, unfortunately ...

 

Hmm, let me see if I can reproduce this here. Thanks for testing!

[fas42]

Also, Paul, I'm noticing that the middle button marquee selecting on a graph is now very erratic - making it almost useless. I don't know whether it has something to do with the code style changes you've done, but basically it is almost random in its deciding as to whether I'm still holding the middle button down; meaning I end up most times with a zoom over a tiny emptiness.

[pkane2001]

4 hours ago, fas42 said:

Also, Paul, I'm noticing that the middle button marquee selecting on a graph is now very erratic - making it almost useless. I don't know whether it has something to do with the code style changes you've done, but basically it is almost random in its deciding as to whether I'm still holding the middle button down; meaning I end up most times with a zoom over a tiny emptiness.

 

That shouldn’t be anything related to the latest changes. 

[Arpiben]

On 8/16/2019 at 1:34 PM, pkane2001 said:

 

Hi folks,

 

Life has been taking priority for a bit, interfering with fun. I've been working on the FFT Windowing in the spare moments, and have reworked most of the FFT logic in DeltaWave. Primarily this was to improve accuracy, I don't believe this will result in any significant improvement in the actual null calculation, since the errors we are talking about are well below -180dB or more. New version needs a lot of testing, so that's what's been holding up its release. I may post an early cut to get your feedback in a day or two.

 

@Arpiben, the noise you were noticing in the pure generated sine-wave is absolutely a numerical truncation/quantization error in floating point calculations. Consider that 32-bit floating point is actually about 23-bits of the actual number (mantissa) and about 7 bits of an exponent -- the precision can be less than a simple 24-bit integer format! Here's a spectrum plot of a 500Hz sine wave sampled @ 88.2k, all pure double floating point format with no conversions:

 

And now, the same exact data, but converted to 32-bit floating point (i.e., losing some precision). All the calculations are still performed using 64-bit floating point, so the only difference from above is the truncation of data when reducing sample size:

 

The only thing that can fix this is using higher-precision samples, so I added a 64-bit WAV file export option to DeltaWave.

I also added a number of new FFT Windows, some with excellent side lobe rejection, such as Kaiser and Taylor (1M point Taylor window is what's used in the plot above).

 

 

Hi Paul,

 

Just back from a tough road trip in Caucasus area and only start looking at the new release.

I agree with you on the quantization errors brought by computation ( truncation,roundings,etc...).

If one uses a linear scale for the frequency/bin one will notice that noise is uniformly distributed among frequency/bin scale.

 

The FFT computational noise floor is around -310 dB this translates into 46 bit. For audio case, bin values are far from reaching such level.

 

Back to questions on DW 1.038b:

• will it be possible to have a rectangular window ( equivalent to no window or boxcar) in spectrum settings ?
• what is the minimum sample length DW is able to deal with (16384)?
• what is Beta value used in Kaiser window? I am expecting Beta in between 14 and 20.
• what does the phase limit mean? Actually not possible to go under -200 dB ( same as previous versions)

With first tests I am having good result with windowing (Kaiser tested only). In one case dealing with only 9600 samples DW got into error, hence my question above.

 

 

[pkane2001]

1 hour ago, Arpiben said:

 

Hi Paul,

 

Just back from a tough road trip in Caucasus area and only start looking at the new release.

I agree with you on the quantization errors brought by computation ( truncation,roundings,etc...).

If one uses a linear scale for the frequency/bin one will notice that noise is uniformly distributed among frequency/bin scale.

 

The FFT computational noise floor is around -310 dB this translates into 46 bit. For audio case, bin values are far from reaching such level.

 

Back to questions on DW 1.038b:

• will it be possible to have a rectangular window ( equivalent to no window or boxcar) in spectrum settings ?
• what is the minimum sample length DW is able to deal with (16384)?
• what is Beta value used in Kaiser window? I am expecting Beta in between 14 and 20.
• what does the phase limit mean? Actually not possible to go under -200 dB ( same as previous versions)

With first tests I am having good result with windowing (Kaiser tested only). In one case dealing with only 9600 samples DW got into error, hence my question above.

 

 

 

Hi Arpiben,

 

-310db is actually a bit over 51 bits, not 46. As it happens, 52 bits is the size of mantissa for 64-bit IEEE floating point number, so pretty much what I'd expect. My answers below, in red:

 

• will it be possible to have a rectangular window ( equivalent to no window or boxcar) in spectrum settings ?
  • Possible, but rectangular window isn't good for any of the computations needed by DW for delta analysis
• what is the minimum sample length DW is able to deal with (16384)?
  • Yes, 16384 is the minimum
• what is Beta value used in Kaiser window? I am expecting Beta in between 14 and 20.
  • I'll have to check what actual value. It was set to get to about -300dB side lobe rejection
• what does the phase limit mean? Actually not possible to go under -200 dB ( same as previous versions)
  • That's not used anymore. It used to control the delta phase chart display by setting the minimum bin amplitude when plotting phase differences. Nonlinear EQ threshold is the setting that controls this in phase computations now. I'll remove it.

 

 

[Arpiben]

1 hour ago, pkane2001 said:

 

Hi Arpiben,

 

-310db is actually a bit over 51 bits, not 46. As it happens, 52 bits is the size of mantissa for 64-bit IEEE floating point number, so pretty much what I'd expect. My answers below, in red:

 

• will it be possible to have a rectangular window ( equivalent to no window or boxcar) in spectrum settings ?
  • Possible, but rectangular window isn't good for any of the computations needed by DW for delta analysis
• what is the minimum sample length DW is able to deal with (16384)?
  • Yes, 16384 is the minimum
• what is Beta value used in Kaiser window? I am expecting Beta in between 14 and 20.
  • I'll have to check what actual value. It was set to get to about -300dB side lobe rejection
• what does the phase limit mean? Actually not possible to go under -200 dB ( same as previous versions)
  • That's not used anymore. It used to control the delta phase chart display by setting the minimum bin amplitude when plotting phase differences. Nonlinear EQ threshold is the setting that controls this in phase computations now. I'll remove it.

 

 

 

Thanks Paul.

Dealing with computational noise floor don't forget to take into account the length of the DFT window.

For a full scale sinusoid with only amplitude quantization and n bit resolution we have:

n DFT is the window size in base 2. 

Xref=1

window size = 1048576 -> nDFT=20

 

FFT length does not affect the computational noise floor

FFT length does reduce the quantization noise floor coefficients (Ak)

[pkane2001]

56 minutes ago, Arpiben said:

 

Thanks Paul.

Dealing with computational noise floor don't forget to take into account the length of the DFT window.

For a full scale sinusoid with only amplitude quantization and n bit resolution we have:

n DFT is the window size in base 2. 

Xref=1

window size = 1048576 -> nDFT=20

 

FFT length does not affect the computational noise floor

FFT length does reduce the quantization noise floor coefficients (Ak)

 

The 52 bit (~312dB) limit is imposed by the precision of the double-floating point number representation. DW can't compute a noise floor lower than that, simply because the numbers used to do the computation and to hold the result cannot represent anything with greater precision. It's a hard limit for DW, and I don't intend to switch to 128 bit representation any time soon 

[fas42]

11 hours ago, pkane2001 said:

 

That shouldn’t be anything related to the latest changes. 

 

Okay, I might go back to an older version of DW, to see if it's something perhaps related to my m/c that's recently occurred.

[pkane2001]

On 8/18/2019 at 7:43 PM, fas42 said:

 

Nope, didn't make any difference, unfortunately ...

 

Hi Frank, can you please try one more test? Turn on the 'Auto-trim' option in settings, and then run the same comparison trimming the 10 seconds off the end of both files? 

[fas42]

Will do ...

[fas42]

18 hours ago, pkane2001 said:

 

Hi Frank, can you please try one more test? Turn on the 'Auto-trim' option in settings, and then run the same comparison trimming the 10 seconds off the end of both files? 

 

Good call ...

 

With Auto-trim:

 

DeltaWave v1.0.38, 2019-08-21T16:27:26.4347467+10:00
Reference:  Original2.wav[L] 4907008 samples 44100Hz 24bits, stereo, MD5=00
Comparison: YamahaSteinbergMR816X.wav[L] 4947968 samples 44100Hz 24bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain EQ:True    Non-linear Phase EQ: True
    EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Non-Linear drift Correction:False
    Upsample:False, Window:Hann
    Spectrum Window:Hann, Spectrum Size:65536
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:True

Discarding Reference:  Start=0s, End=10s
Discarding Comparison: Start=0s, End=10s

Initial peak values Reference: -0.709dB   Comparison: -1.914dB
Initial RMS values Reference: -17.701dB   Comparison: -19.06dB

Null Depth=15.721dB
X-Correlation offset: -93 samples
Trimming 0 samples at start and 1 samples at the end that are below -90.31dB level

Drift computation quality, #1: Excellent (0.15µs)

Trimmed 17100 samples ( 387.755102ms) front, 0 samples ( 0.00ms end)

Final peak values Reference: -0.709dB   Comparison: -0.705dB
Final RMS values Reference: -17.685dB   Comparison: -17.685dB

Gain= -1.387dB (0.8524x) DC=0.00001 Phase offset=-2.109859ms (-93.045 samples)
Difference (rms) = -67.84dB [-85.01dBA]
Correlated Null Depth=72.16dB [81.57dBA]
Clock drift: 0 ppm

Files are NOT a bit-perfect match (match=3.01%) at 16 bits
Files are NOT a bit-perfect match (match=0.01%) at 24 bits
Files match @ 49.9783% when reduced to 11.55 bits

---- Variable Group Delay. Frequency matched from 0Hz to 21.1kHz:
    1kHz = 1.7µs (0.62°)
    2kHz = 892.3ns (0.64°)
    4kHz = 670.6ns (0.97°)
    8kHz = 560.8ns (1.62°)
    16kHz = 377.4ns (2.17°)
Timing error (rms jitter): 236.9ns

RMS of the difference of spectra: -133.576530625421dB
gn=1.17313662705122, dc=5.1987381403954E-06, dr=7.5E-10, of=-93.0447732934

DONE!

Signature: 8046356b2f466318e24dd8d588975fc9

 

 ------------------------

 

Without Auto-trim:

 

DeltaWave v1.0.38, 2019-08-21T16:32:06.2633294+10:00
Reference:  Original2.wav[L] 4907008 samples 44100Hz 24bits, stereo, MD5=00
Comparison: YamahaSteinbergMR816X.wav[L] 4947968 samples 44100Hz 24bits, stereo, MD5=00
Settings:
    Gain:True, Remove DC:True
    Non-linear Gain EQ:True    Non-linear Phase EQ: True
    EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -160dB
    Correct Drift:True, Precision:30
    Non-Linear drift Correction:False
    Upsample:False, Window:Hann
    Spectrum Window:Hann, Spectrum Size:65536
    Spectrogram Window:Lanczos, Spectrogram Size:32768, Spectrogram Steps:1024
    Dither:False
    Trim Silence:False

Discarding Reference:  Start=0s, End=10s
Discarding Comparison: Start=0s, End=10s

Initial peak values Reference: -0.709dB   Comparison: -1.914dB
Initial RMS values Reference: -17.701dB   Comparison: -19.06dB

Null Depth=15.721dB
X-Correlation offset: -93 samples
Drift computation quality, #1: Excellent (0.15µs)

Trimmed 0 samples ( 0.00ms) front, 0 samples ( 0.00ms end)

Final peak values Reference: -0.709dB   Comparison: -0.397dB
Final RMS values Reference: -17.701dB   Comparison: -17.708dB

Gain= -1.2554dB (0.8654x) DC=0.00001 Phase offset=-2.109862ms (-93.045 samples)
Difference (rms) = -32.43dB [-37.8dBA]
Correlated Null Depth=37.49dB [34.16dBA]
Clock drift: 0 ppm

Files are NOT a bit-perfect match (match=0.06%) at 16 bits
Files are NOT a bit-perfect match (match=0%) at 24 bits
Files match @ 49.9837% when reduced to 5.24 bits

---- Variable Group Delay. Frequency matched from 0Hz to 21.1kHz:
    1kHz = 2.3µs (0.82°)
    2kHz = 966.1ns (0.70°)
    4kHz = 647.5ns (0.93°)
    8kHz = 414.5ns (1.19°)
    16kHz = 286.3ns (1.65°)
Timing error (rms jitter): 5.4µs

RMS of the difference of spectra: -104.299435782995dB
gn=1.15549933882472, dc=1.20150967885453E-05, dr=7.85E-10, of=-93.044928729

DONE!

Signature: c7f29c4590ca12526d0486318d9848cf

 

 

[Arpiben]

@pkane2001

 

1. When Reduce Spectral Leakage is applied  is it correct to say that:

- no windowing is applied whatever mentioned in the parameters

- a rectangle window is applied respecting coherent frequency sampling with fmin (sync with the period of the largest sine wave)

Fs/fmin= N/Np where Fs: frequency sampling , N number of samples (FFT window) , Np number of periods of fmin inside N.

 If such are you keeping the FFT size in settings or it may be a different value in order to comply with coherent sampling conditions?

 

The feature is working pretty well.😊

 

2. Regarding Delta Phase curve,its sign (+/-) may change depending on window applied. 

 

[pkane2001]

18 minutes ago, Arpiben said:

@pkane2001

 

1. When Reduce Spectral Leakage is applied  is it correct to say that:

- no windowing is applied whatever mentioned in the parameters

- a rectangle window is applied respecting coherent frequency sampling with fmin (sync with the period of the largest sine wave)

Fs/fmin= N/Np where Fs: frequency sampling , N number of samples (FFT window) , Np number of periods of fmin inside N.

 If such are you keeping the FFT size in settings or it may be a different value in order to comply with coherent sampling conditions?

 

The feature is working pretty well.😊

 

2. Regarding Delta Phase curve,its sign (+/-) may change depending on window applied. 

 

 

1. The Reduce spectral leakage option adjusts the FFT overlap-add step size to to a multiple of the largest frequency component closest to the selected FFT size. The window in settings is still applied, so you can see the effect of each window on the signal. For example, Cosine23 window on 500Hz signal:

 

Bartlett window (same signal and FFT size, just the window type changed):

 

Hann(ing) window:

 

Blackman-Harris 7 term:

 

2. I'll take a look to see why that may be happening

[fas42]

On 8/20/2019 at 7:56 AM, fas42 said:

 

Okay, I might go back to an older version of DW, to see if it's something perhaps related to my m/c that's recently occurred.

 

Okay, looks like it's my laptop, regarding middle button marquee selecting on a graph being erratic ... went back about 8 versions of DW, and it's still poor at registering the mouse actions. Can't think of anything particular to blame - perhaps I had 'learnt' how to compensate for the lack of response, and was automatically zooming using the other methods, when I was constantly playing with DW ...

[Arpiben]

@pkane2001

 

Paul, many thanks to the improvements done in manual corrections 👍.

Nevertheless I have a few remarks. Please correct me, like usual, if I am wrong 😉

 

-Optimize is operating at one side from the starting value.

ex. Drift = -1 ppm  -> Optimize will start testing values greater or equal to -1ppm ( -0.9/-0.8/-0.976 ppm). Then it may jump to lower values but it will not repeat the same process. As a consequence one may miss the target.

 

- How do you estimate the accuracy of drift value for instance?

I have been trying it with single and multi tones with drifts varying from -1000 ppm to 1E-9 ppm and always  get something like -0,974 E xy ppm as optimised value for rms delta. Not complaining about the 2.5% error but finding it strange to have the same radical:

drift of -100 ppm -> -97,4 ppm from DW

drift of -1 ppm -> -0,974 ppm from DW

drift of -0,01 ppm -> - 0,00974 ppm from DW

etc...

 

 

[pkane2001]

10 hours ago, Arpiben said:

@pkane2001

 

Paul, many thanks to the improvements done in manual corrections 👍.

Nevertheless I have a few remarks. Please correct me, like usual, if I am wrong 😉

 

-Optimize is operating at one side from the starting value.

ex. Drift = -1 ppm  -> Optimize will start testing values greater or equal to -1ppm ( -0.9/-0.8/-0.976 ppm). Then it may jump to lower values but it will not repeat the same process. As a consequence one may miss the target.

 

- How do you estimate the accuracy of drift value for instance?

I have been trying it with single and multi tones with drifts varying from -1000 ppm to 1E-9 ppm and always  get something like -0,974 E xy ppm as optimised value for rms delta. Not complaining about the 2.5% error but finding it strange to have the same radical:

drift of -100 ppm -> -97,4 ppm from DW

drift of -1 ppm -> -0,974 ppm from DW

drift of -0,01 ppm -> - 0,00974 ppm from DW

etc...

 

 

 

If you select to optimize using a single parameter, the optimizer measures values above and below the initial value and then uses these as a hint (approximate value of the derivative) to indicate the direction in which to move the value. It could increase the value or decrease it to find the optimum result.

 

When optimizing using more than one variable, the gradient descent algorithm does something similar, but using a vector of initial values, estimating the gradient in different directions from the starting vector. The way this method works is that it's trying move the optimization along the parameter space in directions orthogonal to the other parameters, in effect resulting in optimizing each of the values mostly independently of the others.

 

Very few of the tested results are actually shown in the table. Only those results that improve at least one of the displayed values are shown. All the other combinations of parameters and their results are not listed, although they are computed -- I didn't think it was worth it to show parameters that didn't improve on any of the already discovered results.

[fas42]

On 8/22/2019 at 5:57 PM, fas42 said:

 

Okay, looks like it's my laptop, regarding middle button marquee selecting on a graph being erratic ... went back about 8 versions of DW, and it's still poor at registering the mouse actions. Can't think of anything particular to blame - perhaps I had 'learnt' how to compensate for the lack of response, and was automatically zooming using the other methods, when I was constantly playing with DW ...

 

Well, finally the penny's dropped ... the design of the the plotting library interface was not thought through well enough - if one only has a wheel which is clickable as the middle button, then one has to be paranoid that the wheel is not rotated the tiniest bit while doing the marquee select; if a rotation is detected, at all, then the select operation is terminated  ... so, if one is being casual, or a bit tired, or just normally human - then you will end up swearing at the software ... .

[pkane2001]

17 hours ago, fas42 said:

 

Well, finally the penny's dropped ... the design of the the plotting library interface was not thought through well enough - if one only has a wheel which is clickable as the middle button, then one has to be paranoid that the wheel is not rotated the tiniest bit while doing the marquee select; if a rotation is detected, at all, then the select operation is terminated  ... so, if one is being casual, or a bit tired, or just normally human - then you will end up swearing at the software ... .

 

Maybe the design can be improved if the mouse has multiple scroll wheels and a few more buttons?

 

[fas42]

Hmmm, still not sure why the action is not completely glitch free - I thought that it was the addition of wheel rotation in the movement; but if I deliberately do excessive rotation while selecting it behaves fine. For some reason if I use excessive force to hold down the wheel it's more reliable - but perhaps it's because I am then more deliberate in how I go about doing it.

 

The upshot is that marquee selection is now unreliable enough to put me slightly on edge every time I think about using it - just at that point where one starts to feels uncomfortable with the interface ... will investigate ways of using other keypresses, etc, as a substitute.

[pkane2001]

17 hours ago, fas42 said:

Hmmm, still not sure why the action is not completely glitch free - I thought that it was the addition of wheel rotation in the movement; but if I deliberately do excessive rotation while selecting it behaves fine. For some reason if I use excessive force to hold down the wheel it's more reliable - but perhaps it's because I am then more deliberate in how I go about doing it.

 

The upshot is that marquee selection is now unreliable enough to put me slightly on edge every time I think about using it - just at that point where one starts to feels uncomfortable with the interface ... will investigate ways of using other keypresses, etc, as a substitute.

 

Just so I understand, the issue is with selecting a section of the graph for zooming in using the scroll wheel, correct?

 

Using this on a Mac is a bit different, since I simulate the 'third button press' using the Apple logo key, and then just drag the mouse around the area I want to zoom while pressing the left mouse button. That seems to work very consistently, as long as I press the Apple key first. If I start dragging the mouse first, and press the key down later, this doesn't work.

 

My guess is that your scroll-wheel is sending some scroll commands prior to sending the button-down event, and the logic in the library is interpreting this as a scroll event rather than a button-down followed by scroll event.

[fas42]

6 hours ago, pkane2001 said:

 

Just so I understand, the issue is with selecting a section of the graph for zooming in using the scroll wheel, correct?

 

Using this on a Mac is a bit different, since I simulate the 'third button press' using the Apple logo key, and then just drag the mouse around the area I want to zoom while pressing the left mouse button. That seems to work very consistently, as long as I press the Apple key first. If I start dragging the mouse first, and press the key down later, this doesn't work.

 

My guess is that your scroll-wheel is sending some scroll commands prior to sending the button-down event, and the logic in the library is interpreting this as a scroll event rather than a button-down followed by scroll event.

 

The mouse has a scroll wheel which is pushed down and held to trigger the middle button switch - and it's quite easy to send a wheel rotation signal - which zooms the whole plot - as one does this, by not being absolutely precise about it. Which is what I thought the problem was; except that when I did this in an exaggerated way, things behaved correctly. Perhaps a timing related situation?

 

What I find curious is that in previous months I never noticed section selection as being as unreliable as it is now - so is it really my laptop misbehaving, very slightly? The latter is strongly indicated by my going back to version 30b, and it also playing up.

 

If no easy answers, I'll set up some key press capture software to do like you do with the Mac - I'm after reliability, so that misbehaving doesn't interrupt the activity flow.