Help me to understand filters

[Miska]

3 hours ago, manisandher said:

1. 24/176.4 (original) to 24/44.1_sinc-M to 24/176.4_FUT* -> null against original

 

I used sinc-M for the downsample stage in all cases here, as I wanted as much HF content preserved in the downsampled file as possible. This process produces the nulls shown above.

 

2. 24/44.1 (original) to 24/176.2_FUT* to 24/44.1_sinc-M -> null against original

 

 

That explains part of it, since sinc-M is still apodizing filter, giving somewhat wrong result with for example minringFIR or any other half-band filter. In fact you cannot have a correct result with this type of test with half-band filters. poly-sinc-xtr-lp could get you a little closer.

 

3 hours ago, manisandher said:

With this process, I can't get anywhere near a null with any of the HQP filters. As I mentioned in an earlier post, the 24/44.1 original files I used (both pink noise and music) have a lot of content right up to 22.05 kHz. (The Audacity filter happily nulls to <-200dB in the pass band with these files.)

 

That just gives incorrect results with something like minringFIR-lp or polynomial near Nyquist of the lower rate. Because pass-band comes from the other filter.

 

 

Anyway, for linear phase filters (minringFIR-lp and polynomial, although polyomial is not even a filter but instead interpolator) the non-perfect nulls are due to frequency response difference (amplitude vs frequency). These are more like side-effects of those filters due to other (time domain) design goals. For minimum phase filters the non-perfect null being due to minimum-phase response which is totally intended side effect. So two different reasons.

 

[Miska]

3 hours ago, John Dyson said:

2) Actual music. someone took the care to provide wide bandwidth to the consumer.

3) Distortion products as created by signal processing (of any type.)

 

(2) would be combined with (3), aliasing in this case always. Because you cannot have a decimation filter that passes content exactly up to Nyquist frequency while not aliasing at all.

 

Modern ADC decimation filters go flat or almost up Nyquist and stop-band begins only after 0.5 frequency. Meaning you can see the amount of this high frequency hash they have from the 0.5 to beginning of stop-band response. So if the stop-band starts at 0.55, then 10% of the bandwidth will have this distortion.

 

In addition they have typically fairly small stop-band attenuation of 120 dB, so they will also reach at max about 20-bit resolution. Same goes for DAC chip filters too. On DAC side you can find even lower figures though.

 

[John Dyson]

3 minutes ago, Miska said:

 

(2) would be combined with (3), aliasing in this case always. Because you cannot have a decimation filter that passes content exactly up to Nyquist frequency while not aliasing at all.

 

Modern ADC decimation filters go flat or almost up Nyquist and stop-band begins only after 0.5 frequency. Meaning you can see the amount of this high frequency hash they have from the 0.5 to beginning of stop-band response. So if the stop-band starts at 0.55, then 10% of the bandwidth will have this distortion.

 

In addition they have typically fairly small stop-band attenuation of 120 dB, so they will also reach at max about 20-bit resolution. Same goes for DAC chip filters too. On DAC side you can find even lower figures though.

 

 

I agree 100% that on digital material, the various artifacts can become noticeable - but it is the noise modulation that I see most often.   (As the gain increases on an NR system, it opens up a time window at frequencies other than just the exact signal frequency.)   This is easy to understand with DolbyA, where there is a 9kHz to 20+kHz band, so that any material sitting at 12kHz (like cymbals) will also open up the gain for the tape hiss that is attempted to be hidden.   It is a short timeframe, and the sound of the cymbals does overwhelm the tape hiss.   One reason why DolbyA was sooo very much better than DBX is that the multi-band design was really good.   DBX would open up the hiss even on a BASS note, even though they tried to mitigate it by some aggressive EQ.   Dolby SR takes the A approach several steps further, but the FA encoding as used on most CDs and most digital downloads is probably more extreme  than anything that I have ever seen.  (6 or 7 four band compressor/expanders, each at different signal levels, and also layered in a russian doll scheme.)   Some of the left over latent compression from the unbiquitious scheme used on almost all consumer distribution copies do make the noise modulation more obvious, simply because of the additional compression at HF effectively increasing the hiss level.

 

The digital effects do/can happen, but the NR/dynamics processing noise modulation is strongly prominent on material recorded/sourced with analog tape.   It is a HEINOUS problem, and very deceptitve to those who might actually believe that they are seeing much 'music' above 20kHz on a spectogram.   There is sometimes SOME music above 20kHz, but ignoring totally uncorrelated noise sources, NR noise modulation on analog sourced material is usually the prominent signal above 20kHz.

 

John

 

[Miska]

20 minutes ago, John Dyson said:

I agree 100% that on digital material, the various artifacts can become noticeable - but it is the noise modulation that I see most often.

 

I'm not seeing noise modulation because the sources are newly recorded straight with ADC and of course no Dolby or similar involved. From spectrum analysis you can see it is the signal plus it's aliasing products, since typically the content will hit the Nyquist of RedBook pretty hard.

 

22 minutes ago, John Dyson said:

but the NR/dynamics processing noise modulation is strongly prominent on material recorded/sourced with analog tape

 

I don't think there is so much new material recorded within last ten years on analog tape...

 

[John Dyson]

Just now, Miska said:

 

I'm not seeing noise modulation because the sources are newly recorded straight with ADC and of course no Dolby or similar involved. From spectrum analysis you can see it is the signal plus it's aliasing products, since typically the content will hit the Nyquist of RedBook pretty hard.

 

 

I don't think there is so much new material recorded within last ten years on analog tape...

 

Yes, in the case of the situation that you are talking about -- true.

 

There is so much material, though, until the 1990 timeframe that has the NR hiss modulation.
I am just saying to others, don't be confused about the various sources.   The hiss modulation is VERY VERY profound, very

obvious on high res remasters.)

 

If the aliasing products on new stuff are as bad as the hiss modulation, then there is some really ham handed design going on.

 

John

 

[Miska]

4 minutes ago, John Dyson said:

If the aliasing products on new stuff are as bad as the hiss modulation, then there is some really ham handed design going on.

 

It is pretty bad, but of course not as bad as almost all content driven heavily into digital clipping thanks to loudness wars.

 

Of course another source of hiss modulation you now get is MQA...

 

In any case, if the source is proper, no harm is done with apodizing filters. If the source content is not proper, apodizing filters can help fixing various issues...

[manisandher]

1 hour ago, Miska said:

That just gives incorrect results with something like minringFIR-lp or polynomial near Nyquist of the lower rate. Because pass-band comes from the other filter.

 

So should I expect to get a null with the following:

 

24/44.1 (original) to 24/176.4_non-apodizing-lp to 24/44.1_non-apodizing-lp ?

 

Mani.

[John Dyson]

10 minutes ago, Miska said:

 

It is pretty bad, but of course not as bad as almost all content driven heavily into digital clipping thanks to loudness wars.

 

Of course another source of hiss modulation you now get is MQA...

 

In any case, if the source is proper, no harm is done with apodizing filters. If the source content is not proper, apodizing filters can help fixing various issues...

Interesting about MQA.   As long as they don't normalize the albums, and they are starting with the FA content that is so prevalent today -- then using the DHNRDS FA decoder can hide the added noise.   FA decoding does WONDERS on removing hiss from a lot of the older recordings, and would also work on newer recordings, often are also encoded with the FA encoding mechanism.   (Taylor Swift -- Shake it off, and Carly Rae Jepsen, Call Me Maybe are both compressed with FA...)   So, the FA thing isn't dead, but has little to do with tape noise.   FA is just a way of skipping the need for mastering, and is a common 'sound' of digitial recordings nowadays.

 

What FA means, if used with an un-normalized MQA recording, is that the noise from MQA might be at least partially removable.

 

John

 

[pkane2001]

1 hour ago, manisandher said:

 

So should I expect to get a null with the following:

 

24/44.1 (original) to 24/176.4_non-apodizing-lp to 24/44.1_non-apodizing-lp ?

 

Mani.

 

Mani, for non-linear phase filters, give this setting a try in DeltaWave (assuming your track is a minute or more in length, if shorter, reduce the size of the FFT window):

 

 

This can help correct both phase and amplitude errors that result from non-linear filters, including variable group delay. See if the null in DeltaWave will get better. I often see an improvement of about 2x on the null result when using this with some real DAC/ADC captures. 

 

[Miska]

2 hours ago, manisandher said:

So should I expect to get a null with the following:

 

24/44.1 (original) to 24/176.4_non-apodizing-lp to 24/44.1_non-apodizing-lp ?

 

No, you are going to wrong direction. I'm not sure what you are looking for. You will just get different kind of errors. But for down conversion step with half-band filters you may get better results with poly-sinc-xtr-lp.

 

minringFIR is a half-band filter and there's no way you can get correct result with such if you do upsample with such, followed by downsample.

 

I can understand this can be sort of fun exercise if you know how to interpret the results and also want to assume the source data you begin with is correct in respect to analog signal that entered ADC to begin with, assuming your ADC analog and digital filters, and production chain is perfect. For me the more interesting are the many cases where that is not the case. In such case fixing the problems obviously requires also changing the signal. In addition, what your test doesn't look at all is what comes (or doesn't) come out of the upsampling filter beyond Nyquist of the source, because on down-conversion you are removing all that again.

 

[Miska]

2 hours ago, John Dyson said:

What FA means, if used with an un-normalized MQA recording, is that the noise from MQA might be at least partially removable.

 

Remember that MQA adds it's noise shaped scrambled mess at the final encoding stage, so it will just mess all the other stuff that is there. So it is just a process that adds random looking shaped noise.

 

I've just designed an upsampling filter to clean that up. It also works very well on hires recordings that have either hiss or just noise-shaping noise left overs from the ADC. Since >= 176.4k PCM output of most ADCs look just like DSD128 converted to PCM.

 

[John Dyson]

33 minutes ago, Miska said:

 

Remember that MQA adds it's noise shaped scrambled mess at the final encoding stage, so it will just mess all the other stuff that is there. So it is just a process that adds random looking shaped noise.

 

I've just designed an upsampling filter to clean that up. It also works very well on hires recordings that have either hiss or just noise-shaping noise left overs from the ADC. Since >= 176.4k PCM output of most ADCs look just like DSD128 converted to PCM.

 

Since many original digital recordings, even new ones (except for special 'elite' versions) do have the FA compression, then decoding the FA can remove much additive noise that was added to the FA encoded original version.   It isn't so good at nonlinear type noise, but simple additive noise, it is much better than a straight DolbyA, probably doing more than 20dB NR.

 

If the MQA actually start producing REAL consumer recordings that don't have the FA compression (very unlikely), then the FA decoder doesn't do anything good.

 

John

 

[Miska]

8 hours ago, manisandher said:

Everything remains pure D-to-D in both processes.

 

By the way, your captures are 24-bit, not 32-bit?

 

[Miska]

For example here's digital filter response for AK5572 ADC chip at 48k sampling rate:

 

This a typical on-chip half-band filter. You can see the filter transitions between 24 and 28 kHz. Meaning it will have 4 kHz wide aliasing band, thus between 20 kHz and 24 kHz. Signals above 28 kHz are attenuated 85 dB, so it is fully accurate in pass-band to about 14.5 bit resolution. Good example of modern ADC that needs some fixing at playback stage.

 

This filter would look somewhat similar to minringFIR-lp in the null test, but since the pass-band and stop-band ripple is flat throughout the frequency band it would have higher and horizontal pass-band difference at about -100 dB, unlike minringFIR-lp where it is mostly close to Nyquist:

 

Slope here follows the increasing pass-band ripple. The step at the end is from the sinc-M filter:

 

Quote

 

 

 

While if we look at frequency response:

And pass-band ripple:

Of sinc-M filter it is all totally different. I actually got bored zooming in the pass-band ripple plot at that point, it being less than 0.00000000001 dB.

 

 

Now the question is if the latter is "perfect"? Or if the truth is somewhere in the middle? I personally don't use either one of the two. If you use minringFIR-mp (non-apodizing) or poly-sinc-mqa-mp (apodizing) filter, you get something that is closer to what MQA says. And if you use sinc-M (apodizing) or sinc-L (non-apodizing) you get something that is closer (or actually beyond) what Chord says (since sinc-M has way more attenuation than the Chord filter). My personal take is that they are both partially right, but just rule out the other argument. But also different people are particularly sensitive to different aspects, so there's not even one right answer to this. That is why I prefer to offer various different approaches. It is also educating because you can switch while keeping everything else the same.

 

What you cannot do much about is the source content. So for that, we need to come up with various ways of dealing with it's inherent baked-in properties.

 

 

P.S. You can also compare yourself whom you think is more correct. For me, sinc-S is sort of "sinc-sane" version, and poly-sinc-ext2 and poly-sinc-short-mp do the rest. Depending on source content.

 

[manisandher]

8 hours ago, Miska said:

Modern ADC decimation filters go flat or almost up Nyquist and stop-band begins only after 0.5 frequency. Meaning you can see the amount of this high frequency hash they have from the 0.5 to beginning of stop-band response. So if the stop-band starts at 0.55, then 10% of the bandwidth will have this distortion.

 

In addition they have typically fairly small stop-band attenuation of 120 dB, so they will also reach at max about 20-bit resolution. Same goes for DAC chip filters too. On DAC side you can find even lower figures though.

 

2 hours ago, Miska said:

For example here's digital filter response for AK5572 ADC chip at 48k sampling rate:

 

This a typical on-chip half-band filter. You can see the filter transitions between 24 and 28 kHz. Meaning it will have 4 kHz wide aliasing band, thus between 20 kHz and 24 kHz. Signals above 28 kHz are attenuated 85 dB, so it is fully accurate in pass-band to about 14.5 bit resolution. Good example of modern ADC that needs some fixing at playback stage.

 

2 hours ago, Miska said:

What you cannot do much about is the source content. So for that, we need to come up with various ways of dealing with it's inherent baked-in properties.

 

Thanks!

 

Everything here is exactly what I was hoping from this thread.

 

Mani.

 

 

[manisandher]

3 hours ago, Miska said:

By the way, your captures are 24-bit, not 32-bit?

 

Captures are 24-bit and the nulls are 32-bit. I tried 32-bit captures, but they didn't seem to improve my nulls, so I just assumed HQPlayer must be outputting 24 bits. Also, using dither worsened the nulls ever-so-slightly.

 

Mani.

[manisandher]

4 hours ago, Miska said:

I can understand this can be sort of fun exercise if you know how to interpret the results and also want to assume the source data you begin with is correct in respect to analog signal that entered ADC to begin with, assuming your ADC analog and digital filters, and production chain is perfect.

 

I thought it'd be interesting to take a well-recorded hires file (Reference Recordings, in my case), listen to it natively, and then compare what I heard to an equivalent redbook being upsampled to the same rate. (My DAC allows for a true apples-to-apples comparison here, because it won't perform any additonal processing.) The best way seemed to be to downsample the hires myself to redbook, so that's what I did. It was interesting hearing the effects various filters had on the sound. I then wondered if I could null the upsampled redbook against the hires to determine which filter was actually the most accurate, and whether this correlated with what I was hearing.

 

Until this thread, it never even occurred to me that maybe there was an opportunity to use filtering to improve on the source file perhaps. It's been very educational. Thanks.

 

Mani.

[manisandher]

6 hours ago, pkane2001 said:

Mani, for non-linear phase filters, give this setting a try in DeltaWave (assuming your track is a minute or more in length, if shorter, reduce the size of the FFT window):

 

 

This can help correct both phase and amplitude errors that result from non-linear filters, including variable group delay. See if the null in DeltaWave will get better. I often see an improvement of about 2x on the null result when using this with some real DAC/ADC captures. 

 

 

Thanks Paul. I'll give this a go at some point.

 

Mani.

[Miska]

53 minutes ago, manisandher said:

Captures are 24-bit and the nulls are 32-bit. I tried 32-bit captures, but they didn't seem to improve my nulls, so I just assumed HQPlayer must be outputting 24 bits. Also, using dither worsened the nulls ever-so-slightly.

 

If HQPlayer detects that target device is 24-bit, or looks like S/PDIF or AES/EBU or similar max 24-bit device, it enforces limit at 24-bit. You cannot get past the detected maximum resolution with the settings.

 

45 minutes ago, manisandher said:

It was interesting hearing the effects various filters had on the sound. I then wondered if I could null the upsampled redbook against the hires to determine which filter was actually the most accurate, and whether this correlated with what I was hearing.

 

Yes, it is all valid. The nulling just gives one kind of answer.

 

You need to note here that the algorithm used for the RedBook conversion matters (that's one reason HQPlayer Pro exists). Same way as the ADC filter matters too. And then later also the upsampling algorithm matters.

 

[manisandher]

@PeterSt, there's something that's really bothering me...

 

Using the method I've employed to date (take a 24/176.4, downsample to 24/44.1 in a 'correct' way, upsample back to 24/176.4 using FUT, and then null against original), it seems using a polynomial filter creates a fundamental issue in the passband that I'm finding very hard to accept, now that I know about it.

 

For polynomial filters, the slight attenuation in the passband and the imaging beyond Nyquist are both to be expected. I've accepted the argument that these are worthwhile trade-offs in ordet to gain better transient performance within the passband. However, my recent 'inquiries' suggest to me that the attenuation in the passband might not as harmless as I had imagined. (I'm not going to discuss the imaging here.)

 

I knew that the polynomial filter attenuation in the passband was a non-linear function of frequency (the null gets larger with frequency). What I didn't know until now was that the attenuation is a function of signal amplitude too. When the signal reduces, by 3dB say, the attenuation reduces (the null gets smaller), in a non-linear way with frequency:

 

 

(It may be that the attenuation is a non-linear function of amplitude too, but I didn't look into this.)

 

How does this manifest itself?

 

With a regular filter, there is no correlation of the null with music in the passband:

 

 

(Or if there is anything there, it's below -180dB at least. So any 'ringing' caused by the filter will also be at this level.)

 

With a polynomial filter, there is a clear correlation of the null with music in the passband:

 

 

Surely, this is distortion, plain and simple? The filter is adding to (or in this case, taking away from) the signal in a manner correlated to the signal. The music here is from Reference Recordings, and has an RMS value of -25dB, so pretty low compared to most stuff. And yet look at how the null correlates with the music in the passband. This is going to be easily audible and will distort the signal beyond anything any 'ringing' could accomplish... no?

 

So, how can the supposed benefits of using a non-ringing filter, like a polynomial, be reconciled with the distortion it obviously adds in the passband?

 

I'd be very happy if you (or anyone else) could disavow me of my new way of looking at things...

 

(Thanks again to Paul for making his Deltawave programme available to anyone.)

 

Mani.

[PeterSt]

1 hour ago, manisandher said:

I knew that the polynomial filter attenuation in the passband was a non-linear function of frequency (the null gets larger with frequency). What I didn't know until now was that the attenuation is a function of signal amplitude too.

 

Hi Mani,

 

To me (and my English) it seems that the remainder of your post is about that first sentence only, and is thus what we all expect. And because I don't see how the remainder of your post is about the second sentence, there is not much to work out.

 

Of course I see that you have set up a -0dBFS and a -3dBFS signal, but the relation between the two looks linear to me.

 

 

Even that part still looks linear. So I wonder whether you went off-track on the purpose of your (last) post; I probably won't get it.

 

The higher the frequency, the higher the distortion (for such a filter). But notice that somehow I am missing the roll off in your first picture (it can also be that the roll off goes all the way towards 0Hz and that the whole plot line(s) are thus part of the "not-null" to begin with).

 

Anyway, I don't see as of yet how the distortion would increase towards the high frequencies with more attenuation (this is what you seem to hint at). Otoh, it should be logic that there's more distortion at the lower levels because there's less bits to put in the so small space for bits anyway (now think 2 available samples for 20KHz anyway, which 2 samples also need to find their way for a decent amplitude without error, which in a space of 16 bits and 32767 possibilities (for plus and for minus individually) incurs for error in itself when it's given 8 times less that space (~ -25dB), thus 4096 possibilities only. So in my view when 20KHz allows for 1 sample point for positive value and 1 sample point for negative value, with 4 times less the space, one of those may be missing at all. Surely this would be a very different amount of (additional) distortion than e.g. a 50Hz wave which will also have 4 times less amplitude possibilities, but for which there are 1311 to begin with (I think 32767 (positive voltage) / 25 (half of the cycle for positive voltage only). Compare this with 32767 / 10000 (for 20KHz) and you may see my point.

So while the space of 32767 becomes 8192, the frequency of 20KHz remains the same and 8192/10000 is now 0.8192 or below 1 ... is too short/few.

 

So if that was meant to be your point, then you are granted to have it ... as I see it now. Sort of.

 

Sort of, because in real life (not NOS) the attenuation happens after upsampling, and for now outside of the more samples to projected sample points, the bit space (= space for the level) is 256 times larger.

At this moment I can not fully justify the latter, because it kind of implies that 16 times upsampling allows for 16 times more frequency as well. But, because you went from 176.4 to 44.1 all right, the frequency is definitely lost. Mind you, with the polynomial filter. All we can do is take care of the frequency distortion being as low as possible, which happens with the cascading with other filter means or other trickery. Remember the objective (Miska said it in between the lines): a better (transient) response in the time domain.

 

 

What I would like to see (because you are at it anyway) is the clear correlation of what you implied already and what I tried to make clear (I did not say "clear better" - haha). But for that, a signal (music) which has been attenuated to -25dBFS is not sufficient, *because* your DAC won't lose bits yet. So for a 24 bit DAC your base level should be -48dBFS at least (going towards -144) and compare that with more attenuation. But then not with 3dBFS more, but something significant (say an additinal 15dB attenuation). Your goal:

 

 

let the lower line cross the upper line when going towards the right. Or at least let it go up progressively compared to the upper line.

 

Lastly, notice that the distortion towards the right hand side, could stay more linear by means of deliberate earlier roll off. I implied this at the beginning of this post. Thus, the roll off itself will imply a distortion when looking at the nulling towards the left hand side, while the attenuation of the real distortion (this is again the roll off now looking at the 20KHz side), will virtually *) let drop the null line towards the right.

I think it is important to see through this, because the nulling (test) will only show a difference, while my mentioned two types of distortion are totally different and will sound different from each other.

 

*): This is only virtual and can not be practice because all we'd to is take out the distorting frequencies (while we would not perceive those anyway *when* the frequencies would be in undistorted fashion). But you are not measuring THD here. You compare / null. So attenuating already cause error.

Might I have been able to bring this latter across, then you'd maybe understand that what shows as error on the nulling, sounds better (because higher frequencies not fully reconstructed, causing aliasing and what not what you *can* hear, have been taken out).

 

Bzz