“Gaming” measurements

[March Audio]

32 minutes ago, idiot_savant said:

Hi,

 

trying to get back on topic…

apologies if I’ve not been clear, so here’s a little bit of history for context:

Back in the day, SNR was king - so you had an AP, and a DAC. So to measure SNR, measure the biggest signal, then turn off the signal and measure what’s left. 
now, some DAC manufacturers “gamed” this measurement by detecting digital silence and effectively performed an analog mute when they detected it - now, the SNR ratio can be improved just by having more gain for a bigger number, but the product itself is not any better, indeed it is possibly worse with more gain. 
So, realising this, AP and the AES ratified a new test, called the “Dynamic Range Test”. In this test, you feed in a signal at -60dBFS and measure how far the noise floor is below this, add the 60dB back in and you get a nice number telling you the Dynamic Range. 
So, if you look on the internet for “Sabre DRE data sheet” you can find the data sheet for the 9219. If you search this for DRE, there is a feature that can be enabled that deliberately monitors the *input* signal and modifies the *analog output gain* by up to 30dB - so for our test bench, it can detect it’s being tested, and reduce the analog output of the DAC whilst compensating internally, potentially gaining 30dB in measurement terms, yet subjectively a DAC mucking about with an amplifier will be objectionable. 

Now I don’t know if anybody does this in a product, but from the feedback so far, somebody probably does… and the fact it’s a real feature in the chip…
 

does this help explain the problem?

 

Your friendly neighbourhood idiot 

That won't work. I will look at it but I'm fairly sure whatever this feature is It doesnt do what you think.

 

I get no search results for "sabre dre data sheet".
 

Increase the gain and you will increase the noise floor. It would be obvious in tests such as an FFT or sinad level sweep.

 

The "Mute" trick did happen but was quickly discovered.

 

 

 

 

[semente]

38 minutes ago, idiot_savant said:

now, some DAC manufacturers “gamed” this measurement by detecting digital silence and effectively performed an analog mute when they detected it - now, the SNR ratio can be improved just by having more gain for a bigger number, but the product itself is not any better, indeed it is possibly worse with more gain. 
So, realising this, AP and the AES ratified a new test, called the “Dynamic Range Test”. In this test, you feed in a signal at -60dBFS and measure how far the noise floor is below this, add the 60dB back in and you get a nice number telling you the Dynamic Range.

 

Can't you name the DACs in question?

 

I have the 1kHz @ -60sBFs THD+N measurement of my 30 year old CD player, how does the "add the 60dB back" work?

[March Audio]

22 minutes ago, semente said:

 

Can't you name the DACs in question?

 

I have the 1kHz @ -60sBFs THD+N measurement of my 30 year old CD player, how does the "add the 60dB back" work?

The issue revolves around the distortion added by the dac / measurement system.  A -60dBFS signal is low enough that it puts all the distortion harmonics below the noise floor.  It will also defeat the mute trick.  Add 60 dB and you get the theoretical signal to noise ratio.

 

Note you are measuring thd + noise (sinad) in the plots above not snr.

[semente]

1 hour ago, March Audio said:

Of course.  However this is the objective section so you need to justify why any one particular person's preference is better, or in this case why you think reducing technical quality improves subjective quality.

 

....and I dont support the "anything goes/anything is good as long as someone somewhere likes it" dictatorship. 

 

As a simplistic argument, do you think an amp with 10% thd sounds better than one with 0.1% THD?

 

I don't support relativising measured performance either. But if you look at the Ayre measurements you will see a tiny bit of "euphonic distortion" which is only there because the engineers wanted to, not due to incompetence.

 

The thing about what sounds good to me is that it's only relevant to me.

I see the people deliberately using ≤10W SET amps with low-sensitivity small standmounts. Because they tried other options and preferred the 10% THD...

[idiot_savant]

Right, so thought experiment time: 

you’ve got a DAC with a -100dB noise floor at 0dBFS - DNR 100dB

 

reduce the signal by 30dBFs, noise floor stays where it is, so signal -30 noise -100 so 70dB SNR but we add back in 30 for DNR - 100dB

 

DAC detects -30, adds 30dB digital gain but reduces analogue gain by -30dB. 
Analog gain moves down signal AND noise, so signal -30, noise -130, DNR is -130…

assuming the analogue gain is perfect in this case 

 

now under test conditions, this is a steady state condition so nothing will show up in measurements. 

 

 

Your friendly neighbourhood idiot 

ES9219_Datasheet_v1.0.4.pdf

[idiot_savant]

Oh, and as a proof of concept ( adjusting analog digital gains to get best measurements ) - how do you think an AP works?

 

your friendly neighbourhood idiot 

[March Audio]

19 minutes ago, semente said:

 

I don't support relativising measured performance either. But if you look at the Ayre measurements you will see a tiny bit of "euphonic distortion" which is only there because the engineers wanted to, not due to incompetence.

 

The thing about what sounds good to me is that it's only relevant to me.

I see the people deliberately using ≤10W SET amps with low-sensitivity small standmounts. Because they tried other options and preferred the 10% THD...

I would refer to that as incompetence.

 

You are reinforcing my point.  Any one individuals random preferences (for high distortion in this case) can't be a reason to judge the performance of a piece of equipment.

[March Audio]

22 minutes ago, idiot_savant said:

Right, so thought experiment time: 

you’ve got a DAC with a -100dB noise floor at 0dBFS - DNR 100dB

 

reduce the signal by 30dBFs, noise floor stays where it is, so signal -30 noise -100 so 70dB SNR but we add back in 30 for DNR - 100dB

 

DAC detects -30, adds 30dB digital gain but reduces analogue gain by -30dB. 
Analog gain moves down signal AND noise, so signal -30, noise -130, DNR is -130…

assuming the analogue gain is perfect in this case 

 

now under test conditions, this is a steady state condition so nothing will show up in measurements. 

 

 

Your friendly neighbourhood idiot 

ES9219_Datasheet_v1.0.4.pdf 1.78 MB · 0 downloads

As you mention you won't have perfect analogue gain plus distortion components would change.  It would be obvious in an fft, sinad level sweep or linearity test.

 

I suppose you could be caught out if you just do a very basic snr test in isolation, but not if you performed comprehensive measurements.  The "gamed" snr numbers wouldn't correlate with what you see elsewhere. 

[idiot_savant]

Why would it show up in an FFT? It’s not done on a per-sample, there are even programmable attack and release characteristics - so the DAC waits for the signal to be at a steady level before changing gains. 

The fact that this chip quotes DNR with this feature enabled, implies, does it not that this is what’s happening?

 

As for Ayre, from memory this is a DAC reconstruction filter thing, so it’s a perfectly valid approach to have more than one setting

 

your friendly neighbourhood idiot 

[March Audio]

Just now, idiot_savant said:

Why would it show up in an FFT? It’s not done on a per-sample, there are even programmable attack and release characteristics - so the DAC waits for the signal to be at a steady level before changing gains. 

The fact that this chip quotes DNR with this feature enabled, implies, does it not that this is what’s happening?

 

As for Ayre, from memory this is a DAC reconstruction filter thing, so it’s a perfectly valid approach to have more than one setting

 

your friendly neighbourhood idiot 

The noise floor would change as the gain isn't perfect.  Plus if you test levels away from -60dB the floor levels would again be different.

 

It's only potentially effective if you perform no other testing than the single snr test.

[idiot_savant]

Oh, and I’ll repeat - how does an AP measure things? If your assertion is correct about noise and distortion changing how does an AP not suffer from this?

 

your friendly neighbourhood idiot 

[March Audio]

4 minutes ago, idiot_savant said:

Oh, and I’ll repeat - how does an AP measure things? If your assertion is correct about noise and distortion changing how does an AP not suffer from this?

 

your friendly neighbourhood idiot 

What makes you think the AP doesn't suffer?

 

It has gain ranging signal conditioning to optimise noise and distortion levels for a given signal input level.

 

Also for info, take a look through the data sheets of op amps and see noise and distortion changes with gain level.

[idiot_savant]

Look,

I’ve provided the data sheet that talks about the registers, quotes measurements, describes the feature. 
You haven’t bothered looking, and continue to make statements that make me believe you’ve never used an AP…

 

if you’re going to continue, please have the good grace to look at the data sheet

 

your friendly neighbourhood idiot 

[March Audio]

24 minutes ago, idiot_savant said:

Look,

I’ve provided the data sheet that talks about the registers, quotes measurements, describes the feature. 
You haven’t bothered looking, and continue to make statements that make me believe you’ve never used an AP…

 

if you’re going to continue, please have the good grace to look at the data sheet

 

your friendly neighbourhood idiot 

My background is test and measurement.  I have represented TM companies such as Agilent so please let's not go down the "you don't know what youv are talking about" road.

 

I've said that the feature, if used nefariously just to detect a SNR test as you allude to,  would be quite detectable so long as you do more than a basic snr test.

 

The fact that you don't know the reasons why tells me of your limited TM/electronics experience.

[March Audio]

Right so refreshed my memory on ESS chips.  You are refering to the dynamic range enhancement feature.

 

You have indeed misunderstood it's purpose and effect as I previously thought.

[idiot_savant]

Well, I’m frankly not interested in going down the “I know more than you path”, but “representing” is quite vague… so please just explain how DRE works, and what it’s meant to do

 

your friendly neighbourhood idiot 

 

 

[idiot_savant]

From the datasheet

 

 

[March Audio]

21 minutes ago, idiot_savant said:

From the datasheet

 

 

Yes  whats your point?

 

It's to optimise the analogue gain/digital gain levels for highest dynamic range dependant on signal level.

 

Why have you interpreted that as an snr test "gaming" feature?

 

Can you show a dac product which uses this chip and shows unduly high snr levels that are inconsistent with other test data?

[idiot_savant]

So here we have a feature that’s behaving exactly as I described - monitoring the input and adjusting gains on the fly, which you assured me was a terrible thing to do - which I agree with for *sound quality* purposes, but is *specifically* for *measurement* purposes. What do you think “optimise” means in this context?

How is this not gaming the measurement? If you can’t see this, I can’t help you. 
 

the point of this thread was, is this something to get upset about? Is it a feature to sell this chip, or is it something that manufacturers might enable in real product?

 

your friendly neighbourhood idiot 

[March Audio]

20 minutes ago, idiot_savant said:

So here we have a feature that’s behaving exactly as I described - monitoring the input and adjusting gains on the fly, which you assured me was a terrible thing to do - which I agree with for *sound quality* purposes, but is *specifically* for *measurement* purposes. What do you think “optimise” means in this context?

How is this not gaming the measurement? If you can’t see this, I can’t help you. 
 

the point of this thread was, is this something to get upset about? Is it a feature to sell this chip, or is it something that manufacturers might enable in real product?

 

your friendly neighbourhood idiot 

I did no such thing.  I previously explained that Performance will change with amp gain.  There will be optimal dac output levels / analogue gain levels.  Optimising these are not "gaming".  It's not cheating and is quite relevant to the subjective performance as it is to the measurements.  It could be used, I don't know if any products do.  It's not anything to be concerned about.

 

 

[idiot_savant]

Erm,

 

I’ve found the datasheet, quoted you the exact section showing how it behaved exactly as I described. 
 

changing the gain depending on the input signal is exactly the thing I was talking about. The thing you said would easily show up on an FFT. 
 

you have done nothing but vague comments about noise and distortion. The feature is called Dynamic Range Enhancement - to game a measurement, no more, no less. 
so what *is* it for if not to game measurements? Music is far from steady state

 

your friendly neighbourhood idiot 

[idiot_savant]

Oh,

and to go back to car analogies - VW got in trouble for “optimising” the engine behaviour under test conditions  ( by monitoring the inputs )

 

your friendly neighbourhood idiot 

 

 

[March Audio]

2 minutes ago, idiot_savant said:

Oh,

and to go back to car analogies - VW got in trouble for “optimising” the engine behaviour under test conditions  ( by monitoring the inputs )

 

your friendly neighbourhood idiot 

 

 

No VW got into trouble for deception.  Cheating the emissions test to show better performance than the car could give when operating normally.

 

DRE is not deception, it's making the dac provide better performance.  It's applicable to normal usage.

[March Audio]

21 minutes ago, idiot_savant said:

Erm,

 

I’ve found the datasheet, quoted you the exact section showing how it behaved exactly as I described. 
 

changing the gain depending on the input signal is exactly the thing I was talking about. The thing you said would easily show up on an FFT. 
 

you have done nothing but vague comments about noise and distortion. The feature is called Dynamic Range Enhancement - to game a measurement, no more, no less. 
so what *is* it for if not to game measurements? Music is far from steady state

 

your friendly neighbourhood idiot 

No I said it would show up if it were used as you alluded to to specifically detect and cheat a -60dB snr test. I also have been quite specific.

 

You need to understand volume control.  If you use digital volume control you lose SNR.  There comes a point where it's better to use analogue gain control for optimal performance.

 

This has nothing to do with cheating measurements.

[idiot_savant]

Erm, so there’s already an analog volume control available in this chip - 
DRE is about input levels, not output levels. As I stated in pretty much my very first post the benign view of this is that you have an upstream volume control, but as you say, you shouldn’t use that if you’ve got an analog one downstream. 
 

And the VW analogy stands - why would you want the analog gain to change with the *input* when listening? Assuming the input is full scale? Would you at least agree that this feature could be used to game a DR measurement?

 

your friendly neighbourhood idiot