Jump to content
IGNORED

“Gaming” measurements


Recommended Posts

Right, so this might be a bit inflammatory, but bear me out - I’m not going to mention anybody by name…

 

since time immemorial, manufacturers have sought advantage by better specs - who can forget soundcards with 144dB SNR, phones that magically sped up up running certain workloads, cars changing behaviour if GPS /= engine output?


In audio DACs, muting the output when the input signal is “0” has been a thing for many years, to such an extent that AP had to introduce a new measurement to take account of it ( the Dynamic Range measurement ).

 

How do people feel about this? Our technology has improved leaps and bounds, is there a danger “gaming” has improved? Does it matter?

 

 

 

Your friendly neighbourhood idiot 

Link to comment

I was doing research for a friend earlier today regarding a Porsche 911 and I learned that there is a stop sale going on, on used 991.1 gen cars with a specific option. The Sport Chrono pack. The Sport Chrono package comes with a sport button that when pressed changes a bunch of things in the car to make it sportier. It turns out that one of the things it also changed was the performance tune on the engine and in the Sport+ setting no longer passes emissions.

 

So yeah, I am sure that everyone and anyone is doing whatever they can to get a competitive advantage anywhere they can.

No electron left behind.

Link to comment
2 hours ago, AudioDoctor said:

I was doing research for a friend earlier today regarding a Porsche 911 and I learned that there is a stop sale going on, on used 991.1 gen cars with a specific option. The Sport Chrono pack. The Sport Chrono package comes with a sport button that when pressed changes a bunch of things in the car to make it sportier. It turns out that one of the things it also changed was the performance tune on the engine and in the Sport+ setting no longer passes emissions.

 

So yeah, I am sure that everyone and anyone is doing whatever they can to get a competitive advantage anywhere they can.

Volkswagon tried to fool the emission centers during testing. The government wasn,t to happy. No more diesels sold in the USA. 

Link to comment
34 minutes ago, Racerxnet said:

Volkswagon tried to fool the emission centers during testing. The government wasn,t to happy. No more diesels sold in the USA. 

 Yes, Porsche being a member of the Volkswagen group was caught up in that. However the 911 is not a diesel.

 

And I have already found a workaround the alleged "fix" which neuters the power of the car.

No electron left behind.

Link to comment
53 minutes ago, AudioDoctor said:

 Yes, Porsche being a member of the Volkswagen group was caught up in that. However the 911 is not a diesel.

 

And I have already found a workaround the alleged "fix" which neuters the power of the car.


“Neuters”?

One never knows, do one? - Fats Waller

The fairest thing we can experience is the mysterious. It is the fundamental emotion which stands at the cradle of true art and true science. - Einstein

Computer, Audirvana -> optical Ethernet to Fitlet3 -> Fibbr Alpha Optical USB -> iFi NEO iDSD DAC -> Apollon Audio 1ET400A Mini (Purifi based) -> Vandersteen 3A Signature.

Link to comment

Well, what brought this up for me was the Dynamic Range spec of a well respected DAC chip, which had a figure of xxdB ( can’t remember the precise number ) using “DRE”. So I looked at a data sheet, and this stands for Dynamic Range Enhancement and allows the DAC to dynamically alter the gain depending on the input, and it’s programmable. 
 

Now under some circumstances this could be a good idea ( so volume compensation ), but if taken to extremes, could be used to cheat at measurements to get better numbers whilst actually sounding worse ( as it would be changing gain all the time when music changes gain - think of a piece of music with quiet and loud passages )

 

I’m a bit surprised people here just seem to shrug their shoulders? Isn’t the idea of measurements to have a level playing field? Never mind sacrificing actual sound quality for a bigger number?

 

your friendly neighbourhood idiot 

Link to comment
5 hours ago, idiot_savant said:

Well, what brought this up for me was the Dynamic Range spec of a well respected DAC chip, which had a figure of xxdB ( can’t remember the precise number ) using “DRE”. So I looked at a data sheet, and this stands for Dynamic Range Enhancement and allows the DAC to dynamically alter the gain depending on the input, and it’s programmable. 
 

Now under some circumstances this could be a good idea ( so volume compensation ), but if taken to extremes, could be used to cheat at measurements to get better numbers whilst actually sounding worse ( as it would be changing gain all the time when music changes gain - think of a piece of music with quiet and loud passages )

 

I’m a bit surprised people here just seem to shrug their shoulders? Isn’t the idea of measurements to have a level playing field? Never mind sacrificing actual sound quality for a bigger number?

 

your friendly neighbourhood idiot 

I think there is a distinction between specs and measurements.

 

There has always been an issue around how specs are derived, different methods and parameters making it difficult to compare product performance.

 

Only real solution is for independent tests/measurements performed to a standard, but to get consensus on that would be nigh on impossible.  

 

In the US there is the FTC amplifier power rating test but that has no jurisdiction for the rest of the world.  It's also not very useful as the test parameters bear no resemblance to real world usage.

 

Otherwise I'm afraid it's down to the buyer to be as technically informed as possible to look out for important parameters - you know like what percentage distortion the amp power is rated at.

Link to comment

@March Audio - isn’t a spec a minimum guaranteed performance level for a product - e.g. all DACs model XX made by YY will have ndB SINAD, and then if I measure a sample it should meet that?

 

I think my point is, we could have DAC A & DAC B, both using exactly the same DAC IC, power supply, output stage but DAC A could have more aggressive DRE  settings than B, so measure better using the dynamic range test yet *sound* worse?

 

your friendly neighbourhood idiot 

Link to comment
4 hours ago, idiot_savant said:

@March Audio - isn’t a spec a minimum guaranteed performance level for a product - e.g. all DACs model XX made by YY will have ndB SINAD, and then if I measure a sample it should meet that?

 

I think my point is, we could have DAC A & DAC B, both using exactly the same DAC IC, power supply, output stage but DAC A could have more aggressive DRE  settings than B, so measure better using the dynamic range test yet *sound* worse?

 

your friendly neighbourhood idiot 

Well there is no formal guarantee, no legislation etc to enforce it.  For example in data sheets (let's say for an op amp) you will see "typical" values often along with min and Max. Manufacturing variations make it impossible for every item to be identical.  Then in your example the implementation of the DAC chip will affect its performance.  Pcb layout, surrounding circuit design and component choice.

 

The larger issue revolves around the parameters used in the measurement which you need to pay attention to.  The simplest example is amplifier power ratings.  At what distortion point do you take the reading?  1% is a typical value and possibly the most often used.  I would probably argue that 0.1% would be more appropriate.  However I have seen 10%!  Large differences in the power output between the 3.

 

Even with independent testing there is an an issue because of people doing different things.  A certain other Web site for example (audio “science” review 😉) takes power measurements at different distortion levels on every test! You can't even compare one amp to another on that sites tests.

 

So really unfortunately the reader needs to be technically aware and weed out the relevant information.

Link to comment
15 hours ago, idiot_savant said:

I’m a bit surprised people here just seem to shrug their shoulders? Isn’t the idea of measurements to have a level playing field? Never mind sacrificing actual sound quality for a bigger number?


I think many people have learned that measurements and specs can be used in nefarious ways, in every hobby, business, and endeavor. As such, they take the numbers as a single data point that doesn’t usually mean too much. 

Founder of Audiophile Style | My Audio Systems AudiophileStyleStickerWhite2.0.png AudiophileStyleStickerWhite7.1.4.png

Link to comment

I have to say I’m surprised by the “everybody cheats” shrugging of shoulders - I think there’s a distinction between being “optimistic” with specs and deliberately cheating at a well respected measurement that could be going on.
There’s no such thing as a free lunch - for example, those VWs that had the engine management systems “fixed” were reportedly much slower afterwards - the kind of gaming I’m talking about deliberately trades better measurement performance for poorer sound quality, that is important surely?

 

I’m fairly confident you could up with a scheme to see if this was happening, but if no one cares?

 

your friendly neighbourhood idiot 

Link to comment
8 minutes ago, idiot_savant said:

I have to say I’m surprised by the “everybody cheats” shrugging of shoulders - I think there’s a distinction between being “optimistic” with specs and deliberately cheating at a well respected measurement that could be going on.
There’s no such thing as a free lunch - for example, those VWs that had the engine management systems “fixed” were reportedly much slower afterwards - the kind of gaming I’m talking about deliberately trades better measurement performance for poorer sound quality, that is important surely?

 

I’m fairly confident you could up with a scheme to see if this was happening, but if no one cares?

 

your friendly neighbourhood idiot 

 

For some people measurements, or better measured performance, usually equates to better sound.

For many people measurements have no correlation with their listening impressions; in other words, measurements are meaningless to them (because they don't understand them and/or good measuring equipment may not sound good to them).

 

I find your question a very pertinent one but I have a feeling that it won't find much traction here.

 

 

In regard to the car industry it looks a though “everybody cheats”...

"Science draws the wave, poetry fills it with water" Teixeira de Pascoaes

 

HQPlayer Desktop / Mac mini → Intona 7054 → RME ADI-2 DAC FS (DSD256)

Link to comment

Mankind has an eternal problem, with the conflict between "making it", that is, acquiring lots of money, so that you have full access to any toys you desire - and "doing the right thing" ... gaming is an easy shortcut to making the former happen - so it is always highly tempting to push that button ...

 

The cartoons of a saint, and a devil, having a bit of a barney over the top of a person's head are spot on - we are, indeed, flawed, 😉.

Link to comment
7 hours ago, idiot_savant said:

I have to say I’m surprised by the “everybody cheats” shrugging of shoulders - I think there’s a distinction between being “optimistic” with specs and deliberately cheating at a well respected measurement that could be going on.
There’s no such thing as a free lunch - for example, those VWs that had the engine management systems “fixed” were reportedly much slower afterwards - the kind of gaming I’m talking about deliberately trades better measurement performance for poorer sound quality, that is important surely?

 

I’m fairly confident you could up with a scheme to see if this was happening, but if no one cares?

 

your friendly neighbourhood idiot 

Well I dont think it is a case of "everyone cheats".  I think you may have misunderstood the points I have made.  Its about looking in detail at how the specs are presented and understanding them.

 

For example you can specify an amp power output at 1% THD.  You can specify it at 0.1% and it will be lower.  Neither is wrong, 1% isnt cheating.

 

This isnt the same as what VW did.  They set out to deliberately deceive. The car detected the test and changed its operating parameters to obtain better results.

 

As a consumer, having to be technically literate to understand all of this is not a good solution.  Perhaps there should be standards for comparison to make life easier, but the one example we do have of this (the US FTC amplifier power output tests) is a blunt instrument and misleading in itself.

Link to comment
5 hours ago, March Audio said:

Well I dont think it is a case of "everyone cheats".  I think you may have misunderstood the points I have made.  Its about looking in detail at how the specs are presented and understanding them.

 

For example you can specify an amp power output at 1% THD.  You can specify it at 0.1% and it will be lower.  Neither is wrong, 1% isnt cheating.

 

This isnt the same as what VW did.  They set out to deliberately deceive. The car detected the test and changed its operating parameters to obtain better results.

 

As a consumer, having to be technically literate to understand all of this is not a good solution.  Perhaps there should be standards for comparison to make life easier, but the one example we do have of this (the US FTC amplifier power output tests) is a blunt instrument and misleading in itself.

 

I think that this topic is about electronic equipment manufacturers doing a VW.

 

Ayre has had a "measure" and a "listen" filter digital for more than a decade (e.g. DX-5, QB-9), but they are upfront about it.

The problem would arise if an equipment had been coaxed into "sensing" it is being measured and change into a e.g. filter that will improve its performance in one or three parameters.

 

Like cars, DACs and amplifiers are slowly turning into computers; it's not really rocket science, just a bit of extra code.

"Science draws the wave, poetry fills it with water" Teixeira de Pascoaes

 

HQPlayer Desktop / Mac mini → Intona 7054 → RME ADI-2 DAC FS (DSD256)

Link to comment
1 hour ago, semente said:

 

I think that this topic is about electronic equipment manufacturers doing a VW.

 

Ayre has had a "measure" and a "listen" filter digital for more than a decade (e.g. DX-5, QB-9), but they are upfront about it.

The problem would arise if an equipment had been coaxed into "sensing" it is being measured and change into a e.g. filter that will improve its performance in one or three parameters.

 

Like cars, DACs and amplifiers are slowly turning into computers; it's not really rocket science, just a bit of extra code.

It's not

 

The idea of reducing technical performance to enhance subjective quality is stooooopid to say the least.

 

There is no parallel to what VW did.

 

 

Link to comment
11 minutes ago, March Audio said:

It's not

 

The idea of reducing technical performance to enhance subjective quality is stooooopid to say the least.

 

There is no parallel to what VW did.

 

 

 

Can you rephrase that? It's not clear.

 

I can say straight away that "reducing technical performance to enhance subjective quality is not stooooopid".

For one you get a house sound - good marketing strategy.

Secondly, people have different tastes in "presentation", it's nice and fair if they can find a piece of equipment to their liking.

Also, many audiophiles enjoy the box-swapping quest for their preferred "presentation" and having access to a large assortment of gear from which to choose is important to them.

 

This idea that all equipment should be "transparent" can only come from lab-rat engineers with no connection with the real world.

 

(no Toole quotes please, I read the "Bible")

"Science draws the wave, poetry fills it with water" Teixeira de Pascoaes

 

HQPlayer Desktop / Mac mini → Intona 7054 → RME ADI-2 DAC FS (DSD256)

Link to comment

Yes the idea that you get better subjective quality by reducing technical quality is stooooooooopid. 😛

 

This is the objective section.  Personal preference doesn't come into it.

 

Not to mention the fact that if this is all down to personal preference then there is no such thing as good.  Whatever anyone likes, even AM radio, is good.

Link to comment
2 minutes ago, March Audio said:

Yes the idea that you get better subjective quality by reducing technical quality is stooooooooopid. 😛

 

This is the objective action.  Personal preference doesn't come into it.

 

We're all entitled to our opinions. And preferences.

 

Even though I prefer "transparent" gear, I don't support a "transparent" gear dictatorship.

"Science draws the wave, poetry fills it with water" Teixeira de Pascoaes

 

HQPlayer Desktop / Mac mini → Intona 7054 → RME ADI-2 DAC FS (DSD256)

Link to comment
1 hour ago, semente said:

 

We're all entitled to our opinions. And preferences.

 

Even though I prefer "transparent" gear, I don't support a "transparent" gear dictatorship.

Of course.  However this is the objective section so you need to justify why any one particular person's preference is better, or in this case why you think reducing technical quality improves subjective quality.

 

....and I dont support the "anything goes/anything is good as long as someone somewhere likes it" dictatorship. 

 

As a simplistic argument, do you think an amp with 10% thd sounds better than one with 0.1% THD?

Link to comment

Hi,

 

trying to get back on topic…

apologies if I’ve not been clear, so here’s a little bit of history for context:

Back in the day, SNR was king - so you had an AP, and a DAC. So to measure SNR, measure the biggest signal, then turn off the signal and measure what’s left. 
now, some DAC manufacturers “gamed” this measurement by detecting digital silence and effectively performed an analog mute when they detected it - now, the SNR ratio can be improved just by having more gain for a bigger number, but the product itself is not any better, indeed it is possibly worse with more gain. 
So, realising this, AP and the AES ratified a new test, called the “Dynamic Range Test”. In this test, you feed in a signal at -60dBFS and measure how far the noise floor is below this, add the 60dB back in and you get a nice number telling you the Dynamic Range. 
So, if you look on the internet for “Sabre DRE data sheet” you can find the data sheet for the 9219. If you search this for DRE, there is a feature that can be enabled that deliberately monitors the *input* signal and modifies the *analog output gain* by up to 30dB - so for our test bench, it can detect it’s being tested, and reduce the analog output of the DAC whilst compensating internally, potentially gaining 30dB in measurement terms, yet subjectively a DAC mucking about with an amplifier will be objectionable. 


Now I don’t know if anybody does this in a product, but from the feedback so far, somebody probably does… and the fact it’s a real feature in the chip…
 

does this help explain the problem?

 

Your friendly neighbourhood idiot 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...