Why you can't trust measurements

[GoldenOne]

I figured this may be something worth starting a dedicated thread for.
There are all sorts of reasons as to why measurements may be unreliable, misleading, confusing, or just inconsistent. And I've made a video discussing some of these topics.

 

 

[Speedskater]

Don't trust audiophiles that don't trust measurements.

[botrytis]

One needs both to make decisions. One cannot really make well-informed ones without it.

 

I don't care if people look at measurements or not, one just needs to understand it is easy to fool the human mind.

 

Thanks @GoldenOne

[Speedskater]

5 hours ago, Speedskater said:

Don't trust audiophiles that don't trust measurements.

I don't watch U-tube videos.

So I was referring to tests done be competent independent engineers/technicians.

It seems that the video may be about making deceiving measurements.

I guess I'll have to watch this one.

 

[ASRMichael]

I was looking over some previous emails earlier from a respected DSP engineer. On one email he stated he trusted more his ear than he did maths, but when maths aligns together with results then it’s icing in the cake. This got me thinking, if this engineer is saying this….what does it mean for measurement software/hardware? As it’s all based on maths. 
 

For me measurements are a guide…Ears are what let’s you decide if you like it or not. That’s if you have a open mind that is, and not just going on measurements/maths.

[fas42]

Nicely done. People who like what they deal with neatly tied up in a well presented package will have a strong liking for measurements; even though the latter only tell part of the story, or are 'fudged' to suit the situation. My own experiences with my first high performance amplifier, 30 years ago, a Perreaux, pretty well sum up the realities in the audio world: I chose this amplifier by ear; it easily beat the big Krell that was also in the demo room at the time; and in further listening to other ambitious rigs using high end amplifiers I had my opinion that I had chosen a good'un confirmed. What was also impressive is that the user manual had an excellent rundown of measurement data, which far exceeded what one gets in any current day amplifier guide. And the suite of numbers in the manual "said" that I shouldn't hear any distortion ...

 

Yet the amplifier had a problem with treble distortion. Quite easy to hear - when the amplifier was driving above a certain level, quite moderate in comparison to how loud it could go, the treble content started to show an issue. Over a period of time I determined that the unsophisticated power supply it used was the problem - I made major modifications to this area; and the treble distortion was eliminated.

 

The point is that the fulsome measurements, on paper, gave zero hint that the amplifier had a weakness - only longer term living with the part revealed the lacking. So, if you rely on measurements to "tell the full story", well then, I've got a nice big bridge to sell you ... .

[Allan F]

8 hours ago, Speedskater said:

Don't trust audiophiles that don't trust measurements.

 

Don't trust audiophiles who believe that measurements will tell them what audio gear sounds like. Measurements have their place, but they are no substitute for listening.

[GregWormald]

3 hours ago, fas42 said:

<snip>So, if you rely on measurements to "tell the full story", well then, I've got a nice big bridge to sell you ... .

Hey Frank, the Harbour bridge isn't for sale, it's mine! Quit trying to con people.

[fas42]

Sorry 'bout that ...

[botrytis]

25 minutes ago, GregWormald said:

Hey Frank, the Harbour bridge isn't for sale, it's mine! Quit trying to con people.

 

We can sell him some beach front property in Arizona though 🤣

[GregWormald]

17 minutes ago, botrytis said:

 

We can sell him some beach front property in Arizona though 🤣

See, measurements. It depends on whether "beach front property" is about the sand, or the water.

[botrytis]

19 minutes ago, GregWormald said:

See, measurements. It depends on whether "beach front property" is about the sand, or the water.

 

Sand of course 😁

 

[ASRMichael]

 

[botrytis]

Well, unfortunately, The gent has really turned into a snake oil salesman.

 

PS Audio makes some decent products, others not so decent,

[pkane2001]

On 4/20/2022 at 11:07 AM, GoldenOne said:

I figured this may be something worth starting a dedicated thread for.
There are all sorts of reasons as to why measurements may be unreliable, misleading, confusing, or just inconsistent. And I've made a video discussing some of these topics.

 

 

 

So perhaps a better title would be "Why you can't trust audio measurements that are not well documented". Everything in the video speaks to the differences in setup and configuration of the test. As long as these are properly documented, there's no reason to not trust the measurements as they can then be reproduced by others. The only reason to question well-documented measurements is if someone else gets a significantly different result with the same configuration.

[botrytis]

4 minutes ago, pkane2001 said:

 

So perhaps a better title would be "Why you can't trust audio measurements that are not well documented". Everything in the video speaks to the differences in setup and configuration of the test. As long as these are properly documented, there's no reason to not trust the measurements as they can then be reproduced by others. The only reason to question well-documented measurements is if someone else gets a significantly different result with the same configuration.

 

Or if someone has such an expensive testing rig that no one else can afford....

[yamamoto2002]

Measurement is useful to find and fix design defect of the product.  Sometimes the result tells is not related to perceived sound quality. Sometimes it tells sound degradation overlooked and it can be heard by ears on specific corner-case condition or just by changing framing of the cognition.

 

When I show the measurement result, I tried to create champion data, to minimize mains and computer interference, prevent the click noise of fluorescent lamp or microwave oven switching during measurement etc. In Japan, mains earthing system is TT, wall outlet is 2 prong and isolation transformers for domestic market works well to prevent ground loops 🙂 Recent addition is metal wire mesh bag for measurement laptop, to reduce unwanted Wi-Fi interference or outgoing noise radiation.

 

 

[pkane2001]

10 minutes ago, botrytis said:

 

Or if someone has such an expensive testing rig that no one else can afford....

 

$30k or so isn't out of the realm of the possible. Many audiophile components are way more expensive than that. If you really don't want to spend all that cash on an APx555, you could rent one for under $1k a day :)

 

 

 

[botrytis]

Well, Kippel rigs are 100K for the basic one and go up from there.

 

I don't have 30K IN ALL THE AUDIO EQUIPMENT I OWN. I could buy my next car for 30K - just me (mine is a 2007 and my wife's is a 2001). 

[pkane2001]

3 minutes ago, botrytis said:

Well, Kippel rigs are 100K for the basic one and go up from there.

 

I don't have 30K IN ALL THE AUDIO EQUIPMENT I OWN. I could buy my next car for 30K - just me (mine is a 2007 and my wife's is a 2001). 

 

Forget that new car, or a new house, just get the Klippel analyzer ;)

 

[yamamoto2002]

In Japan, a Ferrari dealer (Cornes and co. ltd) sells Audio Precision products🙂

[pkane2001]

1 minute ago, yamamoto2002 said:

In Japan, a Ferrari dealer sells Audio Precision products🙂

 

That's awesome! I'd recommend a package deal for both :)

 

[botrytis]

7 minutes ago, pkane2001 said:

 

Forget that new car, or a new house, just get the Klippel analyzer ;)

 

 

Yeah, I want to get kicked out of my house. My wife would say something, and it won't be pretty....

 

[pkane2001]

12 minutes ago, botrytis said:

 

Yeah, I want to get kicked out of my house. My wife would say something, and it won't be pretty....

 

I hear you! But you have to have priorities in life. A house + wife vs Klipple… it’s a tough decision!

[botrytis]

29 minutes ago, pkane2001 said:

I hear you! But you have to have priorities in life. A house + wife vs Klipple… it’s a tough decision!

 

It is a no brainer to me. I have been married 35 years to the same wonderful, considerate soul. She puts up with my foibles, so I am very lucky, to say the least. AND SHE LIKES AUDIO!