Is Someone Playing Fast & Loose With Measurements?

[magnum innominandum]

Hi,

 

Comparing measurements made by different people (and likely in different test conditions) strikes me as rather useless.

 

Interesting take.

 

So, you go to Les 5 Saisons at Westmount, you get 500 Gram Kobe Beef.

 

When you get home, you feel something is not quite right, you weight and you find they gave your 300 Gram.

 

You go back and raise the issue, only to be told:

 

"Comparing measurements made by different people (and likely in different test conditions) is useless."

 

The point of measurements is that they SHOULD be comparable and repeatable. Otherwise they would be rather useless.

[Hiro]

Hi,

 

 

 

Interesting take.

 

So, you go to Les 5 Saisons at Westmount, you get 500 Gram Kobe Beef.

 

When you get home, you feel something is not quite right, you weight and you find they gave your 300 Gram.

 

You go back and raise the issue, only to be told:

 

"Comparing measurements made by different people (and likely in different test conditions) is useless."

 

The point of measurements is that they SHOULD be comparable and repeatable. Otherwise they would be rather useless.

 

How did the test conditions differ when making the two measurements?

 

I don't think you have understood my point...

[magnum innominandum]

Hiro,

 

I don't think you have understood my point...

 

I understood your point perfectly.

 

How did the test conditions differ when making the two measurements?

 

Two different scales were used, from different manufacturers.

 

The Deli is on the ground-floor, my flat is high up, so gravity is lower and the meat weights less.

 

Weather changed so air pressure was lower reducing the reading.

 

One can always come up with excuses.

 

Dog ate my homework.

 

There are recommendations regarding how to perform measurements in Audio. One may ignore them of course. But that makes one's test less than useful.

 

We have JA at Stereophile using AP2.

 

We have the European reviewer using AP2.

 

It appears iFi also use AP2.

 

We have Gordon Rankin with a Prism D-Scope.

 

We have LH Labs with an APx525.

 

Multiple tests independent from each other tend to show very similar results DESPITE different conditions and even different test gear, with normal differences attributable to those causes.

 

Only one of the whole set shows 1:10 to > 1:100 differences to the rest.

 

Seems most of the measurements do their job, are repeatable and comparable as should be expected (just as the weight of a piece of meat).

 

Does one need to ask why only one set greatly disagrees?

 

It is good that the the set of data that disagrees is from a independent, uninterested third party with nothing to sell...

 

DM's suggestion that LH-Labs was unlucky enough to get TWO broken units out of three purchased is risible, though he clearly wants to defend LH-Labs (Did I read he is waiting for a Geek out?).

 

Your suggestion that if most sets of test data agree with each other and one is out, the one that is out is correct as well even so? Let me call it quaint.

 

Maybe Mr. Ho's mistake was to test 4 units? Very very bad that, to have anything 4. Maybe it was that 4-unit parallel test 4 that caused those measurements to go wrong? One may fear for Mr. Ho's health with 4 involved. No, that too is risible.

 

I think Mr. Ho owes a better explanation than has been given so far.

 

Seeing Mr. Fish's comments I doubt any will be forthcoming.

 

I had enough of this, so I will not waste more time. I'm not getting paid for shooting the breeze here.

 

The facts are all in the open. Let anyone make up their own minds.

[Hiro]

Hiro,

 

I understood your point perfectly.

 

Two different scales were used, from different manufacturers.

 

These are not different test conditions.

 

Larry Ho provided the following additional information for his test measurements:

 

"Here is my parameters setting, I'm open for any suggestion or discuss from you or JA. ;-)

 

* Output load is not 100k Ohm or 200K, which is usually not the real headphone load situation. I set 300Ohm in Audio Precision which is more real. By this alone, THD+N will be different.

 

 

* I set the bandwidth of measurement to 90K Hz. Not traditional 20K Hz.... Why? Because since everybody is talking about the important of HD music. Why limit the measurement bandwidth to 20k here. It's proven wrong to assume people could hear only below 20KHz.

Again, this single setting will change the SNR and THD results.

 

 

* I use 'un-weighted' numbers. A lot of people they use 'A-weighted'... why? Because A-weighted numbers looks better..."

 

My point is that unless other measurements are done under the exactly same set of parameters we can't really compare them.

[Miska]

* Output load is not 100k Ohm or 200K, which is usually not the real headphone load situation. I set 300Ohm in Audio Precision which is more real. By this alone, THD+N will be different.

 

This makes sense for a device with headphone output, it is quite close to typical hifi headphone impedance.

 

The numbers I quoted were with 33 and 330 ohm load, but the difference is that Dragonfly was version 1.2 that supposedly had the clipping issue fixed and other improvements too.

[lithium1085]

Ultimately a true scientific test is when all the products are compared using the same conditions together. Larry's tests probably need to be validated by a third party because of the obvious conflict of interest. I think JA @ stereophile is the right person to do this and I am sure that when stereophile reviews the Geek the truth will be clear

[USAudio]

Whatever happened with this? I see here that LH ordered a Dragonfly 1.2 to test:

Topic: RESPONSE NEEDED: Audioquest questions measurements (1/3) - Geek Temple - LH Labs

 

Did LH ever measure the Dragonfly with the 1.2 version? If so, can someone point me to the results?

 

Looks like the results here are still for the Dragonfly 1.0:

Topic: Geek Out vs. the Others, 3rd Installment (1/2) - Geek Temple - LH Labs

 

Thanks!