Misleading Measurements

[pkane2001]

3 minutes ago, The Computer Audiophile said:

Measurements are misleading just like statistics. They are created for those who understand them and presented to those who have no clue. Higher is better, lower is better, etc... We can't expect Joe Sixpack to understand them, so we must take some responsibility when presenting them. 

 

Presenting information that can't possible be heard by humans can only serve to mislead consumers. it can be a fun "mine is bigger than yours" fest for those seeking to engineer stuff that doesn't matter, but when presented to consumers who don't understand, don't care to understand, or just can't understand, it's up to those presenting the information to be responsible.

 

You're not arguing against measurements, you're arguing against interpretations.

 

Measurements are factual data points that can be tested, repeated, and confirmed by others. It is not the goal of a measurement to educate the public or to provide recommendations: that's the interpretation part. There's a huge value to published measurements, because they quantify the behavior of a piece of equipment.

 

The interpretation, on the other hand, can be completely personal (I do this all the time for my own use), or generalized to a group, as most published studies try to do. One can go back to the same measurements and re-interpret their implications later, once some new study is published, or when one's hearing changes, or when one becomes more educated in the art of interpreting measurements, or when applying it to a different group. None of this negates the original measurements.

[The Computer Audiophile]

4 minutes ago, pkane2001 said:

 

You're not arguing against measurements, you're arguing against interpretations.

 

Measurements are factual data points that can be tested, repeated, and confirmed by others. It is not the goal of a measurement to educate the public or to provide recommendations: that's the interpretation part. There's a huge value to published measurements, because they quantify the behavior of a piece of equipment.

 

The interpretation, on the other hand, can be completely personal (I do this all the time for my own use), or generalized to a group, as most published studies try to do. One can go back to the same measurements and re-interpret their implications later, once some new study is published, or when one's hearing changes, or when one becomes more educated in the art of interpreting measurements, or when applying it to a different group. None of this negates the original measurements.

Not really. 

 

Publishing measurements that are meaningless to the consumer, with absolutely zero interpretation, can only be a disservice.  Take for example, all the specs produced via measurements and published for components. There is no interpretation done, just specs published. These specs are really measurements. Posting this stuff can't help a consumer. 

 

It's really just a bunch of guys having fun with themselves, looking at the lowest measurements etc... Joe Sixpack can't be helped by this, even with zero interpretation. 

 

[Summit]

Measurements and statistics are not misleading per se, therefore better don't throw out the baby with the bathwater.

 

Measurements and statistics can of course be presented in an attempt to mislead, but that’s another thing, don’t you think?

 

I get that you think that measurements that can't be heard by humans only serve to mislead consumers. Though many audiophiles believes that Signal integrity, Jitter, Phase noise, Leakage current, Rise and fall time, Signal reflections, Electrical conductivity, Resistance and many other characteristics is of great importance and like to know how they measures.

[pkane2001]

6 minutes ago, The Computer Audiophile said:

Not really. 

 

Publishing measurements that are meaningless to the consumer, with absolutely zero interpretation, can only be a disservice.  Take for example, all the specs produced via measurements and published for components. There is no interpretation done, just specs published. These specs are really measurements. Posting this stuff can't help a consumer. 

 

It's really just a bunch of guys having fun with themselves, looking at the lowest measurements etc... Joe Sixpack can't be helped by this, even with zero interpretation. 

 

 

Let me see if I understand your point: facts are irrelevant and shouldn't be shared because a consumer is too dumb to understand them? 

[The Computer Audiophile]

6 minutes ago, Summit said:

Measurements and statistics are not misleading per se, therefore better don't throw out the baby with the bathwater.

 

Measurements and statistics can of course be presented in an attempt to mislead, but that’s another thing, don’t you think?

 

I get that you think that measurements that can't be heard by humans only serve to mislead consumers. Though many audiophiles believes that Signal integrity, Jitter, Phase noise, Leakage current, Rise and fall time, Signal reflections, Electrical conductivity, Resistance and many other characteristics is of great importance and like to know how they measures.

 

I'm talking about measurements touted by the objective crowd that they also profess can't possibly be heard by human ears. 

 

 

 

 

1 minute ago, pkane2001 said:

 

Let me see if I understand your point: facts are irrelevant and shouldn't be shared because a consumer is too dumb to understand them? 

 

Go round about as many times as you wish Paul. What if you needed a stent put in an artery and the stent manufacturers showed you all kinds of competing measurements about the materials they use, but in reality none of it mattered. Would you be best served by this information or should I call you too dumb to understand?

 

I'm suggesting a pass / fail approach is much better. If you can't hear it, you shouldn't care and it can only serve to confuse consumers who have zero interest in getting an EE degree just to purchase a DAC. 

 

[kumakuma]

14 minutes ago, The Computer Audiophile said:

 

I'm suggesting a pass / fail approach is much better. If you can't hear it, you shouldn't care and it can only serve to confuse consumers who have zero interest in getting an EE degree just to purchase a DAC. 

 

 

So you'd be okay with the "objective crowd" not showing you the measurements and just telling you that a device passed or failed?

[The Computer Audiophile]

Just now, kumakuma said:

 

So you'd be okay with the "objective crowd" not showing you the measurements and just telling you that a device passed or failed?

 

Showing me text is no different from showing me a graph. 

[kumakuma]

Just now, The Computer Audiophile said:

 

Showing me text is no different from showing me a graph. 

 

Not sure what that means.

[The Computer Audiophile]

Just now, kumakuma said:

 

Not sure what that means.

It means that I'm OK with whatever the objective crowd presents. It's more about the people conducting the measurements. If they'll fudge text that says pass or fail, they'll fudge measurements. I trust they won't do either, so I'm OK with a pass / fail. 

[kumakuma]

3 minutes ago, The Computer Audiophile said:

It means that I'm OK with whatever the objective crowd presents. It's more about the people conducting the measurements. If they'll fudge text that says pass or fail, they'll fudge measurements. I trust they won't do either, so I'm OK with a pass / fail. 

 

I'm not. I'd rather not rely on someone else's subjective interpretation of the data.

[pkane2001]

20 minutes ago, The Computer Audiophile said:

 

I'm talking about measurements touted by the objective crowd that they also profess can't possibly be heard by human ears. 

 

Go round about as many times as you wish Paul. What if you needed a stent put in an artery and the stent manufacturers showed you all kinds of competing measurements about the materials they use, but in reality none of it mattered. Would you be best served by this information or should I call you too dumb to understand?

 

I'm suggesting a pass / fail approach is much better. If you can't hear it, you shouldn't care and it can only serve to confuse consumers who have zero interest in getting an EE degree just to purchase a DAC. 

 

Perhaps we can agree that there are some consumers that want to know more than just a pass/fail mark? You know... an educated consumer is our best customer? 😃

 

If I ever need a stent, I'll spend my time to educate myself on what matters and interpret "facts" and measurements published by a manufacturer rather than just trust someone else to tell me. I've spent too much time around doctors and hospitals to think they are superhuman. They make as many mistakes as any of us, if not more. 

[The Computer Audiophile]

7 minutes ago, pkane2001 said:

You know... an educated consumer is our best customer?

 

I hate to constantly disagree, but there is no best consumer. People can't be experts in every ares for which they purchase products. 

 

 

7 minutes ago, pkane2001 said:

If I ever need a stent, I'll spend my time to educate myself on what matters and interpret "facts" and measurements published by a manufacturer rather than just trust someone else to tell me.

 

Chances are good if you need one, you won't have several months to educate yourself. Fortunately, you realize that not everyone is in your position to have the time to educate themselves on every matter in the world. 

 

I just don't understand the willful ignorance by objective leaning people. They refuse to admit that consumers are mislead equally by measurements as they are subjective prose. And, the objective crowd has zero interest in dealing with this fact. They seem to be ideologues who see things as they want rather than as they are.  

[The Computer Audiophile]

Another perplexing thing to me is the reason why objective leaning people want to see measurements that are below the threshold of human hearing. If one is truly objective, then the measurements below human hearing don't matter. 

[kumakuma]

45 minutes ago, The Computer Audiophile said:

I just don't understand the willful ignorance by objective leaning people. They refuse to admit that consumers are mislead equally by measurements as they are subjective prose. And, the objective crowd has zero interest in dealing with this fact. They seem to be ideologues who see things as they want rather than as they are.  

 

How do you know that the ignorance is "willful"? Perhaps they just need to be educated about the follies of their ways.

 

Why don't you head over to ASR to start this process?

 

I'm sure you would be welcomed just as warmly as you welcome the hard core objectivists who visit here from time to time to point out the logical fallacies that exist at the core of the subjectivist lifestyle.

[The Computer Audiophile]

8 minutes ago, kumakuma said:

 

How do you know that the ignorance is "willful"? Perhaps they just need to be educated about the follies of their ways.

 

Why don't you head over to ASR to start this process?

 

I'm sure you would be welcomed just as warmly as you welcome the hard core objectivists who visit here from time to time to point out the logical fallacies that exist at the core of the subjectivist lifestyle.

I get it, you hate this site since I made the change to place objective threads in a sub-forum. You are free to leave at any time. Or, you can actually help answer questions to which you seemingly know the answers. 

 

Why do you want to see measurements that have no effect on a product's performance?

[pkane2001]

1 hour ago, The Computer Audiophile said:

Chances are good if you need one, you won't have several months to educate yourself. Fortunately, you realize that not everyone is in your position to have the time to educate themselves on every matter in the world. 

 

If you want to know the answer, and it's important to you, then you'll educate yourself. Ignorance is not my preferred state, and I would hope that's true for most people. 

 

1 hour ago, The Computer Audiophile said:

I just don't understand the willful ignorance by objective leaning people. They refuse to admit that consumers are mislead equally by measurements as they are subjective prose. And, the objective crowd has zero interest in dealing with this fact. They seem to be ideologues who see things as they want rather than as they are. 

 

Again, major generalization. Painting all "objective leaning people" with the same brush. Please stop arguing with a straw man, Chris. Please recognize that you are talking to individuals, with their own experience, opinions, and ability to think and speak independently, and not with some incoherent, malicious group hell-bent on causing harm to innocent consumers.

[The Computer Audiophile]

If measurements for items that fall below human hearing are important, wouldn't it be just as important to note the color of the component being measured?

[The Computer Audiophile]

1 minute ago, pkane2001 said:

 

If you want to know the answer, and it's important to you, then you'll educate yourself. Ignorance is not my preferred state, and I would hope that's true for most people. 

 

 

Again, major generalization. Painting all "objective leaning people" with the same brush. Please stop arguing with a straw man, Chris. Please recognize that you are talking to individuals, with their own experience, opinions, and ability to think and speak independently and not with some incoherent, malicious group hell-bent on causing harm to innocent consumers.

 

I believe most people like to educate themselves, but it just isn't possible in many circumstances. 

 

I realize objective leaning people aren't hell bent on causing harm, but I believe many have lost the plot and are actually misleading people. If measurements that don't matter to human ears are being published, why not publish what the person measuring the equipment was wearing during the measurements. 

 

Perhaps I can make this easier to understand for myself by asking an easy question to which there are no wrong answers. 

 

Why do you want to see measurements that make no difference to any human ears?

[pkane2001]

10 minutes ago, The Computer Audiophile said:

If measurements for items that fall below human hearing are important, wouldn't it be just as important to note the color of the component being measured?

 

It's important to me to know that my car can reach 180mph even though it has a 150mph limiter, and 99% of the time I keep it under 100mph. So sue me. It's irrational.

 

I'm an engineer and like well-designed and built machinery with tolerances to handle extreme conditions. Same with audio. I like the idea of a perfectly transparent system. One that I don't need to worry about adding or subtracting anything to the sound, regardless of whether I can hear it or not. But, given a specific budget, I'll pick the best performing device (read: best measuring) among all those that fall below my threshold of audibility. 

 

[The Computer Audiophile]

2 minutes ago, pkane2001 said:

 

It's important to me to know that my car can reach 180mph even though it has a 150mph limiter, and 99% of the time I keep it under 100mph. So sue me. It's irrational.

 

I'm an engineer and like well-designed and built machinery with tolerances to handle extreme conditions. Same with audio. I like the idea of a perfectly transparent system. One that I don't need to worry about adding or subtracting anything to the sound, regardless of whether I can hear it or not. But, given a specific budget, I'll pick the best performing device (read: best measuring) among all those that fall below my threshold of audibility. 

 

 

All good information. Thanks.

 

It seems all subjective, why you like the objective that doesn't matter to human hearing? 

 

Do you think you're contributing to an arms race of levels that don't matter, by saying, "given a specific budget, I'll pick the best performing device (read: best measuring) among all those that fall below my threshold of audibility?" 

 

Do you also believe a product is better if it measures lower than another, all else being equal?

 

[John Dyson]

1 hour ago, The Computer Audiophile said:

It means that I'm OK with whatever the objective crowd presents. It's more about the people conducting the measurements. If they'll fudge text that says pass or fail, they'll fudge measurements. I trust they won't do either, so I'm OK with a pass / fail. 

I think that these matters about specsmanship would be a little more honest if there were open explanations when/if some initially extreme value for a measurement might be helpful.

Just saying:  my circuit has 0.0001% distortion and is so much better than 'Sams' circuit with 0.0002% distortion is a rather useless argument or comparison.  Those are blind comparisons with no context.  I have this wonderful little paper that some person wrote on op-amps, creating his own specs with LOTS more details than normal manufacturers specs.   That 'wonderful' paper helps to show the behavior with much more circuit context involved.  It provides much more helpful behavior information for more realistic circuit configurations (of course, not perfect.)   There are so many choices of op-amps, that good, understandable objective measurements are so helpful to get started.

 

I can agree that a blind spec without explanation of context is just a little better than irrelevent.  This reminds me of the old 'lines of resolution' spec for SVHS and VHS decks.  The number was meaningless, but we consumers always know that 'bigger is better', right?   Heh -- they way that those 'lines of resolution' were measured was almost meaningless WRT actual quality of video reproduction.  They were especially meaningless when comparing consumer vs pro video equipment.  (Nowadays, such matters are anachronistic -- we are so spoilt with almost flat & more linear video response in comparison.)

 

This would be similar to the 0.001% vs 0.00001% distortion...   For example (another one of my diversions): how many such 'wonderful' preamps characteristics are measured with a source that truly emulates (for example) a MM cartridge?   Such transducers are well known to have very wide ranging characteristics and high impedance in certain frequency bands.   Such a high transducer impedance can certain cause negative effects WRT modulation of input impedance vs. signal waveform/frequency/level.   The noise matter bothers me less, because it is almost impossible for a competent designer to create much more preamp noise than an MM cartridge at frequencies where the ear is sensitive.   However, I seldom see a real distortion measurement where the source is at least a model of a cartridge (or, perhaps a low impedance signal feeding through a cartridge of choice, so that an actual distortion measurement can be done.)

 

Big, fat low noise jFETS or medium geometry BJTs can have noticeably changing input capacitance in a normal amplifier circuit.  This changing capacitance acts superficially similar to a changing resistance, and in certain cases can produce significant distortion.

 

It WOULD be nice if the measurements actually measured in-circuit/in-situ behavior, and comparisons were made with the priorities openly explained.   At least, when someone says that (for example) the 0.25dB down at 20kHz and 1dB down at 30kHz has the effect of significantly changing the sounds of the cymbals crashing -- the judgement can then be prioritized by the person reading the spec.   At least, I know how I would prioritize that interpretation of that raw frequency response spec, esp at 30kHz.  (BTW, the change in cymbals crashing might be caused more by dynamic input impedance effects, say, on MM preamps.)

 

The general categories of objective & subjective can matter -- the problem with relevence has to do with the priority/usefulness of the actual spec, and the measurement/usability situation for that spec.

 

John

 

[pkane2001]

9 minutes ago, The Computer Audiophile said:

All good information. Thanks.

 

It seems all subjective, why you like the objective that doesn't matter to human hearing? 

 

We've been around this bush before. My answer is still the same as it was about a week ago in this thread: any devices with distortion measuring below the audibility threshold should be picked by their subjective attributes: color, price, brand, esthetics, weight, engineering, marketing, etc.

 

12 minutes ago, The Computer Audiophile said:

Do you think you're contributing to an arms race of levels that don't matter, by saying, "given a specific budget, I'll pick the best performing device (read: best measuring) among all those that fall below my threshold of audibility?" 

 

What arms race and how am I doing anything to contribute to it by stating something as my own, personal opinion/preference? I guess if I had that much sway on the audiophile community, I would consider it to be a possible undue influence, but no, I don't see it.

 

But I would ask you to turn the mirror around. You do have a much greater influence here than anyone else. And yet, you consistently engage in attacks and take actions that amount to censorship against a fairly sizable part of the audiophile community. I would consider that to be much more of engaging in an "arms race". 

 

19 minutes ago, The Computer Audiophile said:

Do you also believe a product is better if it measures lower than another, all else being equal?

 

Yes. A better measuring product is a better engineered, better designed, better thought-out, better assembled one. To me, that's attractive. But like I said, it's my own preference and I'm not forcing it on anyone else.

 

[The Computer Audiophile]

3 minutes ago, pkane2001 said:

And yet, you consistently engage in attacks and take actions that amount to censorship against a fairly sizable part of the audiophile community. I would consider that to be much more of engaging in an "arms race". 

Pics or it didn't happen. 

 

 

3 minutes ago, pkane2001 said:

A better measuring product is a better engineered, better designed, better thought-out, better assembled one. To me, that's attractive.

 

This is interesting. I don't believe it's true that a better measuring product is better engineered, designed, better thought-out, or assembled. Once the level of human hearing is hit, it seems anything further is just men challenging themselves or making products for those who like specs that don't matter. It's like saying one product has a better color red than another. Neither matter to the ears, but matter to the heart. Pursuits of the lowest measurements or the best color red are all fine with me, I just don't like one touted as more important than the other when it comes to sound quality. 

 

Better is of course subjective though and we all have our own better. 

[pkane2001]

14 minutes ago, John Dyson said:

It WOULD be nice if the measurements actually measured in-circuit/in-situ behavior, and comparisons were made with the priorities openly explained.   At least, when someone says that (for example) the 0.25dB down at 20kHz and 1dB down at 30kHz has the effect of significantly changing the sounds of the cymbals crashing -- the judgement can then be prioritized by the person reading the spec.   At least, I know how I would prioritize that interpretation of that raw frequency response spec, esp at 30kHz.  (BTW, the change in cymbals crashing might be caused more by dynamic input impedance effects, say, on MM preamps.)

 

That's why there are more complete measurements being published. Nobody (except for some manufacturers) publish just the THD % -- and in those cases, I fully agree with you -- that's mostly meaningless, even when stated as 1kHz @ 0dBFS.

 

Here's an example of the types of evaluation I find useful, having performed these myself. I look for these when published by others, as they do provide a lot more detail about a device than just a single number.

 

Distortion vs. level with levels of individual harmonics, noise, etc:

 

Distortion vs Frequency:

 

And yes, Chris, these are mostly below threshold of audibility, although the noise floor is higher than I'd like to see.

 

[pkane2001]

35 minutes ago, The Computer Audiophile said:

Pics or it didn't happen. 

I think I can find some, if you'd really like to go down this path.

 

35 minutes ago, The Computer Audiophile said:

This is interesting. I don't believe it's true that a better measuring product is better engineered, designed, better thought-out, or assembled. Once the level of human hearing is hit, it seems anything further is just men challenging themselves or making products for those who like specs that don't matter. It's like saying one product has a better color red than another. Neither matter to the ears, but matter to the heart. Pursuits of the lowest measurements or the best color red are all fine with me, I just don't like one touted as more important than the other when it comes to sound quality. 

 

Better is of course subjective though and we all have our own better. 

 

It is harder to design a product with less distortion, better linearity, lower noise. All of these are part of a measure of transparency of the device. It's possible (though not very likely) that a poorly engineered device can achieve good transparency by accident, but mostly, it's done by design and hard work.