Misleading Measurements

[Jud]

15 hours ago, jabbr said:

Kinda like I could care less what ASR measures a Pass Labs component at ...

Thought of you the last time I read an ASR Pass Labs review.   

[bluesman]

16 minutes ago, The Computer Audiophile said:

I certainly hear you but if there’s harm in unmeasurable USB cables then the same harm is there for unhearable measurements. 
 

I don’t see any harm, but the double standard is blatant. 

Perhaps you missed the animated disclaimer I added at the end, Chris. I love it so much that here it is again to underscore how seriously you should take my response above.

[The Computer Audiophile]

51 minutes ago, Summit said:

 

Not all objectivists believe that. Many objectivists and subjectivist know that noise, jitter etc despite not being directly audible can have negative affect on other electrical devices and other audio gear downstream.

 

What’s the point to manipulate the measurements, it is literally like open a can of worms?

We don't need to go around the loop again and state that not all objectivists believe etc... Of course not all of anything is one way. 

 

Nobody is suggesting manipulating measurements. 

[Qhwoeprktiyns]

Don't know if this covers noise across all components, but it seems to me that the question of noise in the digital domain should be treated seperately, as it may be completely inaudible but may disrupt clocks and the DAC and have consequently audible effects - or maybe that's obvious? (not to Archimago, though, who was mentioned here). 

[fas42]

There is absolutely nothing "below the level of audibility" that's important, in a direct sense - what it may tell you is how robust the component is, when faced with interference factors, and similar possibly degrading influences.

 

What I'm sure deeply offends many objectivists is that, as claimed by many, the sound of a rig can dramatically change when absolutely nothing is done which will impact all the measurements they like to rely on - many of them resolve this dilemma by insisting that "it's all in your head!!" ... they are not happy that variables that they don't have a firm grasp on are that important - so, sweep it under the carpet is the strategy they will to some degree cling to.

 

Current active speakers I'm playing with are a case in point - I can have the presentation range from "run out of the room with hands over the ears" SQ, up to approaching magic territory, simply by altering the electrical environment external to the units - as has been the case to some degree with every rig I've dealt with, over decades. Now, what the use of measurements of such items, in some laboratory setting, if those behaviours are not tested? This consideration would be highly disturbing to objectivists, so at least some will insist that such things are impossible, and will "over measure" what they think is important, as a type of compensation.

[Speedskater]

Why do automobile magazines measure dozens of parameters (and to a level of precision) that won't matter to the readers of the magazine, thinking about getting a new car?

[Currawong]

I think that much of the problem with noisy, online "objectivist" talk is that it is based on a lack of genuine knowledge of what measurements show. I've noticed a recent trend for hyped Chinese DACs to show their low SINAD numbers, yet avoid showing the measurements of their leaky filters. The problem with both is that neither have their relation, if any, to what we actually hear explained. 

 

For example, how often do you read about the effect of digital filters on our perception of soundstage?  The claims regarding the effects of distortion down to -200dB were specifically related to this.  While running a file-based blind test to determine blatant audibility is interesting, basing it on comments/claims which are related to specific hardware is not the same thing.

 

What is more, I think the contradiction between "objectivists" seeming to claim distortion differences are inaudible one minute, and the next minute complaining about inaudible side-bands on SINAD tests, comes down to a significant number of them wanting to be part of the bandwagon, and blindly following whatever their at-the-moment chosen leader is saying, without having any actual knowledge themselves, let alone a desire to consider the blatant contradictions in their their beliefs. 

 

A major issue, which I have given an example above in my wording in the paragraph above, and I think which Chris' original post is also a good example of, is attachment to the same, repeated, simplistic ideas about both audio science and people, starting with the idea of anything, whether it be a measurement artefact or changing a physical product, being straight-up audible or inaudible, as with the huge variety of electronics and music we have available to us, as well as the complexity of it, these things clearly cannot be declared as absolutes using numbers.

[fas42]

Or ... objectivists are by instinct reductionists, https://en.wikipedia.org/wiki/Reductionism, and the human hearing system is inherently holistic in operation; subjectivists by contrast are attracted to https://en.wikipedia.org/wiki/Antireductionism

[pkane2001]

27 minutes ago, Currawong said:

I think that much of the problem with noisy, online "objectivist" talk is that it is based on a lack of genuine knowledge of what measurements show. I've noticed a recent trend for hyped Chinese DACs to show their low SINAD numbers, yet avoid showing the measurements of their leaky filters. The problem with both is that neither have their relation, if any, to what we actually hear explained. 

 

For example, how often do you read about the effect of digital filters on our perception of soundstage?  The claims regarding the effects of distortion down to -200dB were specifically related to this.  While running a file-based blind test to determine blatant audibility is interesting, basing it on comments/claims which are related to specific hardware is not the same thing.

 

What is more, I think the contradiction between "objectivists" seeming to claim distortion differences are inaudible one minute, and the next minute complaining about inaudible side-bands on SINAD tests, comes down to a significant number of them wanting to be part of the bandwagon, and blindly following whatever their at-the-moment chosen leader is saying, without having any actual knowledge themselves, let alone a desire to consider the blatant contradictions in their their beliefs. 

 

A major issue, which I have given an example above in my wording in the paragraph above, and I think which Chris' original post is also a good example of, is attachment to the same, repeated, simplistic ideas about both audio science and people, starting with the idea of anything, whether it be a measurement artefact or changing a physical product, being straight-up audible or inaudible, as with the huge variety of electronics and music we have available to us, as well as the complexity of it, these things clearly cannot be declared as absolutes using numbers.

Could you conceive that there are some objectivists that actually understand measurements, SINAD, jitter, filters, clocks, phase noise, etc, or are they all just ignorant? This being an objective forum, I’d like to see some objective evidence for your statement.

[kumakuma]

36 minutes ago, fas42 said:

Or ... objectivists are by instinct reductionists, https://en.wikipedia.org/wiki/Reductionism, and the human hearing system is inherently holistic in operation; subjectivists by contrast are attracted to https://en.wikipedia.org/wiki/Antireductionism

 

The tendency of subjectivists to reduce objectivists to a caricature is more in tune with the former philosophy rather than the latter.

[bluesman]

2 hours ago, fas42 said:

There is absolutely nothing "below the level of audibility" that's important, in a direct sense - what it may tell you is how robust the component is, when faced with interference factors, and similar possibly degrading influences.

If something is inaudible in isolation but has audible effects on other factors, its presence is audible even if it makes no sound of its own.  I think this is directly important.  Look no further than a silent person on a creaky step - you hear the normally silent step because an inaudible person stepped on it.  DC is silent - but a voltage drop that makes its way to an audio signal as a DC offset can affect SQ.

 

Then there’s the question of why it’s “below the level of audibility”.  Is it making sound at an SPL below the threshold of audibility? Is it producing AC out of the frequency range of audibility?  Or is it making otherwise audible sound that’s masked to inaudibility by other sounds in its environment?  Each cause has its own set of direct, audible consequences, eg intermodulation or sucking amplifier power.

[fas42]

42 minutes ago, bluesman said:

If something is inaudible in isolation but has audible effects on other factors, its presence is audible even if it makes no sound of its own.  I think this is directly important.  Look no further than a silent person on a creaky step - you hear the normally silent step because an inaudible person stepped on it.  DC is silent - but a voltage drop that makes its way to an audio signal as a DC offset can affect SQ.

 

Which is a variation on what I'm saying - measuring the output of the system in the electrical sense, and everything there being "below audibility" is what I was referring to - measuring somewhere within a component, or at a link in the chain is entirely different; because there the causal links that you refer to may exist. If the linked to component is extremely sensitive to some parameter that is well within reasonable limits from the preceding component, then everything changes.

 

[Josh Mound]

I could be wrong, but I think Chris’s point is that if we take “beyond audibility“ seriously, then it becomes a checked box rather than a “moar please” issue. There’s no “even more inaudible.” I’ve said before that I don’t see the point of someplace like ASR continuing to measure DACs. According to its own standards, a bunch of $99 DACs are “perfect.” 

 

But I do think there’s a danger in reducing DACs to one or two numbers, dumping them into a graph, then saying that higher is better. That’s especially the case if it’s really “all DACs with distortion below 80 dBFS are audibly transparent.” Even more so the case when, as Marv at SBAF has demonstrated, little issues with kinked cords, ground loops, etc. can make the same unit measure differently.
 

This is not, of course, to say that measurements don’t have immense value. But I think the limits to measurement fundamentalism (for lack of a better way to make clear that I’m not lumping everyone who likes measurements together) are already being seen with ASR’s foray into speaker measurements. People are trying to reduce Amir’s speaker measurements to one or two preference numbers. Yet he’s saying in his subjective reviews that speakers with virtually identical preference ratings sound very different (and resorting to some of the biggest audiophile writing cliches in the process!). This is, understandably, vexing some of his most loyal followers. 

I believe Amir can hear differences between two 6.8 (or whatever) preference-rated speakers. But I also think there are some people who can, even in blind testing, hear differences between two “perfect” DACs. There are certainly some people who can’t hear either. There are even many people who couldn’t tell a 4.5 preference rating speaker from a 6.8 one, or a 60 SINAD DAC from a 110 one (or, if they could tell a difference, couldn’t say which is more neutral).
 

I’m interested in measurements to catch major issues and to understand why we’re hearing what we’re hearing. I’m not interested in them as an arms race with no meaningful purpose. 

[Audiophile Neuroscience]

 

2 hours ago, kumakuma said:

 

The tendency of subjectivists to reduce objectivists to a caricature is more in tune with the former philosophy rather than the latter.

 

Caricatures of objectivists ? I dunno about anyone else but i see them as funny little bears holding cups !

[The Computer Audiophile]

8 minutes ago, JoshM said:

I could be wrong, but I think Chris’s point is that if we take “beyond audibility“ seriously, then it becomes a checked box rather than a “moar please” issue. There’s no “even more inaudible.” I’ve said before that I don’t see the point of someplace like ASR continuing to measure DACs. According to its own standards, a bunch of $99 DACs are “perfect.” 

 

But I do think there’s a danger in reducing DACs to one or two numbers, dumping them into a graph, then saying that higher is better. That’s especially the case if it’s really “all DACs with distortion below 80 dBFS are audibly transparent.” Even more so the case when, as Marv at SBAF has demonstrated, little issues with kinked cords, ground loops, etc. can make the same unit measure differently.
 

This is not, of course, to say that measurements don’t have immense value. But I think the limits to measurement fundamentalism (for lack of a better way to make clear that I’m not lumping everyone who likes measurements together) are already being seen with ASR’s foray into speaker measurements. People are trying to reduce Amir’s speaker measurements to one or two preference numbers. Yet he’s saying in his subjective reviews that speakers with virtually identical preference ratings sound very different (and resorting to some of the biggest audiophile writing cliches in the process!). This is, understandably, vexing some of his most loyal followers. 

I believe Amir can hear differences between two 6.8 (or whatever) preference-rated speakers. But I also think there are some people who can, even in blind testing, hear differences between two “perfect” DACs. There are certainly some people who can’t hear either. There are even many people who couldn’t tell a 4.5 preference rating speaker from a 6.8 one, or a 60 SINAD DAC from a 110 one (or, if they could tell a difference, couldn’t say which is more neutral).
 

I’m interested in measurements to catch major issues and to understand why we’re hearing what we’re hearing. I’m not interested in them as an arms race with no meaningful purpose. 

Absolutely. 100%
 

I don’t care where people are on the continuum of subjective and objective views. I just hate double standards and people on high horses looking down at others. 
 

If someone is a “measurements or it isn’t real” type of person that’s fine. However, they must also use all objective information rather than cherry pick. There are serious limits to human hearing that are ignored when convenient but touted when needed. For example, saying high resolution audio doesn’t matter because humans can’t hear above 20 kHz is fine with me, but then one should also say measurements below the level of human hearing don’t matter. 

[Audiophile Neuroscience]

Trust my measurements and trust my ears can both mislead -

IMO the problem in the case of measurements, presuming they are accurate, is in their interpretation. Double standards may come into play if *some* people manipulate that interpretation to serve their agenda or belief system.

[pkane2001]

59 minutes ago, JoshM said:

I’m interested in measurements to catch major issues and to understand why we’re hearing what we’re hearing. I’m not interested in them as an arms race with no meaningful purpose. 

 

I agree 100%.

 

[fas42]

1 hour ago, JoshM said:

I’m interested in measurements to catch major issues and to understand why we’re hearing what we’re hearing. I’m not interested in them as an arms race with no meaningful purpose. 

 

Now, this is what intrigues me ... why are people interested in measurements "to catch major issues" ?? If the SQ is audibly faulty then you have a significant issue - getting numbers is irrelevant; what needs to be done is to determine what is causing the problem ...if you take a misbehaving TV to a service technician he is not going to waste his time extracting numbers - he can see the picture is not up to scratch, and the only use of numbers is to help diagnose internal to the circuitry - his goal is to locate the fault, and decide whether to chuck the set, or replace some misbehaving parts.

[John Dyson]

1 hour ago, Audiophile Neuroscience said:

Trust my measurements and trust my ears can both mislead -

IMO the problem in the case of measurements, presuming they are accurate, is in their interpretation. Double standards may come into play if *some* people manipulate that interpretation to serve their agenda or belief system.

A simplistic interpretation of an objective measurement can be very misleading, and bias the evaluation.   Likewise, depending on ones hearing for comparisons, especially when *accurate* aural memory is pretty short, can also be misleading and bias the evaluation.  This is why subjective measurements should be done with proper scientific/statistical methods.  Likewise, comparing measurements without knowing & controlling the conditions will likely be misleading.  (Even certain OBJECTIVE measurements might require experimental/statistical discipline.)

 

It seems in the audio realm -- much of the misunderstanding/disagreement comes from the *understandable* lack of knowledge about the math, the electronics to the physics level, lack of understanding of human hearing, understanding the value of statistics -- and lack of realization of personal variabilities in hearing.   Most importantly, not respecting the how/why to control experiments.   As a collection, being such a wide expert is NOT easy, and this is NOT an undergraduate subject -- this is a field that requires understanding, study and lack of bia$$$.  Otherwise, erroneous info WILL be presented.   * (I am not claiming to be such an expert -- just throwing in some REALISM about the variables we are dealing with.)   Usually, in a commercial situation, there is a group of people that MUTUALLY provide the needed skills and checks and balances.

 

One lifesaving discipline for 'us little people' who have limited resources and perhaps not enough knowledge in all of the fields:  *controlled experiments* with proper application of simple statistics.

 

* Knowing and believing  the potential mistakes about comparisons and measurements would make claims a lot more measured.

 

'My opinion' is a good thing.   Or measured suggestions for improvements, those are also good things.  There are LOTS of variables in the real world -- and they sometimes scare me away from participating unless I know exactly the design/environment that we are talking about.

 

John

 

[Audiophile Neuroscience]

1 hour ago, John Dyson said:

A simplistic interpretation of an objective measurement can be very misleading, and bias the evaluation.   Likewise, depending on ones hearing for comparisons, especially when *accurate* aural memory is pretty short, can also be misleading and bias the evaluation.  This is why subjective measurements should be done with proper scientific/statistical methods.  Likewise, comparing measurements without knowing & controlling the conditions will likely be misleading.  (Even certain OBJECTIVE measurements might require experimental/statistical discipline.)

 

It seems in the audio realm -- much of the misunderstanding/disagreement comes from the *understandable* lack of knowledge about the math, the electronics to the physics level, lack of understanding of human hearing, understanding the value of statistics -- and lack of realization of personal variabilities in hearing.   Most importantly, not respecting the how/why to control experiments.   As a collection, being such a wide expert is NOT easy, and this is NOT an undergraduate subject -- this is a field that requires understanding, study and lack of bia$$$.  Otherwise, erroneous info WILL be presented.   * (I am not claiming to be such an expert -- just throwing in some REALISM about the variables we are dealing with.)   Usually, in a commercial situation, there is a group of people that MUTUALLY provide the needed skills and checks and balances.

 

One lifesaving discipline for 'us little people' who have limited resources and perhaps not enough knowledge in all of the fields:  *controlled experiments* with proper application of simple statistics.

 

* Knowing and believing  the potential mistakes about comparisons and measurements would make claims a lot more measured.

 

'My opinion' is a good thing.   Or measured suggestions for improvements, those are also good things.  There are LOTS of variables in the real world -- and they sometimes scare me away from participating unless I know exactly the design/environment that we are talking about.

 

John

 

 

I agree John. I think to some degree we all may have difficulty seeing beyond our own conditioned myopia or way of looking at things......

 

[Kal Rubinson]

5 hours ago, Audiophile Neuroscience said:

I agree John. I think to some degree we all may have difficulty seeing beyond our own conditioned myopia or way of looking at things...... 

Can you post/link a better copy of the cartoon?  I cannot read the text even if I enlarge it.  Thanks.

[sphinxsix]

10 minutes ago, Kal Rubinson said:

Can you post/link a better copy of the cartoon?  I cannot read the text even if I enlarge it.  Thanks.

Just try to click on it 3 times, it's huge.

[kumakuma]

11 hours ago, The Computer Audiophile said:

For example, saying high resolution audio doesn’t matter because humans can’t hear above 20 kHz is fine with me, but then one should also say measurements below the level of human hearing don’t matter. 

 

This is unworkable because it would require everyone to agree on what this level is.

[The Computer Audiophile]

5 minutes ago, kumakuma said:

 

This is unworkable because it would require everyone to agree on what this level is.

Why everyone? Can’t we just used scientifically established objective data? Paul says it has existed for quite a while. 

[kumakuma]

1 minute ago, The Computer Audiophile said:

Why everyone? Can’t we just used scientifically established objective data? Paul says it has existed for quite a while. 

 

If it exists, I don't know what it is. Do you?