Misleading Measurements

[The Computer Audiophile]

I was thinking about this the other day and came to the conclusion that measurements are almost always published in a misleading way. Here’s why. 
 

Objectivists believe they know the threshold or level of audibility with respect to measurements. Any anomaly below that level is thus meaningless for people listening to music. It stands to reason that objective leaning people who are truly out to follow the data and help people unearth the truth, should only publish a pass fail style of measurement. 
 

If a component has no issues above the threshold of human hearing, it can only mislead people if these measurements are published. Components without issues should be given a stamp of approval and that’s it, if the true goal was objective info and to stop people from wasting money like is so often said. 
 

I would love to know why objective people are interested in anything below the threshold of human hearing and why they are interested in this info given that it can cause the same issues they rail against with respect to subjective opinions. 
 

Perhaps I should ask @Archimago why he publishes measurements below the threshold of human hearing. 

[The Computer Audiophile]

I’m seeking input from objective members of the community on this topic. Their logic is incongruous to me and I want to understand it. 

[yamamoto2002]

Measurement is interesting. It shows engineers' design decision. Sometimes it is trade-off.

 

Also sometimes it shows glitch. Even if the noise caused by the glitch is well below the hearing threshold, it is better to be glitch-free IMO. Here it is example of my coding bug. The periodical click noise and ultrasonic tone is below hearing threshold but the graph implies off-by-1 error in my 2x upsample program 😁

 

[The Computer Audiophile]

But the whole thing about objectivists is audibility. They rail against subjectivists for talking about things that can’t be heard, yet they do the exact same thing. 
 

If it’s problematic for people to discuss that which can’t be heard, it must work both ways. 

[fas42]

Putting on an objectivist's hat, I would argue that it's a https://en.wikipedia.org/wiki/Factor_of_safety, an engineering safeguard to ensure that in all possible circumstances that the audibility will always lie well below what's borderline audible.

[jabbr]

The saying: lies, damn lies and then statistics comes to mind.

 

Scientists demand reproducibility. 
 

Think of a scientific experiment as: what will it take for me to convince you?

 

If you have a reputation for bias, or an agenda,  then I am not taking your measurements at face value. Kinda like I could care less what ASR measures a Pass Labs component at ...

[The Computer Audiophile]

16 minutes ago, fas42 said:

Putting on an objectivist's hat, I would argue that it's a https://en.wikipedia.org/wiki/Factor_of_safety, an engineering safeguard to ensure that in all possible circumstances that the audibility will always lie well below what's borderline audible.

I don’t see how this relates to objectivists disliking that which can’t be heard but also publicizing that which can’t be heard when it fits an agenda. 

[Audiophile Neuroscience]

Okay, I see a few posts have been deleted as presumably OT. If understanding, the premise is that of a double standard - that objectivists might be publishing misleading measurements deemed to be inaudible (below differential, absolute thresholds whatever) while criticizing subjectivists for pondering the possibility of hearing such things.

 

I would say it is only misleading/disingenuous/ of a double standard if those measurements are said to be in some way relevant to performance.One would expect that, if they believed in what they say elsewhere, they should point out that whatever measurement is irrelevant.

 

The question still remains then, why publish measurements of "inaudibility" in the first place. Being objective (but not an objectivist) I would simply say you do the test/s and the results fall where they fall. It is objectively helpful to know that any flaws revealed might be (allegedly are) inaudible.

 

 

[fas42]

53 minutes ago, The Computer Audiophile said:

I don’t see how this relates to objectivists disliking that which can’t be heard but also publicizing that which can’t be heard when it fits an agenda. 

 

It's also an indicator of quality - that the design and manufacture is to such a high standard that they are well below what's necessary for what they believe to be "the threshold or level of audibility" - implying that other qualities which are not so easily measured are of a commensurate standard.

 

As an objectivist, I would want the best that's possible in performance, for peace of mind - I don't feel that I'm missing out of anything; that there's no chance that there is a better system out there that has better specs, which may give better subjective results, 😉.

[MetalNuts]

May be the specs are not published for one to hear rather they are there to be seen as a comparison or competition with the others.  Too bad if you cannot hear the difference even there is big difference in the specs however, if one can prove the specs are not accurate as claimed then there will be adverse consequences.  However, how many will take action if it still sounds good despite the deviation from the specs.

[pkane2001]

13 hours ago, The Computer Audiophile said:

I was thinking about this the other day and came to the conclusion that measurements are almost always published in a misleading way. Here’s why. 
 

Objectivists believe they know the threshold or level of audibility with respect to measurements. Any anomaly below that level is thus meaningless for people listening to music. It stands to reason that objective leaning people who are truly out to follow the data and help people unearth the truth, should only publish a pass fail style of measurement. 
 

If a component has no issues above the threshold of human hearing, it can only mislead people if these measurements are published. Components without issues should be given a stamp of approval and that’s it, if the true goal was objective info and to stop people from wasting money like is so often said. 
 

I would love to know why objective people are interested in anything below the threshold of human hearing and why they are interested in this info given that it can cause the same issues they rail against with respect to subjective opinions. 
 

Perhaps I should ask @Archimago why he publishes measurements below the threshold of human hearing. 

 

Chris, perhaps it's hard to believe, but not every objectivist is the same. We don't all think the same way and believe the same things.

 

As an example, @Archimago and I just ran a blind test to determine audibility of harmonic distortion. We ran it not because we believed it was inaudible at -50dB or audible to the -200dB level (as some in the industry will claim), but because we wanted to find out.

 

There's a large contingent on ASR that will tell you that harmonic distortion at -120dB is much better than -115dB. In an engineering sense, this is true: lower distortion is more transparent than higher, more 'pure'. But is this difference really audible? And could it be that we, audiophiles, prefer a little distortion to none?  That's why it's important to study these things, and running tests and experiments is the way to do this.

 

Proper engineering has a lot to do with lowering distortions and producing better measurements, even when these are well below audibility. Some of us prefer better engineered products, especially if they are less expensive than some high-end ones that are engineered poorly. Measurements are the way to determine this. Not if it sounds better, but if the design is sound and engineering is done well.

 

The point of doing measurements is that they allow anyone to interpret the results based on their knowledge and understanding, which may change over time. Measurements do not include bias and personal opinion. I know you take issue with Amir's subjective recommendations. I do too. I don't find them useful, as I don't believe he's completely free of bias (no one is). But his measurements are, and this is confirmed by others producing the same results.

[The Computer Audiophile]

4 minutes ago, pkane2001 said:

 

Chris, perhaps it's hard to believe, but not every objectivist is the same. We don't all think the same way and believe the same things.

 

As an example, @Archimago and I just ran a blind test to determine audibility of harmonic distortion. We ran it not because we believed it was inaudible at -50dB or audible to the -180dB level (as some in the industry will claim), but because we wanted to find out.

 

There's a large contingent on ASR that will tell you that harmonic distortion at -120dB is much better than -115dB. In an engineering sense, this is true: lower distortion is more transparent than higher, more 'pure'. But is this difference really audible? And could it be that we, audiophiles, prefer a little distortion to none?  That's why it's important to study these things, and running tests and experiments is the way to do this.

 

Proper engineering has a lot to do with lowering distortions and producing better measurements, even when these are well below audibility. Some of us prefer better engineered products, especially if they are less expensive than some high-end ones that are engineered poorly. Measurements are the way to determine this. Not if it sounds better, but if the design is sound and engineering is done well.

 

The point of doing measurements is that they allow anyone to interpret the results based on their knowledge and understanding, which may change over time. Measurements do not include bias and personal opinion. I know you take issue with Amir's subjective recommendations. I do too. I don't find them useful, as I don't believe he's completely free of bias (no one is). But his measurements are, and this is confirmed by others producing the same results.

Hi Paul, thanks for the thoughtful comments. 

 

As you can imagine, I see this differently, not with respect to your opinions of course, but with the efficacy of some measurements in light of the objectivist party line that berates audiophiles for talking about stuff that they believe can't be heard. I just don't see how objective leaning people can have it both ways, with a straight face. 

 

Objectivists often hate discussions of things like USB cables, claiming there are no measurements that can show a difference between them. The discussion often includes that these cables are bad for the industry, scare people away and mislead people into purchasing stuff they don't need. 

 

I see the discussion of inaudible measurements as being the other side of that coin. A DAC that measures -130 dB is worse than a DAC that measures at -131 dB. The better measuring DAC will be put on a pedestal and listed at the #1 DAC. This will no doubt cause people to purchase the DAC over others that may measure at -129 dB, -128 dB etc... It's human nature and there's no getting around it. 

 

Personally I don't mind the measurements and think adults can make up their own minds and purchase what they want. It just irks me that objectivists, who have goals other than looking at graphs to satisfy themselves, eschew one thing they claim is inaudible but consider the other inaudible items laudable. 

 

If we are solely talking about engineering feats of something like the lowest noise floor, then by all means show the measurements. But, that reminds me of the car audio competitions for the loudest sounds within the cabin of the car. What's the point. I suppose some people could purchase equipment for reasons other than listening and that's OK, but if the reason for components is to listen, then it makes zero sense to care about that which is inaudible. 

 

Surely the audibility of jitter is something that objectivists can agree on. There must be a generally accepted number, below which is inaudible. Take that number and go lower by 10%. All jitter measurements below that shouldn't matter and can only serve to mislead people, if the accepted party line of objectivists is true and inaudible stuff doesn't matter. Thus, showing a pass fail for jitter should be the prudent way to display this info. 

 

Note: I'm not arguing for this position. I'm only pointing out what I see as a double standard and I'm seeking to understand why it's pushed so hard.

 

 

 

 

 

 

[kumakuma]

23 minutes ago, pkane2001 said:

 

Chris, perhaps it's hard to believe, but not every objectivist is the same. We don't all think the same way and believe the same things.

 

 

And why I find this thread both pointless and insulting...

[The Computer Audiophile]

23 minutes ago, kumakuma said:

 

And why I find this thread both pointless and insulting...

Please help me understand rather than just complain. 

 

If we all posted all the caveats and possibilities, there's be nothing worth reading. For example, objectivists who love measurements, except those who like them and don't view them as the end all be all, and except those who only go by measurements, and except those who only love some of them, and except those who continue to do experiments themselves, etc... 

 

It gets pointless. We have to have some leeway when writing and reading that allowed people to discuss topics without carving out exceptions for all possibilities. 

[pkane2001]

19 minutes ago, The Computer Audiophile said:

Hi Paul, thanks for the thoughtful comments. 

 

As you can imagine, I see this differently, not with respect to your opinions of course, but with the efficacy of some measurements in light of the objectivist party line that berates audiophiles for talking about stuff that they believe can't be heard. I just don't see how objective leaning people can have it both ways, with a straight face. 

 

Objectivists often hate discussions of things like USB cables, claiming there are no measurements that can show a difference between them. The discussion often includes that these cables are bad for the industry, scare people away and mislead people into purchasing stuff they don't need. 

 

I see the discussion of inaudible measurements as being the other side of that coin. A DAC that measures -130 dB is worse than a DAC that measures at -131 dB. The better measuring DAC will be put on a pedestal and listed at the #1 DAC. This will no doubt cause people to purchase the DAC over others that may measure at -129 dB, -128 dB etc... It's human nature and there's no getting around it. 

 

Personally I don't mind the measurements and think adults can make up their own minds and purchase what they want. It just irks me that objectivists, who have goals other than looking at graphs to satisfy themselves, eschew one thing they claim is inaudible but consider the other inaudible items laudable. 

 

If we are solely talking about engineering feats of something like the lowest noise floor, then by all means show the measurements. But, that reminds me of the car audio competitions for the loudest sounds within the cabin of the car. What's the point. I suppose some people could purchase equipment for reasons other than listening and that's OK, but

 

Surely the audibility of jitter is something that objectivists can agree on. There must be a generally accepted number, below which is inaudible. Take that number and go lower by 10%. All jitter measurements below that shouldn't matter and can only serve to mislead people, if the accepted party line of objectivists is true and inaudible stuff doesn't matter. Thus, showing a pass fail for jitter should be the prudent way to display this info. 

 

Note: I'm not arguing for this position. I'm only pointing out what I see as a double standard and I'm seeking to understand why it's pushed so hard.

 

Well, audibility thresholds are not some voodoo magic, they are not a new concept. These can be (and have been) studied. I myself invested a lot of time and effort to create tools to aid in such studies. I wouldn't waste my time if I believed that everything is known and there's nothing new to learn in this space.

 

24 minutes ago, The Computer Audiophile said:

It just irks me that objectivists, who have goals other than looking at graphs to satisfy themselves, eschew one thing they claim is inaudible but consider the other inaudible items laudable. 

 

There's no accounting for tastes or preferences. A "true objectivist", if such a thing exists, would question any claims of audibility/inaudibility and look for real evidence to demonstrate that there is a correlation between some measurement and audibility, as well as measurements and preferences. Many of the papers I cite on the BIAS in Testing thread are designed to study exactly this.

 

33 minutes ago, The Computer Audiophile said:

if the reason for components is to listen, then it makes zero sense to care about that which is inaudible

 

Agreed!

 

 

[The Computer Audiophile]

3 minutes ago, pkane2001 said:

Well, audibility thresholds are not some voodoo magic, they are not a new concept. These can be (and have been) studied. I myself invested a lot of time and effort to create tools to aid in such studies. I wouldn't waste my time if I believed that everything is known and there's nothing new to learn in this space.

I'm with you on this one. There are studies that the objective crowd accepts and that's totally cool with me. In essence, that's the basis for my entire topic here. 

 

I just don't understand why objectivists love some inaudible aspects of this hobby and eschew others. 

 

[pkane2001]

6 minutes ago, The Computer Audiophile said:

Please help me understand rather than just complain. 

 

If we all posted all the caveats and possibilities, there's be nothing worth reading. For example, objectivists who love measurements, except those who like them and don't view them as the end all be all, and except those who only go by measurements, and except those who only love some of them, and except those who continue to do experiments themselves, etc... 

 

It gets pointless. We have to have some leeway when writing and reading that allowed people to discuss topics without carving out exceptions for all possibilities. 

 

I'm not at all complaining about this thread, I like the ability to have a rational discussion. But, I've experienced over and over again this attempt to lump all the objectivists into a single straw-man, with all the extreme views rolled into one. As if we are all one individual and all think the same. I assume that's what @kumakuma is reacting to. Perhaps it's human nature, because I see the same occurring on ASR, but in the opposite direction.

[pkane2001]

5 minutes ago, The Computer Audiophile said:

I just don't understand why objectivists love some inaudible aspects of this hobby and eschew others. 

 

Beats me . It's irrational.

 

[The Computer Audiophile]

6 minutes ago, pkane2001 said:

 

I'm not at all complaining about this thread, I like the ability to have a rational discussion. But, I've experienced over and over again this attempt to lump all the objectivists into a single straw-man, with all the extreme views rolled into one. As if we are all one individual and all think the same. I assume that's what @kumakuma is reacting to. Perhaps it's human nature, because I see the same occurring on ASR, but in the opposite direction.

It's just not possible to carve out all the exceptions while maintaining some readability. I highly encourage people to read with their glass half full and realize All and None statements aren't meant to group everyone, they are just used to further discussion. 

 

If people don't want to have a discussion without all the caveats carved out, that's fine too but I believe they'll be looking quite a while to find an acceptable discussion. 

[bluesman]

50 minutes ago, The Computer Audiophile said:

I highly encourage people to read with their glass half full

As a flexible and rational objectivist, I focus first and foremost on what’s in the glass. Then I analyze the data to see if the level is stable, rising or falling.  If it’s stable, is it static or in a dynamic equilibrium?  If it’s falling, where’s the leak and where is the loss going?  If it’s rising, what’s the source and do I really want or need more of whatever’s in it?  
 

Etc.

[Summit]

First of all not all objectivists or subjectivist are the same. I know objectivists that strongly believe that equipment that measures better also will sound better. Objectivists which truly believes that every significant aspect can be measured, and that people which prefer inferior measuring gear does it solely because “they like a bit of coloration”. The correlation between measured noise and actual SQ is very strong according to them. We of course have other types of objectivists like the naysayers and anti-audiophiles, but I believe those are small subgroup, even if they tend to be quite vocal.

 

I don’t get why presenting measurements "below level of audibility" would be consider misleading. To do the opposite and not show how a gear actually measured OTOH would be far more misleading and confusing IMO.  

[The Computer Audiophile]

30 minutes ago, Summit said:

don’t get why presenting measurements "below level of audibility" would be consider misleading. To do the opposite and not show how a gear actually measured OTOH would be far more misleading and confusing IMO

Great point. 
 

If, as objective leaning people believe and according to studies, there is something below the level of audibility, what is the benefit to consumers to know this information? What is the possible harm to consumers?

[bluesman]

14 minutes ago, The Computer Audiophile said:

What is the possible harm to consumers?

Sleepless nights wondering why they can’t hear a difference.

[The Computer Audiophile]

2 minutes ago, bluesman said:

Sleepless nights wondering why they can’t hear a difference.

I certainly hear you but if there’s harm in unmeasurable USB cables then the same harm is there for unhearable measurements. 
 

I don’t see any harm, but the double standard is blatant. 

[Summit]

27 minutes ago, The Computer Audiophile said:

Great point. 
 

If, as objective leaning people believe and according to studies, there is something below the level of audibility, what is the benefit to consumers to know this information? What is the possible harm to consumers?

 

Not all objectivists believe that. Many objectivists and subjectivist know that noise, jitter etc despite not being directly audible can have negative affect on other electrical devices and other audio gear downstream.

 

What’s the point to manipulate the measurements, it is literally like open a can of worms?