Misleading Measurements

[pkane2001]

11 hours ago, opus101 said:

 

Measurements are analogous to the grammar of audio. Try convincing someone you met in the East End of London with a Cockney drawl that he needs grammar. What he has works for him, why bother him with grammar?

 

Unlike language and grammar, there is an underlying physical reality to audio. Ignoring physical reality is always an option, but there are often consequences to doing so (try to ignore gravity when jumping off a cliff, for example).

[opus101]

Grammar is analysis of words - text and speech have physical reality.

[Clockmeister]

Just now, opus101 said:

 

Yes. The precise instantiation is immaterial.

 

Give the level of pedantry on this thread I just felt in the interests of equality, with readers on both sides of the pond it seemed appropriate.

 

One of the posters made a most pertinent observation regarding both camps on this subject, trying to persuade one side or the other with the benefits  or system 'x' methodology compared to process 'y' methodology with their mindset buried firmly on one camp or the other is akin to convincing BMW owners that indicators are standard fitment on their vehicles or conversely Budwiser is anything other that goats piss in a can?

 

Stubborn intransigence coupled with self affirmation is not a good place from being totally objective with a clean open sheet.

[The Computer Audiophile]

55 minutes ago, pkane2001 said:

 

Unlike language and grammar, there is an underlying physical reality to audio. Ignoring physical reality is always an option, but there are often consequences to doing so (try to ignore gravity when jumping off a cliff, for example).

Getting back on topic.

 

The underlying physical reality of the ear/brain is that we can’t pick and choose what we can hear. We can however pick and choose which unhearable measurements we like to champion and which ones we write off as snake oil. That’s what interests me. 

[Racerxnet]

Can the numbers evolve into something more useful in the future by trying to identify shortcomings, or do we ignore them because we cant hear below a certain value?  

[pkane2001]

14 minutes ago, The Computer Audiophile said:

Getting back on topic.

 

The underlying physical reality of the ear/brain is that we can’t pick and choose what we can hear. We can however pick and choose which unhearable measurements we like to champion and which ones we write off as snake oil. That’s what interests me. 

 

You're asking for feedback from some amorphous group that's picking and choosing what measurements they like, and arguing that it's misleading. Unless this is a rhetorical question, you may want to identify at least some members of that group, and ask them. Or, at least, point to some examples where they are being misleading. @Archimago was one that you identified in the OP. Can you perhaps point to where he's sharing something that's misleading or hypocritical (words you used throughout this thread)? Not that I want to argue on his behalf, but my perception is that what he's been sharing is not misleading in the least.

[Bill Brown]

1 hour ago, opus101 said:

Grammar is analysis of words - text and speech have physical reality.

 

Definitely.  Otherwise a whole field of science isn't really science.  Thankfully our brains are very real, though admittedly, our understanding of it relative to other organs could still be described as fairly primitive. 

 

Bill

[Jud]

50 minutes ago, pkane2001 said:

 

You're asking for feedback from some amorphous group that's picking and choosing what measurements they like, and arguing that it's misleading. Unless this is a rhetorical question, you may want to identify at least some members of that group, and ask them. Or, at least, point to some examples where they are being misleading. @Archimago was one that you identified in the OP. Can you perhaps point to where he's sharing something that's misleading or hypocritical (words you used throughout this thread)? Not that I want to argue on his behalf, but my perception is that what he's been sharing is not misleading in the least.

 

I'll jump in with a what rather than a who: I have seen praise for low jitter figures in Benchmark DACs by people who have elsewhere said the importance of jitter is greatly exaggerated. (Benchmark themselves have stated that asynchronous USB input and DSD are unnecessary, then built both into their DACs; and have praised at least the former in their marketing. This is entirely aside from the DAC's performance - I don't own one but certainly the measured performance appears to be excellent.)

[pkane2001]

24 minutes ago, Jud said:

 

I'll jump in with a what rather than a who: I have seen praise for low jitter figures in Benchmark DACs by people who have elsewhere said the importance of jitter is greatly exaggerated. (Benchmark themselves have stated that asynchronous USB input and DSD are unnecessary, then built both into their DACs; and have praised at least the former in their marketing. This is entirely aside from the DAC's performance - I don't own one but certainly the measured performance appears to be excellent.)

 

I generally agree, Jud. But quoting the actual performance of a device is not misleading. The measurement is a physical property of the device, unless the measurement itself is badly mangled.

 

Saying that 1fs of jitter is much better than, say, 100fs of jitter would be misleading (using a very simplistic example). It certainly could be true in an engineering sense, but not significant to an average consumer. Might be useful to me, as I often use DACs and ADCs for measurement. As long as the reviewer indicates that the difference between 1fs and 100fs jitter isn't audible, I don't see any hypocrisy in reporting the actual measurement.

Archimago, to use an already mentioned example, not only quotes measured jitter numbers, but also has a full blog article on the audibility of jitter. Nothing misleading in that, at least to me.

[manisandher]

Talking of Benchmark, the current DAC3 measures well below the threshold of audibily, if 'objectivists' are to be believed. And yet there are a whole bunch of people over at ASR who can't wait for the DAC4 to be released.

 

I suppose Chris's question would be, why?

 

Mani.

[pkane2001]

3 minutes ago, manisandher said:

Talking of Benchmark, the current DAC3 measures well below the threshold of audibily, if 'objectivists' are to be believed. And yet there are a whole bunch of people over at ASR who can't wait for the DAC4 to be released.

 

I suppose Chris's question would be, why?

 

Mani.

 

Ask them? I doubt any of them are active participants here.

[manisandher]

19 minutes ago, pkane2001 said:

Ask them?

 

I don't think so. It's not in my nature to cause a stir 😉.

 

FWIW, I had a Matrix Audio Element X here for a couple of months earlier this year, with a view to buying it for streaming in my office system. Taking my own measurements, it measured better than the Phasure DAC I had in my office at the time - substantially lower THD and noise floor.

 

But it didn't sound anywhere near as good to my ears - washed out and flat in comparison to the Phasure. So it went back to the dealer for a full refund. (I still changed the Phasure though... but for one of my old turntables. It's been fun exploring my record collection again... though I'd never go back to vinyl permanently.)

 

Mani.

[Clockmeister]

Well the BM dac's may measure well but sonically they are very dry and matter of fact.  The BM dac2 is more preferable for many listeners although not as technically competent specifications wise as the MK3.

[tapatrick]

51 minutes ago, Clockmeister said:

Well the BM dac's may measure well but sonically they are very dry and matter of fact.

As a possible member of an 'amorphous' group and as someone only mildly interested in measurements, if this statement is true (i.e. putting me off a BM DAC) can anyone explain why I should be more interested in measurements? Honest question.

[pkane2001]

2 hours ago, manisandher said:

 

I don't think so. It's not in my nature to cause a stir 😉.

 

FWIW, I had a Matrix Audio Element X here for a couple of months earlier this year, with a view to buying it for streaming in my office system. Taking my own measurements, it measured better than the Phasure DAC I had in my office at the time - substantially lower THD and noise floor.

 

But it didn't sound anywhere near as good to my ears - washed out and flat in comparison to the Phasure. So it went back to the dealer for a full refund. (I still changed the Phasure though... but for one of my old turntables. It's been fun exploring my record collection again... though I'd never go back to vinyl permanently.)

 

Mani.

 

Mani, you have all the right to enjoy what you like. That doesn't really mean that others will hear or like the same thing you do. You may have seen this play out in some older discussions of a certain Schiit DAC.  Not everyone agrees on what sounds best.

 

[fas42]

 

7 hours ago, manisandher said:

 

But it didn't sound anywhere near as good to my ears - washed out and flat in comparison to the Phasure. So it went back to the dealer for a full refund. (I still changed the Phasure though... but for one of my old turntables. It's been fun exploring my record collection again... though I'd never go back to vinyl permanently.)

 

Mani.

 

Yes, this is the one of those "highly relevant but ignored, because we don't how to measure it" distortion behaviours - to compensate, just measure everything where it's easy to do so twice as much ... surely in the end all that vigorous probing will deliver a superlative result, just from all the energy put into the number readin', 😉.

[Audiophile Neuroscience]

5 hours ago, pkane2001 said:

 

Mani, you have all the right to enjoy what you like. That doesn't really mean that others will hear or like the same thing you do. You may have seen this play out in some older discussions of a certain Schiit DAC.  Not everyone agrees on what sounds best.

 

 

The point is not what Mani likes or doesn't like, the point is measurements often do not correlate with what we hear, like it or not.

[Audiophile Neuroscience]

10 hours ago, pkane2001 said:

there is an underlying physical reality to audio.

 

Yes but there is also an underlying physical reality to neurophysiology.

 

Both these realities can be measured but at the end of the day you have to decide what measurements matter . The measures aren't what we hear. At the end of the day, if music enjoyment faithful to the experience of real physical life matters, measurements can be useful guides when judiciously applied, not a substitute for the real thing. 😀😉

[pkane2001]

37 minutes ago, Audiophile Neuroscience said:

 

The point is not what Mani likes or doesn't like, the point is measurements often do not correlate with what we hear, like it or not.

The point is that there can be no generalized correlation if individual’s preferences are involved. There will always be someone who will prefer something different.

[fas42]

The agenda of those with an objectivist stance is that measurements always win. Over people. If there is a dispute whether the measurements are right, or people's hearing is right - those on the other side will always give the thumbs up to the numbers; because, people can't be trusted, 😜.

 

Their determination is so strong that this is "how things work!" ... which means that they do whatever is needed to undermine the 'subjectivists' - ridicule, humour, contempt, anger, patronising tone, scorn ... whatever it takes - all that matters is that their stance is never threatened to any degree, by the possibility that they may be even a tiny bit wrong ... 😉

[The Computer Audiophile]

3 minutes ago, fas42 said:

The agenda of those with an objectivist stance is that measurements always win. Over people.

I don’t see it that way and that isn’t the crux of this thread. If measurements always win, this thread wouldn’t exist because objectivists would accept all products which measure better, not just those in certain categories.  

[Audiophile Neuroscience]

9 hours ago, pkane2001 said:

 

I generally agree, Jud. But quoting the actual performance of a device is not misleading. The measurement is a physical property of the device, unless the measurement itself is badly mangled.

 

Saying that 1fs of jitter is much better than, say, 100fs of jitter would be misleading (using a very simplistic example). It certainly could be true in an engineering sense, but not significant to an average consumer. Might be useful to me, as I often use DACs and ADCs for measurement. As long as the reviewer indicates that the difference between 1fs and 100fs jitter isn't audible, I don't see any hypocrisy in reporting the actual measurement.

Archimago, to use an already mentioned example, not only quotes measured jitter numbers, but also has a full blog article on the audibility of jitter. Nothing misleading in that, at least to me.

 

I totally agree that quoting the actual performance of a device is not misleading as as long as the reviewer offers an unbiased interpretation of what he thinks they mean. In my experience this does not always happen and indeed there appears at times to be a double standard in existence just as @The Computer Audiophile has described.Measurements used as a weapon to promote an agenda.

 

Without keeping a 'forensic' file on the matter nor wanting necessarily to play the name and shame game (the evidence you desire notwithstanding being important), sometimes being mislead is implied. Taking your example of the LPS/meanwell PSU, did the measurement indicating a glitch make any difference to audibility? Did you hear it? Did anyone on ASR hear it?

 

Now, yes there is another point. I totally get that if one seeks to discredit a manufacturer they will eagerly report a shortcoming in what the device is purported to do at a technical level. It is certainly fair enough to report any deficiency. If however, the "objectivist" view is that the device is irrelevant and unnecessary in the first place, any partial shortcoming should be ignored as inaudible, not matter. This is not the way this has gone down IMO. YMMV.

 

FWIW IMO Uptone's response in this matter in ASR (as well as AS) has been perfectly reasonable, honorable and shows a degree of openness and integrity which should be lauded.

 

[Audiophile Neuroscience]

13 minutes ago, pkane2001 said:

The point is that there can be no generalized correlation if individual’s preferences are involved. There will always be someone who will prefer something different.

No the point is the measurement does not correlate with what we hear, preferred or not. Paul, you can go around this loop all day but it won't change that fact.

[pkane2001]

19 minutes ago, Audiophile Neuroscience said:

No the point is the measurement does not correlate with what we hear, preferred or not. Paul, you can go around this loop all day but it won't change that fact.

Your loop. You were answering my reply to Mani, and my point was exactly as I made it. You can make your own point, which you appear to be doing in a loop, but that has nothing to do with what I stated.

[Audiophile Neuroscience]

5 minutes ago, pkane2001 said:

Your loop. You were answering my reply to Mani, and my point was exactly as I made it. You can make your own point, which you appear to be doing in a loop, but that has nothing to do with what I stated.

 

 

You're sounding a little loopy 🙄🤷‍♂️😁