Misleading Measurements

[Josh Mound]

I could be wrong, but I think Chris’s point is that if we take “beyond audibility“ seriously, then it becomes a checked box rather than a “moar please” issue. There’s no “even more inaudible.” I’ve said before that I don’t see the point of someplace like ASR continuing to measure DACs. According to its own standards, a bunch of $99 DACs are “perfect.” 

 

But I do think there’s a danger in reducing DACs to one or two numbers, dumping them into a graph, then saying that higher is better. That’s especially the case if it’s really “all DACs with distortion below 80 dBFS are audibly transparent.” Even more so the case when, as Marv at SBAF has demonstrated, little issues with kinked cords, ground loops, etc. can make the same unit measure differently.
 

This is not, of course, to say that measurements don’t have immense value. But I think the limits to measurement fundamentalism (for lack of a better way to make clear that I’m not lumping everyone who likes measurements together) are already being seen with ASR’s foray into speaker measurements. People are trying to reduce Amir’s speaker measurements to one or two preference numbers. Yet he’s saying in his subjective reviews that speakers with virtually identical preference ratings sound very different (and resorting to some of the biggest audiophile writing cliches in the process!). This is, understandably, vexing some of his most loyal followers. 

I believe Amir can hear differences between two 6.8 (or whatever) preference-rated speakers. But I also think there are some people who can, even in blind testing, hear differences between two “perfect” DACs. There are certainly some people who can’t hear either. There are even many people who couldn’t tell a 4.5 preference rating speaker from a 6.8 one, or a 60 SINAD DAC from a 110 one (or, if they could tell a difference, couldn’t say which is more neutral).
 

I’m interested in measurements to catch major issues and to understand why we’re hearing what we’re hearing. I’m not interested in them as an arms race with no meaningful purpose. 

[Josh Mound]

9 minutes ago, pkane2001 said:

This may be a good start, as a summary:

 

http://www.aes-media.org/sections/pnw/ppt/other/limitsofhearing.ppt

 

 

 

 

 

 

 

 

I followed the ASR attempt to establish agreed upon thresholds for a while, and it seems like it petered out. My sense of why that happened is that we can mean “audibility threshold” in two different senses: what humans physiologically can detect and what research has established listeners hear in audio reproduction. For example, the audibility threshold of distortion was stricter in the initial ASR threshold attempt than in Archimago’s recent experiment. So, over at ASR there seemed to be a mix of physiology, AES-type research, and random assertions used to support the “lenient” and “strict” thresholds. 
 

Some of the results, such as a “strict” linearity of 120 seemed far beyond realistic audibility IMHO. At the same time, I kind of think IMD is more audible than that ASR post suggested. However, it definitely seems like there’s a lack of good research on some topics.
 

Then, of course, we’d get into issues of age and hearing and trainer versus untrained listeners.

 

Ultimately, this uncertainty and complexity is why I think a lot of people default to the “moar is better” approach. But I think a “these DACs all clear the bar of flat FR, linearity to 90 dBfs etc., so I’m going to listen to them and see if I notice any differences and which I like better” is more advisable. 

[Josh Mound]

On 9/22/2020 at 3:01 PM, Blake said:

Speaking of the topic of this thread, I'm not sure if this has been posted about and discussed here on AS, but I found it to be an interesting read as I have always had some suspicions about ASR:

 

https://www.superbestaudiofriends.org/index.php?threads/audio-science-review-review.9827/

 

IMO this is a pretty devastating critique. 

[Josh Mound]

3 hours ago, pkane2001 said:

 

Depends on what people expect to find when they go to ASR and read Amir's reviews. I personally don't have any interest in his subjective opinions, relative SINAD rankings, or recommendations. These are just as fallible as anyone else's, and I have no way to verify that they are correct, regardless of how many times Amir claims he's better at hearing differences than anyone else :)

 

Measurements and teardowns, on the other hand, are extremely valuable and give me a lot more information that I normally wouldn't have for most equipment out there. To me, that's the value of Amir's reviews and why I read them. If you don't understand measurements and come to ASR for a subjective opinion or recommendation, then you are looking in the wrong place, IMHO.

I generally take the same view you do. I’m happy he’s measuring speakers, in particular. There’s a real paucity of good speaker measurements on the internet. 
 

However, I have doubts about some of his other measurements, based on critiques and contradictory measurements at SBAF and elsewhere. He also seems to subject unfavored brands to critical teardowns, but not favored ones. 

 

The main issue seems to be that many people at ASR take the measurements as gospel and think the SINAD chart conveys something important. They don’t seem to realize that getting good measurements is as much an art as a science, and that it’s easy to make mistakes. And that’s not even getting into the question of what’s audible, what’s not, and what other non-standard measurements might be valuable. (The SBAF post about the JBL speakers’ amp is crucial in that respect. If Amir couldn’t tell how “bad” that amp was, why are we worried about every point in SINAD on DACs, etc.?)
 

Amir also has encouraged people to take his subjective reviews as truth by claiming he can hear things other people can’t, despite the fact that he has serious hearing loss and did not score high enough on Harman’s listening software to qualify as a trainer listener by Herman’s standards. 
 

I have no problem if he wants to offer subjective reviews. But he can’t claim, as he does, that his subjective reviews are the product of enlightened hearing while other people’s are garbage. 

[Josh Mound]

46 minutes ago, pkane2001 said:

 

It's the case of caveat emptor:  do your own homework. Anyone who blindly buys into what is said on ASR or AS or SBAF, Facebook, Twitter, or whatever, is likely to be making a mistake. Do you really think the guys at SBAF have no favorites and are completely unbiased? Or reviews or recommendations posted here, on AS? ASR is just another web forum.

I do think Amir had a very unique set of biases and a very large ego. And, in my experience, AS, SBAF, GearSlutz, etc. all tend to allow a wider array of viewpoints than ASR does. 

[Josh Mound]

56 minutes ago, pkane2001 said:

 

So what viewpoint was banned by Amir on ASR?

Are you kidding? Threads get closed or individual users warned or banned for disagreeing with Amir all the time! Check out the discussions of MQA or Amir’s subjective ratings of speakers. Anyone who disagrees with Amir repeatedly gets shown the door, regardless of the topic. 
 

Amir’s been banned from a variety of forums for trolling, including What’s Best, which I believe he cofounded. ASR bills itself as an objectivist site, but it’s really an outlet for Amir’s personal views. But Amir doesn’t want to present his view as simply one view among many. Instead, it’s that he has golden ears and superior knowledge to everyone...and, by the way, did he mention that he once worked at Microsoft and helped design WMA? (Yikes.)

[Josh Mound]

55 minutes ago, The Computer Audiophile said:

No clue. Don’t read that site. 

Smart man. I’m a glutton for punishment. (And I do appreciate some of the measurements, if not the commentary that goes with them.)