My Lying Ears

As a diehard card-carrying audiophile I am interested in all things related to this wonderful hobby. I've published articles based solely on my subjective listening experience and I've published articles detailing only objective measurements and facts about products. I enjoy publishing and reading articles that cover the gamut. I also think it's healthy and interesting to be open to perspectives completely incongruent with our own. With this in mind, I was recently sent a link to the JRiver forum to read a post about one person's perspective and experience as an inquisitive listener. I really liked what I read, in the sense that it's a real world story to which many people can probably relate and it was written in a non-confrontational way. In fact every audiophile I know, golden-eared or not, has at one time or another experienced something very similar to the follow story. I'm not pushing any agenda or endorsing a point of view by publishing this article. I simply think a worthwhile read for all who enjoy this hobby as much as I do.

Here is a a re-written, more complete version of the post, sent to me for publication by the author Michael.[PRBREAK][/PRBREAK]

Recently on the Jriver forums (Link) a forum regular was describing his experience at an audio shootout where three bit perfect players were compared. Jriver had not done particularly well in the tests (only receiving 4 out of 38 votes), and there was some discussion of why that might've been the case, given that all three players were (at least notionally) bit-perfect. There were some questions about the test methodology (you can see all the gory details in the linked thread), and some good discussion about how bit-perfect players might or might not conceivably sound different. Ultimately several forum members were of the opinion that the test was basically invalid, while others thought that surely, because so many people heard a difference that there must've been a real difference to be heard.

My own view on this issue is complex. I will confess that I have occasionally heard differences between bit-perfect players. But I don't believe that bit-perfect players actually sound different. That may sound like a paradox, so I'll follow it up with a second one: I don't trust my own ears to correctly detect those kinds of differences in audio. You might well ask “Why not?” Let me offer an embarrassing personal anecdote to explain my point of view about listening tests and the fallibility of the ear:

Several years ago I built a pair of home-made bi-amped speakers. They're each the size of a large washing machine and they took me the better part of a year to build (more than a month of Sundays). Because they were entirely home-made and I was trying to do an active crossover from scratch, even after they were structurally complete, they still required quite a bit of tweaking to get the crossovers dialed in and the EQ set. 

So I started by just dialing in the EQ that seemed to make sense based on the specifications of the drivers, and taking a couple of quick RTAs with pink noise. That sounded alright, and all of my friends (several of whom are musicians and/or “sound guys”) dutifully told me how great they sounded. There was just one hitch: I kept getting headaches whenever listening to the speakers, and the headaches would go away right after I turned them off. So I tried to solve the problem by tweaking some frequencies with EQ. After some tweaks, I'd think I'd made some progress (it sounded better!), and everyone who heard the changes thought the new EQ sounded better.

Eventually, I even started dutifully "blindly" A/Bing new EQ with the old EQ (I'd switch between them during playback without telling my guests what I was switching, which isn't really blind at all), and my guests would invariably swear the new EQ sounded better. And I kept going with this "tuning by ear" method, often reversing previous decisions, backing and forthing and adding more and more convoluted filters. 

The most embarrassing moment (and something of a turning point) was when I was A/Bing a filter, and a friend and I were convinced we were on to something really excellent. After ten minutes of this, we realized that the filter bank as a whole was disabled. I had been toggling the individual filter, but the bank of filters wasn't on, so it wasn't actually even affecting playback at all. And we had been very convinced we heard a difference. And the headaches never went away.

Eventually the headaches (and a growing skepticism) prompted me to stop screwing around and take some real log sweep measurements (at the suggestion of one my more empirically-minded friends). Once I did, I realized that there was apparently a huge (10+ dB) semi-ultrasonic resonant peak at 18.5KHz that I couldn't even actually hear. So I fixed it and verified the fix with measurements. And then my headaches went away. 

This prompted me to take an agonizing look at the rest of the measurements and noticed that my "tuning by ear" which I (and my friends) all felt was clearly superior had turned the frequency response into a staggering sawtooth. So I systematically removed the EQ that was pushing things away from "flat," and kept the EQ that contributed to flatness, and re-verified with measurements. The result sounded so different, and so much more natural that I was embarrassed to have wasted months messing around trying to use my "golden ears" to tune my speakers. And my wife (who had been encouraging, but politely non-committal about my EQ adventure) came home and asked unprompted if I had done something different with the speakers, and said they sounded much better. And she was right; they did. In a few afternoons, I had done more to move things forward than I had in months of paddling around. 


The point of this anecdote is not to try and prove to anyone that my measurement-derived EQ sounded better than my ear-derived EQ or that a flat frequency response will sound best: as it happens, I ultimately preferred a frequency slope that isn't perfectly flat, but I couldn't even get that far by ear. 

The point is that taking actual measurements had allowed me to:


1) Cure my ultrasonic frequency-induced headaches;


2) Improve the fidelity of my system (in the literal sense of audio fidelity as "faithfulness to the source"); and


3) Ultimately find the EQ curve that I liked best (which looked nothing like my ear-tuned curve).



My ears (and the inadvertently biased ears of my friends) did not allow me to do any of those things, and in fact led me far astray on issue 2). My ears couldn't even really get me to 3) because I kept reversing myself and getting tangled up in incremental changes. Most damning, my ears were not even reliably capable of detecting no change if I thought there was a change to be heard. 

Once I realized all this, it was still surprisingly hard to admit that I had been fooling myself, and that I was so easily fooled! So I have sympathy for other people who don't want to believe that their own ears may be unreliable, and I understand why folks get mad at any suggestion that their perception may be fallible. I've been accused by many indignant audiophiles of having a tin ear, and if I could only hear what they hear, then I'd be immediately persuaded. But my problem is not that I am unpersuaded: it's that I'm too easily persuaded! I'll concede, of course, that it's possible that I have tin ears and other people's ears are much more reliable than mine, but the literature concerning the placebo effect, expectation bias, and confirmation bias in scientific studies suggests that I'm probably not entirely alone. 

And I've seen the exact same phenomenon played out with other people (often very bright people with very good ears) enough times that I find it embarrassing to watch sighted listening tests of any kind because they are so rarely conducted in a way designed to produce any meaningful information and lead into dark serpentines of false information and conclusions. 





So to bring things back around: if some bit perfect audio players have devised a way to improve their sound they have presumably done so through careful testing, in which case they should be able to provide measurements (whether distortion measurements on an analog output, digital loopback measurements, measurements of the data stream going to the DAC, or something) that validates that claim. If they claim that their output "sounds better" but does not actually measure better using current standards of measurement, they should be able to at least articulate a hypothetical measurement that would show their superiority. If they claim that the advantage isn't measurable, or that you should "just trust your ears" than they are either fooling themselves or you.

In a well-established field of engineering in which a great deal of research and development has been done, and in which there is a mature, thriving commercial market, one generally does not stumble blindly into mysterious gains in performance. Once upon a time you could discover penicillin by accident, or build an automobile engine at home. But you do not get to the moon, cure cancer, or improve a modern car's fuel efficiency by inexplicable accident. In an era where cheap-o motherboard DACs have better SNR's than the best studio equipment from 30 years ago, you don't improve audio performance by inexplicable accident either. If someone has engineered a "better than bit perfect" player they should be able to prove it, as they likely did their own testing as part of the design process. If they can't rigorously explain why (or haven't measured their own product!), let them at least explain what they have done in a way that is susceptible of proof and repetition. Otherwise what they are selling is not penicillin, it's patent medicine. 

Bottom line: if you and a group of other people hear a difference, there may really be a difference, but there may not be too. Measurements are the easy way to find out if there is really a difference. Once you've actually established that there is a real, measurable difference, only then does it make sense to do a properly conducted listening test to determine if that difference is audible. Otherwise you're just eating random mold to find out if it will help your cough (or headache, as the case may be).

Or you can do what I do for the most part these days: just relax and enjoy the music.

- Michael