Audio Blind Testing

[knickerhawk]

I’ve recently conducted a personal “blind” a/b test of MQA vs. CD on Tidal. Here’s how I did it. On my iPad BlueSound app, I opened Tidal and looked up the CD and MQA versions of an album. I then added the same track from each version to the play queue and then kept adding the same track from each version until I had a 20 track playing queue consisting of the alternating versions.  By clicking on the album cover in the playing queue, the display changes to a page that shows the queue scrolling horizontally across the bottom of the page. What you’ll see when you do the queue sequencing like this is just a horizontal list of identical thumbnail images of the album cover. You can’t tell from the thumbnails which individual track is the CD version and which is the MQA version. The only way to visually tell the difference is to actually play one of the tracks. A little circle appears elsewhere on the page with either “CD” or the MQA symbol. This can be easily covered by a Post It or something else.

 

To start the “randomization” I just randomly scrolled through the playing queue at the bottom to approximately the midway point in the queue and selected one of the identical thumbnails. As long as I avoided scrolling to a starting point that showed either the first or last item in the queue it was impossible to tell which version had been selected. After listening to the first randomly selected track, I then listened to it’s “neighbor” (which because of the a/b sequencing of the queue I knew was from the other format). I went back and forth several times until I was ready to pull the Post-It off and see which track was which format. Worked like a charm for allowing me to do a “blinded” personal a/b comparison of CD and MQA versions in Tidal!

 

In case anyone is interested, I tried this on four different tracks, two of which I had previously listened to “sighted” and two of which I had never before listened to on Tidal in either format. I was able to identify which was which 100% of the time. In the two previously “sighted” tests, I did not even need to do the “B” test to correctly identify which format the “A” test came from. In the two unsighted tests, I needed to repeat the A/B test twice on one of the tracks and three times on the other track before I was sufficiently confident in my decision. Not statistically signifcant for you? You’ll get no argument from me on that. However, that was not my personal goal in doing this little experiment. I simply wanted to challenge the strong sense I had developed over the past month from extensive “sighted” comparisons that I could consistently recognize and distinguish between the CD and MQA versions of recordings. The experiment satisfied me that I’m, indeed, really hearing the difference and not just submitting to confirmation bias in my general preference for MQA recordings. (Apologies for introducing a very controversial topic in the middle of another very controversial topic!)

[knickerhawk]

6 hours ago, botrytis said:

 

First, that is not blind test. the reason being you are comparing high res vs CD quality. Second, you do not know if the MQA and CD files are from the same master, so this is a HUGE problem with this test. Before you do any test likse this please read on how to actually perform a blind test. You have done at least 4 blunders that basically negates your conclusion.

Perhaps you should take your own advice and read up on what (who, actually) is blinded in a blind experiment. It is the test participant. You seem to be laboring under a false impression that a blind experiment is not blind unless the items being tested are equalized to some common standard. Not so. While such equalization might be important for purposes of isolating perceivable differences and drawing conclusions about the sources of any such perceivable differences, it is not related to whether the test itself is blinded. 

 

In my test I was, indeed, blinded. The personal purpose for conducting the test was confirmation of ANY perceivable differences, REGARDLESS of the cause of such differences. Put another way, the purpose of the test was to eliminate bias as a factor in my differentiations between CD versions and MQA versions of tracks played back on Tidal via my BlueSound front end and DAC. I do not make any claims beyond that (and beyond the assertion that I am capable of reliably distinguishing between CD versions and MQA versions on my own system using Tidal based content). I leave it to others and other threads to hash out the underlying reasons for those audible differences. I mentioned my experiment here because it relates to the subject of the thread - namely, audio BLIND testing.

[knickerhawk]

12 hours ago, esldude said:

Were levels precisely the same?  Question #1.  Deciding this by ear is nowhere near good enough. 

 

Given your test conditions it will be difficult to determine this. 

 

I understand your limited claims, but if all you found were louder files sound better than quieter files then this is not much of a useful finding in any regards to MQA.

 

The next question already asked is about the mastering being the same.  Same as above, finding two masters sound different doesn't tell us much about MQA.  

 

So if you can somehow confirm an MQA and non-MQA track are same master, and levels match you'll be onto something. 

 

Without at least those two conditions being met, two files sounded different for reasons unknown.  

Regarding sound levels being the same, the answer is “Yes” per my iPad spectrum analyzer app. If you expect anything more than that, then I can’t accommodate and, frankly, don’t care that much one way or the other. My testing is ultimately for my own edification and confirmation of what I’m hearing (and almost always preferring). In addition to these more “controlled” tests, the fairly serious listening I’ve done over the past month isn’t resulting in a sense of differences in sound levels but, rather, in detail and sweetness of the sound.

 

Regarding mastering sources, your guess is as good as mine, but again my interest is in the results and what sounds better to me personally. In virtually all cases, I’m preferring the MQA version (when I can make out any difference, which is most of the time I listen carefully and especially when doing rapid A/B comparisons). If that’s due to “better” mastering rather than the MQA magic, then so be it. I’ll happily accept the improvement unless/until Tidal makes available for my listening enjoyment remastered CD versions that are just as good/better than the MQA versions. 

[knickerhawk]

51 minutes ago, esldude said:

Nope, ipad spectrum analyzer won't cut it.  If you don't care, I don't either. 

 

I'll reiterate ," Without at least those two conditions being met, two files sounded different for reasons unknown."

 

You'll, like many others feel edified.  And that is about all your methodology is worth.  If that is good enough, then fine.  Simply means not much of anything.   

Whatever...The volume difference in one step on my pre-amp’s volume control is far greater than what I’m detecting in the tests (or seeing on my iPad spectrum analyzer app) for the tracks/albums I’ve compared. And for purposes of my tests I’m using newly released recordings with identical track times and otherwise no obvious reasons to assume significant diffferences in the (re)mastering between the CD and MQA versions. Of course, that doesn’t guarantee they’re from the same source, but it avoids obvious differences like what’s going on in The Nightfly or Getz/Gilberto and other audiophile “classics” that have been re-released as MQAs. 

 

And it’s not simply that the “two files sound different for reasons unknown.” What I’ve learned is that there is a general “signature” to the MQA sound. I’ve learned that the signature I’m detecting is not attributable to subject bias (thanks to the blinded testing I’ve now conducted). Likewise, I’ve learned that my personal preference for that identifiable sound is not attributable to subject bias. That certainly doesn’t mean much for others’ preferences, other listening setups, etc., but it means quite a lot for my own personal listening enjoyment.