Testing MQA: Is it worse than FLAC?

[GoldenOne]

TLDR: MQA isn't lossless, is arguably worse than normal flac, and is seemingly nothing more than a (quite effective) scheme to generate licensing fees. With the frustrating addition that if you are a Tidal user, even if you have no MQA dac, and use the "Hifi" streaming quality setting, MQA encoded/lossy files will still be served to you. And the only way to avoid that being to switch to Qobuz.
If you disagree with this post, or if someone from MQA/Meridian is reading this, it would be excellent if you could provide alternative evidence supporting MQA's claims. If they are true it'd be EXTREMELY simple to demonstrate/prove and so the current lack of any evidence other than marketing claims is concerning.

I figured that given how aggressively Tidal has been expanding their use/incorporation of MQA (with now many redbook files coming MQA encoded even if they are not able to be unfolded to hires), and there seems to be an awful lot of debate about whether or not MQA is good or lives up to the claims, and not much testing going on, (including lack of evidence from MQA themselves), I should try to remedy that.

I'd like to preface this by saying a few things:
- This is not a dig at any manufacturer that incorporates MQA. MQA has been very successful from a business/marketing standpoint, and so customers are demanding it. Therefore its understandable that manufacturers like PS Audio are going to add it to their products even though they openly say they do not like MQA. 
- If you feel MQA sounds better, that's totally fine. Lots of things sound good and objectively perform bad, many tube amps for example. This is not addressing what sounds good to YOUR ears, this is addressing the hostile business practices and unsubstantiated marketing claims of MQA. 
- Further testing will be done by performing some null tests with the final unfolded analog output of an MQA dac soon, and i'll post here once that is completed.

- I am not writing this because I have an issue with MQA existing, I am writing it because MQA's business practices are detrimental to anyone who is not using their product (as we can no longer access native hires content), and arguably to those using their product too (because evidence shows MQA is likely inferior to native FLAC at any sample rate). MQA is POTENTIALLY a solution to a very niche problem (the need for HiRes audio when data is limited), but is being forced upon everyone in the market.

This post is intended to answer test and answer a few questions about MQA, namely:

1) Are MQA releases the same master as non-MQA?

2) If you don't have an MQA dac, is standard FLAC and MQA-FLAC the same / does MQA provide a benefit even on a normal dac?

3) Is unfolded MQA lossless or as good as native HiRes?

 

This is normally quite tricky to test because MQA ensures that there are no native HiRes releases for tracks that are released in MQA on tidal. So you cannot directly compare them. This is actually my biggest issue with MQA. If it were simply a choice of whether one wanted to use MQA or native HiRes, this discussion wouldn't be needed. But MQA seems to prevent the native HiRes release of tracks elsewhere, and in some cases some HiRes tracks have even vanished from other platforms such as Qobuz once the MQA Tidal version comes out.
However, there are a couple which seem to have slipped by.

Absofacto's "Thousand Peaces" for example has ONE of the songs in 96khz on qobuz (the rest are 44.1) and 88.2khz via MQA on tidal. (The fact that the MQA version isn't even the correct base sample rate is concerning in itself) I initially tested this, however it turned out that the Qobuz redbook and tidal redbook versions were different, meaning they are using different masters and could not be directly compared.
 

Answer 1: MQA/Masters SOMETIMES uses a different master source. Meaning the file formats themselves cannot be compared as the information/recording itself is different. This is likely done to give the impression of sounding better even though it's nothing to do with the file format.
 

So then, we need a different test track/album. Sam Smith's "The thrill of it all" however was ideal. It has a native 24 bit 88.2khz version on qobuz as well as the standard 44.1khz release. And on tidal there is also a 44.1khz release and can be 'unfolded' to 88.2khz via MQA. Meaning we can compare identical sample rates.

The first thing to do was to check whether the Tidal and Qobuz redbook/non-MQA files were actually the same. ie: Are tidal and qobuz using the same master for the song. To do this I downloaded the Redbook 16 bit 44.1khz version from Qobuz, and then the same from the release on tidal that was not marked "Master".

Deltawave showed that these two files were 100% absolutely bit for bit identical. So we can conclude that Tidal and Qobuz are using the same master for the song. Perfect.

 

Next, I downloaded the "Master"/MQA release, but without any MQA unfolding. ie: keeping it as a non-MQA dac owner would be playing it. Both these files are 44.1khz, but are not the same. In fact they are only 0.43% bitperfect with a 40dB null (24 bit accuracy is 146dB) We can see that the master is clearly the same as the majority of the track is identical, but the MQA version has a significant amount of high frequency noise compared to the lossless FLAC.


 

(Y axis is frequency, X axis is time. Green means that part is the same, purple/red means it is higher or lower in level and different from the original).
Just as a reminder, this is a direct digital rip, not an analog recording, so these differences are NOT due to variation in my recording or the DAC(s).
 

Answer 2: If you do not have an MQA DAC, MQA should be avoided, the content is NOT the same as the lossless original, and has more high frequency noise.

Additionally, MQA releases in many instances seem to be several dB louder than the redbook version. Again likely done to give the illusion of sounding better.

So then, now we need to see what happens if you unfold the MQA version to 88.2khz and compare it with the native 88.2khz version. I did this by using Roon, which has MQA decoding support, and recording bitperfect output, then comparing against the native hires 88.2khz version from qobuz.

Now things are really quite messy. The unfolded version differs significantly from the native hires, with again a lot more high frequency noise, as well as a band from about 11.5khz to 13.5khz where content differs a concerning amount in this specific instance.


 

Therefore

Answer 3: No, MQA is NOT lossless (a claim which MQA has recently removed from their marketing material), and even when unfolded does not match native HiRes content. I would love to test a full decoder/renderer, but MQA does not allow any "Full Unfolding" device to have a digital output, meaning we can never test a fully unfolded file other than by recording analog output of a DAC (which will never be perfect), and thus we will never be able to check if it is indeed identical to the original.
 

Additional arguments:
 

• MQA is actually probably worse than native playback. MQA makes it basically impossible to obtain a "normal" and MQA version of the same hires file. BUT, Stereophile did manage to convince them to send an MQA encoded single-impulse file. Their testing showed three things:
  1 - Playing back an MQA encoded file on a non-MQA dac caused issues, and created an asymmetric impulse response.
  2 - Playing it back on an MQA capable dac, it was minimum phase, not linear.
  3 - Playing back a NORMAL, non-MQA encoded impulse response file, with the MQA filter turned on on the DAC, produced an IDENTICAL result to the MQA file, suggesting that MQA is nothing more than a basic minimum-phase upsampling filter in this situation, and absolutely nothing to do with the source file. https://www.stereophile.com/content/mqa-tested-part-1
  There is significant evidence from multiple third party sources to show that MQA has all sorts of problems. http://archimago.blogspot.com/2018/02/musingsmeasurements-on-blurring-and-why.html
  

 

• MQA incurs an additional cost to you. You are paying for the licensing fees that are tacked on to products to get MQA support, and at every other step in the process. A good post from the manufacturer Linn is available here: https://web.archive.org/web/20201111211105/https://www.linn.co.uk/blog/mqa-is-bad-for-music Given as we have now demonstrated that MQA is NOT a substitute for native HiRes content, its hard to argue that MQA is doing much more than charging you for a sub-par version of something you already had (native hires music). If you want the best quality, demand native hires releases, not licensed, closed-source, proprietary compression. Schiit audio has also spoken on it: https://www.schiit.com/news/news/why-we-wont-be-supporting-mqa

 

• MQA IS NOT sourced from a HiRes master. Even if you are happy with it not being lossless, it is not actually even compressed from a HiRes source. Neil Young removed his music from tidal when after providing 44.1khz masters, Tidal suddenly released MQA versions, which would have been created simply by altering/upsampling the original. He did NOT provide them with HiRes masters to release in MQA, and you can read about this here: https://neilyoungarchives.com/news/1/article?id=Tidal-Misleading-Listeners
  
  "Tidal's master is a degredation of the original to make it fit in a box that collects royalties. That money ultimately is paid by listeners, I am not behind it. I am out of there. Gone. My masters are the original."
  MQA is at least in some situations simply an upsampled version with a licensing fee slapped on.....
   

 

• There is ZERO proof of any of MQA's claims. There is absolutely zero evidence to support any of their marketing, claims that they can fit 24 bit 192khz content into a 16 bit 44.1khz file, and in fact, all objective evidence and testing so far (including this post) conclude that MQA's claims don't make sense at all. The claims they make would be VERY easy to demonstrate and prove if they were true....
  Most MQA content cannot be obtained in native HiRes anywhere. And they do not allow any "full unfolding" device to have a digital output to prevent anyone from recording or testing the result.

Plenty of manufacturers have spoken out about MQA as well, some more aggressively than others.
- Linn: https://web.archive.org/web/20201111211105/https://www.linn.co.uk/blog/mqa-is-bad-for-music
- Schiit: https://www.schiit.com/news/news/why-we-wont-be-supporting-mqa
- PS Audio: https://www.youtube.com/watch?v=lPfmWKjiccA

Thanks for reading, hopefully this helped some people!

 

[_dbw_]

I just signed up so I could comment on this. 

 

Thanks for taking the time to write up your theory and the tests undertaken. Interesting, insightful read. I recall the first time I heard about MQA and I spent hours reading their literature and other people analysis, what a farce yet genius way the extract money from the punters. 

 

I look forward to future updates when your time permits. Thanks for sharing this. Best wishes, Dan

[saturdayboy]

I think your ignoring the question that might be most important to some:

 

Does an MQA file played through a dac with full MQA capabilities sound better than the 16/44.1, 24/96, 24/192, etc.. alternatives?

 

Who cares if it isn’t truly lossless if it actually sounds a little better?

 

I’m starting to see posts that claim MQA cd’s sound better than SACD’s. It certainly is interesting.

 

[firedog]

11 minutes ago, saturdayboy said:

I’m starting to see posts that claim MQA cd’s sound better than SACD’s. It certainly is interesting.

???? Posts? I'm starting to see posts that the world is flat and that Covid doesn't exist. 

There are also posts that claim "hi-res"  MQA sounds worse than CD. 

You might like the alterations MQA introduces. It happens. Possible that it's euphonic distortion for some. That means nothing other than that you like it - or that you are subject to expectation bias. 

[saturdayboy]

I wonder, does expectation work both ways? i.e. those who think MQA is fake news will think it sounds bad?

 

Don’t have any MQA gear, cd’s, or a Tidal subscription personally, but I have a hard time dismissing everyone who says they like how it sounds as flat earthers just because someone analyzed some wave forms and made some observations about them.

[saturdayboy]

This actually involved some listening, which I know is discouraged when MQA bashing is involved.

 

https://audiophilereview.com/cd-dac-digital/a-comparison-of-sacd-vs-mqa-in-physical-format/

[GoldenOne]

1 hour ago, saturdayboy said:

I think your ignoring the question that might be most important to some:

 

Does an MQA file played through a dac with full MQA capabilities sound better than the 16/44.1, 24/96, 24/192, etc.. alternatives?

 

Who cares if it isn’t truly lossless if it actually sounds a little better?

 

I’m starting to see posts that claim MQA cd’s sound better than SACD’s. It certainly is interesting.

 

Unfortunately its a question that can't be properly addressed.
Though I did address in the post WHY it can't be addressed.

MQA does not allow any device with 'full unfolding' capabilities to output a digital stream. It is ONLY allowed for analog only output devices. 
This means that we can never get a proper test of the native file vs fully unfolded MQA.
This is also assuming that you can find an MQA track that has a 192khz native hires release.

We can ONLY test by recording and comparing the results of analog output.
I will be doing some null tests recording from the analog output of a fully MQA capable DAC using the ADI-2 Pro ADC (performance info for which can be seen here: http://archimago.blogspot.com/2020/09/measurements-rme-adi-2-pro-fs-r-black.html ) 
This will not be able to tell us if the fully unfolded file is 'bit perfect' as it is recording from the analog output. But it will be able to show if there are any obvious discrepancies between native and MQA.

I'll additionally compare the audible band only content of the fully unfolded MQA file to the native FLAC audible band content to see if there are differences.

Its not the most ideal test, but unfortunately MQA has deliberately made it impossible to do an ideal test.

In regards to MQA being better than SACD, its an apples to oranges comparison.
SACD is 1 bit and so the quality of playback will be much more dependent on the source file, how it was produced and modulated down to 1 bit etc. (Much in the same way as you can use HQPlayer DSD modulators to get some pretty fantastic sound out of otherwise mediocre DACs by effectively 'replacing' the delta sigma modulator)

Whereas MQA is still PCM, and so will be more dependent on which DAC you're using.

Subjectively, to me, MQA sounds similar to minimum phase upsampling. And other third party testing suggests that it's quite likely a lot of MQA content IS just minimum phase upsampling. 
What's the best subjectively will always be up for debate unfortunately. So I won't get into that other than being open about the fact that I don't like it though mostly because of the attached business practices, if it were a choice it would actually be nice to have for certain situations where data or storage is limited. But for anyone at home with a good internet connection or cheap hard drive to put their music on, MQA is a solution to a problem that doesn't exist, being forced upon you and something good (native hires audio) getting taken away in the process.

[saturdayboy]

Really thoughtful post, although I don’t agree that it’s unfortunate that people will always debate what is subjectively best. We do that with almost everything, why should it be any different with digital file formats/conversion?

[firedog]

7 hours ago, saturdayboy said:

This actually involved some listening, which I know is discouraged when MQA bashing is involved.

 

https://audiophilereview.com/cd-dac-digital/a-comparison-of-sacd-vs-mqa-in-physical-format/

It's a  bunch of blah blah which has zero meaning. He didn't even mention if he knows the MQA and SACD were derived from the same master. Good chance they aren't. If so, you are comparing masters at least as much as formats. 

You aren't a veteran here Lots of us have listened to MQA and even blind tested it. Not sure your attitude is appropriate.

[firedog]

7 hours ago, saturdayboy said:

I wonder, does expectation work both ways? i.e. those who think MQA is fake news will think it sounds bad?

 

Don’t have any MQA gear, cd’s, or a Tidal subscription personally, but I have a hard time dismissing everyone who says they like how it sounds as flat earthers just because someone analyzed some wave forms and made some observations about them.

Expectation works both ways. Caveat: it a subconscious process. By definition, you don't know what your expectation bias is. Saying "I expected it to sound better/worse" (take your pick) means nothing.  That's why we have blind testing.

[fiske]

I assume the closest testbench is the one from 2L.
http://www.2l.no/hires/index.html?

 

I'm not into MQA and have no intention of checking it out.

[firedog]

2 hours ago, fiske said:

I assume the closest testbench is the one from 2L.
http://www.2l.no/hires/index.html?

 

I'm not into MQA and have no intention of checking it out.

Not necessarily. Those 2L files were appparently made using an MQA "white glove" treatment - that's nothing like the batch MQA processing that is used for 99.9% of the cases. 

[saturdayboy]

5 hours ago, firedog said:

Expectation works both ways. Caveat: it a subconscious process. By definition, you don't know what your expectation bias is. Saying "I expected it to sound better/worse" (take your pick) means nothing.  That's why we have blind testing.

The expectation can be conscious or subconscious. Blind testing eliminates the conscious elements. 

[firedog]

2 minutes ago, saturdayboy said:

The expectation can be conscious or subconscious. Blind testing eliminates the conscious elements. 

Nope.  Look it up. Conscious expectations and subconscious expectations are different. They can also be shown by testing to be opposite; that is you think your bias is one way, but it's shown to be another. Blind testing, done properly, eliminates bias of both types. That's the point. 

[saturdayboy]

16 minutes ago, firedog said:

Nope.  Look it up. Conscious expectations and subconscious expectations are different. They can also be shown by testing to be opposite; that is you think your bias is one way, but it's shown to be another. Blind testing, done properly, eliminates bias of both types. That's the point. 

From the wiki

 

“It may include conscious or unconscious influences on subject behavior including creation of demand characteristics that influence subjects, and altered or selective recording of experimental results themselves.^[2]”

 

 

[saturdayboy]

5 hours ago, firedog said:

It's a  bunch of blah blah which has zero meaning. He didn't even mention if he knows the MQA and SACD were derived from the same master. Good chance they aren't. If so, you are comparing masters at least as much as formats. 

You aren't a veteran here Lots of us have listened to MQA and even blind tested it. Not sure your attitude is appropriate.

Ok. What about people who are of the opinion that the MQA versions streaming on Tidal are better than the non MQA? I’m pretty sure that more often than not those are the same masters.

The point is are all those people who think it sounds better wrong, and you’re right?

Is personal taste irrelevant in listening?
 

How many times have you challenged someone to perform double blind testing when they tell that one pizza joint is better than another, or they prefer the smell of one rose over another?

[botrytis]

52 minutes ago, saturdayboy said:

The expectation can be conscious or subconscious. Blind testing eliminates the conscious elements. 

 

Sorry not always. Conscious is what MQA does when they demo their files. What I mean is, they actually tell you what to expect to hear compared to a non-MQA CD and that is conscious bias. Your brain then fills in what the MQA peeps tell you. 

 

Unconscious bias is based on a person's likes and dislikes and this happens in the background w/o the person knowing about it.

 

Double blind tests, which are random, basically do away with both. 

[firedog]

1 hour ago, saturdayboy said:

Ok. What about people who are of the opinion that the MQA versions streaming on Tidal are better than the non MQA? I’m pretty sure that more often than not those are the same masters.

The point is are all those people who think it sounds better wrong, and you’re right?

Is personal taste irrelevant in listening?
 

How many times have you challenged someone to perform double blind testing when they tell that one pizza joint is better than another, or they prefer the smell of one rose over another?

You have no idea which master is which on Tidal. So same problem.

It's not a question of being wrong or right. It's saying "I hear this in sighted conditions, therefore MQA is shown to be 'better'". 

 

Your final question is irrelevant and has nothing to do with the issue. I haven't said people can't like what they like. Personal taste exists.

 

But that's not the situation with MQA. Saying it as a format sounds better than: CD, SACD, hi-res, etc is a different kind of statement. If you are going to make that kind of statement you need to do a proper comparison. Otherwise it's a comparison and conclusion based on false premises. 

 

BTW, MQA has gone to great lengths to make sure that proper comparisons don't happen. Such as refusing Mark Waldrep's request that they use his own masters to make an MQA'd version so he could directly compare and analyze them. His is just one example.

 

The only properly blinded test of MQA we know of (McGill) didn't show any listener preference for MQA. It also wasn't a full blown test comparison, but it's the best we've got so far. 

[firedog]

1 hour ago, saturdayboy said:

From the wiki

 

“It may include conscious or unconscious influences on subject behavior including creation of demand characteristics that influence subjects, and altered or selective recording of experimental results themselves.^[2]”

 

 

Again wrong. You looked up only one kind of bias. There are multiple types. I suggest you give up this part of the argument as you are only repeatedly showing you don't understand the basics. 

[saturdayboy]

20 minutes ago, firedog said:

Again wrong. You looked up only one kind of bias. There are multiple types. I suggest you give up this part of the argument as you are only repeatedly showing you don't understand the basics. 

What I looked up was “expectation bias”, which I thought was what was being discussed. If you search “expectation bias” on Wikipedia you’ll find my quote was taken from that page.

 

Are you an observational psychologist? You seem to claim expert level knowledge in all things relating to perception and bias.

[firedog]

26 minutes ago, saturdayboy said:

What I looked up was “expectation bias”, which I thought was what was being discussed. If you search “expectation bias” on Wikipedia you’ll find my quote was taken from that page.

 

Are you an observational psychologist? You seem to claim expert level knowledge in all things relating to perception and bias.

And you got an article about observer-expectancy bias, which isn't really what we are talking about in this setting. I didn't claim to be an expert. Just more expert than you. Quoting not particularly relevant passages from Wikipedia doesn't do a lot for your argument. I suggest you talk about something audio related instead. 

[saturdayboy]

1 hour ago, firedog said:

You have no idea which master is which on Tidal. So same problem.

It's not a question of being wrong or right. It's saying "I hear this in sighted conditions, therefore MQA is shown to be 'better'". 

 

Your final question is irrelevant and has nothing to do with the issue. I haven't said people can't like what they like. Personal taste exists.

 

But that's not the situation with MQA. Saying it as a format sounds better than: CD, SACD, hi-res, etc is a different kind of statement. If you are going to make that kind of statement you need to do a proper comparison. Otherwise it's a comparison and conclusion based on false premises. 

 

BTW, MQA has gone to great lengths to make sure that proper comparisons don't happen. Such as refusing Mark Waldrep's request that they use his own masters to make an MQA'd version so he could directly compare and analyze them. His is just one example.

 

The only properly blinded test of MQA we know of (McGill) didn't show any listener preference for MQA. It also wasn't a full blown test comparison, but it's the best we've got so far. 

The McGill study compared 16/44.1 MQA files to their uncompressed 24/96 counterparts, who really expected MQA to win that battle?

 

Isn’t 16/44.1 MQA vs regular 16/44.1 the actual fair fight? 

[botrytis]

2 hours ago, saturdayboy said:

Ok. What about people who are of the opinion that the MQA versions streaming on Tidal are better than the non MQA? I’m pretty sure that more often than not those are the same masters.

The point is are all those people who think it sounds better wrong, and you’re right?

Is personal taste irrelevant in listening?
 

How many times have you challenged someone to perform double blind testing when they tell that one pizza joint is better than another, or they prefer the smell of one rose over another?

 

People also use Shun Mook discs to improve the sound of their audio system, even though proof that the discs work is non-existent. I guess it is all in their brain, that there is an improvement.

 

If they like MQA, fine. I have no issues with people saying they like it. I do have an issue with the idea, to pay more for a lossy file with a questionable master (as we don't know what master they use for encoding). The other issue is the fact the the audiophile press has done NO TESTING on MQA, to prove what they are saying about this format. They test everything else, why take BS/MQA's word for it?

[firedog]

15 minutes ago, saturdayboy said:

The McGill study compared 16/44.1 MQA files to their uncompressed 24/96 counterparts

 

 

Nope.

[saturdayboy]

7 minutes ago, firedog said:

Nope.

https://www.aes.org/e-lib/browse.cfm?elib=19396