Article: MQA: A Review of controversies, concerns, and cautions

[mcgillroy]

The difference between the stuff Stereophile put up on MQA and @Archimago‘s article is the difference between a Popular Mechanics and a peer-reviewed science article. Archimagos measurements have been checked by third parties and are open to replication. Stereophile is invited to do so or stay on their PM-track while the rest of the audiophile train rolls on.

 

Instead Mr. Atkinson argues about anonymity. Well that is how science works John, without blind peer-review none of the funny gadgets on your measuring-bench would exist.

 

Man up and have a go at @mansr‘s and Archimagos numbers. If you prove them wrong progress has been made. If you prove them right  progress has been made. You really cannot loose but the friendship of a British pal who put you in an increasingly tight spot.

[tmtomh]

34 minutes ago, The Computer Audiophile said:

One other note about not engaging or not refuting data from an "anonymous" person, this is one of the first things Bob mentioned to me when he called. I'm not saying that Bob is setting JA's agenda or giving him talking points, but I just don't see why JA is sticking to that argument so hard. 

 

I think this is very common. People who oversee or are part of an enterprise, whether it be MQA or Stereophile or whatever, have to stay focused on their main goal and not get distracted or drawn into situations where there is no upside for them.

 

Trying to put myself in their shoes, I don't see any upside to Stuart or Atkinson engaging with Archimago's data, or with CA in general (beyond Atkinson's self-appointed fact-checker role). Better for Stuart to say on message, and better for Atkinson to incorporate any valuable insights here into Stereophile's articles as he sees fit. Why elevate CA's stature if they don't feel it will help them, and why participate in someone else's format and narrative when you can instead tell  your story with your own narrative structure?

 

I of course agree with you Chris, and with @mcgillroy above, and with many others here, that it's a shame (or worse) that Stuart and Atkinson see Arch's info and CA's discussions as not having any upside for them to engage with. That tells you a lot about their agendas and attitudes, and in particular it speaks volumes about @John_Atkinson's level of professional and collegial respect for @The Computer Audiophile, unfortunately.

[miguelito]

39 minutes ago, The Computer Audiophile said:

One other note about not engaging or not refuting data from an "anonymous" person, this is one of the first things Bob mentioned to me when he called. I'm not saying that Bob is setting JA's agenda or giving him talking points, but I just don't see why JA is sticking to that argument so hard. 

The scientific method is anonymous. There should be no truth value assigned to a statement by a "known authority", but rather if anyone including Bob Stuart disagrees with an analysis, they can come here and refute it with facts.

 

I read with great care Bob Stuart's articles including your Q&A here. I have found all of these quite devoid of details, and including no information that hasn't been assessed here. So I find the argument to "not engage with anonymous critiques" empty and in fact elitist. 

 

I just don't see how MQA can pull this stunt off.

[Doug Schneider]

42 minutes ago, realhifi said:

He was commenting on something I wrote directly to you. It concerned equating a way of recording and music distribution to the Iraq war. 

 

 

Interesting. My apologies if anyone took it that way. But my point was/is that the media is there to do a job of asking some questions. That early-2000s issue was seen as a media blunder, at least in the US. I could link some YouTube videos discussing it, but they appear to take up the entire screen. If anyone took offence at the comparison, perhaps others less sensitive could've been used. But the point -- ask some questions, put people making claims on the spot to prove them. That's all.

Doug
SoundStage!

[Doug Schneider]

7 hours ago, realhifi said:

 

Please tell me you’re not equating a run up to a war to an advertising campaign for a music format. A war in which millions of lives were affected, countries were torn apart and our own military forces lost not only members but squandered billions of dollars in the pursuit of those “weapons” which were never found. 

 

Shame on you. 

 

I'm not equating them, though I apologize if you took offense. My point was that questions need to be asked, even if they put people on the spot.

Doug

[james45974]

57 minutes ago, tmtomh said:

 

I think this is very common. People who oversee or are part of an enterprise, whether it be MQA or Stereophile or whatever, have to stay focused on their main goal and not get distracted or drawn into situations where there is no upside for them.

Yes, but a no upside calculation for MQA or Stereophile is not necessarily neutral, there could be a downside, like to your reputation!

[tmtomh]

1 hour ago, james45974 said:

Yes, but a no upside calculation for MQA or Stereophile is not necessarily neutral, there could be a downside, like to your reputation!

 

No disagreement there - especially given how big CA's readership/membership base is.

[miguelito]

21 hours ago, John_Atkinson said:

 

With respect, yes, this is a core belief of mine, and has been since I worked in a research lab at the end of the 1960s. You have something relevant o say, hang it on the peg of your actual name. Just as I have done all my life - and even Doug Schneider has done! Until then, while I have read your article, and certainly have opinions on what you have written, I shall keep my thoughts to myself.

 

John Atkinson

Editor, Stereophile

Ok fine. The name is Stormy Daniels. Ok?

[adamdea]

 

2 hours ago, james45974 said:

Chris, I personally believe that John is reacting as such because that is the way it has always been in the print media world.  The last few years we have had this thing called the World Wide Web where there is more and faster interaction than you ever had in the print world.  I think some of John's behavior is that it is hard to teach an old dog new tricks!

Quite agree 

• The uncritical regurgitation of manufacturers’ claims is not confined to MQA: it is the staple of the first half of most equipment reviews. Only in the case of conspicuously outlandish claims (and the bar is set really high here) is an eyebrow raised. Whether this is our of laziness and space filling or conscious desire to provide advertorial, I don’t know. The result is the same.
• The desire for product differentiation and the need to justify “upgrades” requires manufacturers to provide “new” “features” all the time. Often these are not new at all- for example much was made for many years of “upsampling” as a feature even though it was technically indistinguishable from oversampling which was present in early CD players and most CD players and DACs ever since. 
• Because of new things get a good press, something technically sound like 16/44 pcm curiously got more accurate coverage early on. JA’s grasp of orthodox theory was impeccable back in the day. At some point he forgot that the ringing in a sinc function was necessary for the proper interpolation between the samples and discovered that it was something to do with time smear. 
• Unfortunately the universally friendly smile becomes at some point a gross disservice to the consumer.

[sullis02]

On 3/9/2018 at 6:26 PM, kissov said:

"Like I said in the past, there are other roles I have to play in this world (not in the audio domain of course) and it's good to keep boundaries between them."

 

I need to end this discussion, my apologies, I had no idea.

 

 

Your linking to Beekhuyzen  as a credible authority told us that already.

[sullis02]

On 3/10/2018 at 2:38 AM, firedog said:

The Meyer Moran study has been fairly thoroughly discredited. Even one of the authors said he no longer stands by the conclusions. 

Relly? Meyer, or Moran?  Can you supply a link?

 

 

 

Quote

One of the big problems with the study was that they didn't find out the provenance of SACDs they used, and several of them were produced from upsampled Redbook. I wouldn't exactly call that testing "rigour". 

 

Oh really, *who cares*?  This was blatant post-hoc goalpost-moving on the critics' parts, because until then, SACDs and DVDA had been routinely hyped and reviewed as sounding better than CD *regardless of their provenance*

 

They (and later-generation hi-rez formats) still are, on 'golden ear' forums, for that matter.   

 

 

Quote

 

 So their study wasn't comparing hi-res recordings to Redbook at all in those cases, it was comparing Redbook source to Redbook source. And somehow they got to the conclusion that there was no discernable difference between Redbook and hi-res.

 

Yes, and?  That's pretty much what physiology and psychoacoustics would predict.  

 

 

Quote

 

There were also some statistical issues with the study that put the findings in doubt. 

And as far as studies go, see this: http://www.aes.org/e-lib/browse.cfm?elib=18296

And that meta-analysis rejected the Meyer-Moran results for inclusion as being statistically suspect, i.e.,  results appearing to not be statistically random.

 

That meta-analysis itself is an...interesting...piece of work.  In any such analysis, what is included and excluded is crucual, for example. Hop over the Hydrogenaudio and look up the thread(s) on that study , for a dissection.

 

Not to mention, why should it take a *meta analysis* , for heaven's sake (a re-sifting of past results, mostly negative) to reveal a difference that is supposedly so *nonsubtle*  as CD vs hi-rez?  if the claims for hi rez are true,  DBTs should be a slam dunk. 

 

 

Quote

 

I'm not actually arguing the point of whether hi-res is audible - I'm just arguing that the Meyer - Moran study isn't where you should go if you want scientific proof it isn't. 

And I'd argue its flaws are overrated, in the context of what it was responding to, and that much of the critique is disingenuous if not outright hypocritical.

[Doug Schneider]

31 minutes ago, adamdea said:

Often these are not new at all- for example much was made for many years of “upsampling” as a feature even though it was technically indistinguishable from oversampling which was present in early CD players and most CD players and DACs ever since. 

 

Hi,

 

I've seen this up- and oversampling come up many times and I have a story about it -- pardon the pun, involving Mike Story.

 

Back in about 2000, I was at dCS,  as part of a group, when Mike Story was still there.  We asked about upsampling versus oversampling, since dCS was using the term at that time, and he didn't have much of a reply -- not one any of us understood, anyway. His definition might've made sense, but I'll be honest, eight of us were in the room and it flew right over our heads.

 

Years later, I recounted the story to the guys at Anagram Technologies and they were surprised, because, according to them, they had a very clear and easy definition. To them, oversampling was a change in clocking frequency -- 2X, 4X, 8X, etc. So, a strict change in the clock speed, that's all. On the other hand, to them, upsampling was the interpolation performed when the frequency increased, regardless of the frequency chosen.

 

Whether anyone else used that as their definition, I don't know -- chances are, most are using oversampling and upsampling interchangeably. But I've always thought about oversampling and upsampling in those terms since that was the only time I'd heard the difference be so clearly defined.

 

Doug
SoundStage! 

[sullis02]

On 3/10/2018 at 2:54 PM, John_Atkinson said:

 

Not correct, as you can see from reading my article on Listening to MQA:

https://www.stereophile.com/content/listening-mqa

"I scored four out of seven correct; though this is insufficient to prove formal identification, I feel that it is relevant information," adding in the comments that "it was the Steely Dan track that I got wrong—twice. Without it I would have scored 4 out of 5."

 

 

 

4/5: p= 0.188

 

A revolutionary improvement in audio, and it still doesn't pass a 95% significance threshold in a difference test, even when the ears belong to the editor of Stereophile.

 

Tsk. 

 

[sullis02]

On 3/10/2018 at 10:50 PM, Doug Schneider said:

Which brings up another thing related to MQA. I have no doubt that people ARE hearing differences -- and Charles had no doubt either. What's "better" versus was more "accurate" can be two different things. But when it comes to these slow-roll-off filters, which Charles preferred, too (his Listen versus Measure filters), what ARE people hearing? MQA would like to tell you it's timing accuracy, but that doesn't seem to be the case. Yet people are hearing something and this is where some interesting testing could/should be done.

 

Doug
SoundStage!

 

 

Really? The non-anecdotal evidence is rather sparse.  

[mrvco]

On 3/8/2018 at 3:56 PM, mitchco said:

Wrt what is audible or not,  I would love to see more of this type of audibility testing in the consumer audio industry:

 

 

The best I could do is around 12 bits of resolution before auditory masking became too much.  The experiment posted is repeatable if anyone would like to try, plus files can be downloaded and listened to.

 

When Archimago and I attended the Vancouver Audio Show to listen to MQA files, we had an expectation that we would be presented with some AB testing, so we could hear the difference, as the system was certainly resolving enough :-) However, there were no comparisons and instead listened to some gobbledegook from the MQA sales rep, then a few nice sounding recordings, but no AB comparisons.

 

Given @Archimago's article, we now know why there are no audibility tests.

 

I had a similar experience in the MQA room at CES a few years ago (2016 I believe).  No A/B comparisons and the Meridian system they were using to demo wasn't very impressive in the first place.  I got the impression that no one bothered with system setup or room tuning , someone just dropped the speakers in the corner by the couch and called it good.  It all felt rather smarmy, not a positive first impression of MQA (this coming from a former owner and fan of Meridian proper).

[miguelito]

On 3/8/2018 at 5:56 PM, mitchco said:

Wrt what is audible or not,  I would love to see more of this type of audibility testing in the consumer audio industry:

 

 

The best I could do is around 12 bits of resolution before auditory masking became too much.  The experiment posted is repeatable if anyone would like to try, plus files can be downloaded and listened to.

 

When Archimago and I attended the Vancouver Audio Show to listen to MQA files, we had an expectation that we would be presented with some AB testing, so we could hear the difference, as the system was certainly resolving enough :-) However, there were no comparisons and instead listened to some gobbledegook from the MQA sales rep, then a few nice sounding recordings, but no AB comparisons.

 

Given @Archimago's article, we now know why there are no audibility tests.

Actually, I DID get "A/B" played to me at Meridian in NYC around March of 2014 or so (maybe 2015).

 

The MQA versions were substantially better. As I've mentioned elsewhere, the most amazing case was a 24/192 recording and it's MQA version. However, after listening to a lot of MQA in the last couple of months, I realize it MUST have been an MQA version from a different mastering altogether. 

 

On top of it all, I got such gibberish answers to my questions that I was left with the impression of a poorly set up magic trick. Sad.

[sullis02]

On 3/11/2018 at 6:20 PM, Archimago said:

 

Whether Stuart is nefarious or not, I honestly don't care. I think the problem is that, in many ways like Neil Young before when they tried to sell us "high resolution" with Pono, this "cult of personalities" gets injected into the discussion unnecessarily. We can have celebrities endorse products, but to make it so overly attached to one individual is a disservice to the product and the hobbyists the product is advertised to. To make it so that this individual appears to be treated with reverence (notice the religious connotation) as to be more important than external validation is clearly wrong. I would not be surprised if Bob Stuart himself regrets being placed in the spotlight like so for the sake of his name and legacy.

 

Stuart has been flogging, and seeking ways to monetize, hi rez for two decades.   His "Coding Methods for High-Resolution Recording Systems," was presented at the 103rd AES Convention, New York, 1997, and by 2000 became the Meridian white paper "Coding High Quality Audio", which begat his March 2004 JAES paper "Coding for High-Resolution Audio Systems". ...the latter prompting skeptical comments  to JAES by Meyer, Moran, Roy Allison, and David Hardaway a few issues later.  Meyer and Moran's paper was a direct result of Stuart's ( at the time, virtually unsupported by published blind tests) claims.

 

There is lots of history here.  The AES for much of the past two decades was skeptical of the audible benefits of and 'need' for hi rez at the consumer playback end; but the 'political' tide in the organization  on that, seems to have changed in recent years.  There is likely a story there , too.  

 

 

 

[sullis02]

23 hours ago, John_Atkinson said:

 

With respect, yes, this is a core belief of mine, and has been since I worked in a research lab at the end of the 1960s.  

 

Research labs rely on test protocols that include controls.  Pity that core belief didn't stick with you.

[sullis02]

9 hours ago, The Computer Audiophile said:

I had no clue Sam wasn’t his name until a couple years ago. 

Ditto.  Hell, I didn't know until *this thread*.  And I've been sampling the risible scribblings in 'hi end' rags long enough to remember the debacle that ensued when Tellig told readers they could greatly improve the sound of their CDs by coating them with Armor-All. 

[John_Atkinson]

9 hours ago, Doug Schneider said:

It might not have sat well with you, but as editor, you're basically the boss. Since the 1990s, Stereophile has been sold a number of times. Each new publishing company instills a new set of rules. There were many opportunities to change that.

 

Thank you for the advice, Doug. As I said, I honored the agreement that had been made 2 years before I joined Stereophile. End of the story as far as I am concerned. YMMV.

 

John Atkinson

Editor, Stereophile

 

[John_Atkinson]

7 hours ago, wdw said:

I am concerned that we may just run John A. outta town if we continually post strong negatives about him and his magazine.

 

Don't worry about it. I have a professional thick skin.

 

John Atkinson

Editor, Stereophile

 

[wdw]

Just now, John_Atkinson said:

 

Don't worry about it. I have a professional thick skin.

 

John Atkinson

Editor, Stereophile

 

 

good to hear....Best, wdw

[adamdea]

1 hour ago, Doug Schneider said:

 

 

Years later, I recounted the story to the guys at Anagram Technologies and they were surprised, because, according to them, they had a very clear and easy definition. To them, oversampling was a change in clocking frequency -- 2X, 4X, 8X, etc. So, a strict change in the clock speed, that's all. On the other hand, to them, upsampling was the interpolation performed when the frequency increased, regardless of the frequency chosen.

 

Whether anyone else used that as their definition, I don't know -- chances are, most are using oversampling and upsampling interchangeably. But I've always thought about oversampling and upsampling in those terms since that was the only time I'd heard the difference be so clearly defined.

 

Doug
SoundStage! 

Interesting Doug 

The first 14 bit players were called 4 x oversampling I’m fairly sure. Does your first category mean changing the sample rate by an integer multiple?

The only really distinct meaning of oversampling  I am aware of is simply sampling by a higher rate than is needed to capture the information in the signal. 

[Doug Schneider]

32 minutes ago, John_Atkinson said:

 

Thank you for the advice, Doug. As I said, I honored the agreement that had been made 2 years before I joined Stereophile. End of the story as far as I am concerned. YMMV.

 

Oh, convenient that's over for you. But come on, this double-standard is getting precariously close to jumping the shark...

 

You had a guy with a pseudonym writing for the magazine for over 30 years -- while you were the boss.  Subjective, fly-by-the-seat-of-your-pants opinions some people took seriously. And that's OK (as someone pointed out here, he didn't know Sam Tellig wasn't Sam Tellig until today!).

 

"Archimago," on the other hand, writes a highly technical article with points in it that someone with the technical chops can refute, dispute, or confirm, yet you won't entertain the thought of addressing it. Right...

 

BTW, without too much work, I contacted Archiamago, which wasn't all that hard, and do understand better that he might not want to mix his professional and hobby lives. I'm not saying I'd do the same if I were in his shoes, but I don't know for sure that I wouldn't. What I do know is that he's not in the industry and certainly doesn't have an ax to grind. Perhaps reach out -- it's not that hard.

Doug
SoundStage!

 

 

[John_Atkinson]

6 minutes ago, Doug Schneider said:

Oh, convenient that's over for you. But come on, this double-standard is getting precariously close to jumping the shark...

 

Thank you for your comment, Doug.

 

6 minutes ago, Doug Schneider said:

BTW, without too much work, I contacted Archiamago, which wasn't all that hard...

 

I believe the correct pseudonym is Archimago.

 

John Atkinson

Editor, Stereophile