AES: High Resolution Audio Is Now Audible?

[Bromo33333]

Meta-studies are interesting and are normed for the effect of "hidden negative results" in most circumstances.

 

I'm glad that is is showing that people who know what to listen for can tell Hi Rez from regular more than 50% of the time.

[jabbr]

What do you guys think? Does this paper state that high resolution audio is now deemed audible by the AES?

A Meta-Analysis of High Resolution Audio Perceptual Evaluation

 

http://files.computeraudiophile.com/2016/0629/AES-Paper-18296.pdf

 

An interesting finding is that the percentage of people who can distinguish significantly increases with training.

 

People should understand that this paper is essentially a systematic review of the literature. People who value data and research should not be dismissive of the analysis.

[YashN]

What do you guys think? Does this paper state that high resolution audio is now deemed audible by the AES?

 

Within their research, yes:

 

Overall, there was a small but statistically significant ability to discriminate between standard quality audio (44.1 or 48 kHz, 16 bit) and high resolution audio (beyond standard quality). When subjects were trained, the ability to discriminate was far more significant.

[YashN]

Red herring. The point is that the study indicates hi-res is audible-can be differentiated from Redbook. For years we've been told that "science proves" that hi-res isn't audible. "highly audible" and "night and day" aren't a claim being made and aren't relevant to the discussion.

 

But artists and recording/mixing/mastering people do like what they hear with the hi-res formats and they rely a lot on their ear to earn a living.

 

The thing is: some people really can't hear the differences for a variety of reasons.

 

Others don't know better than to focus on frequency reproduction only.

 

Then there are those who are statists or laggards of some sort.

[esldude]

An interesting finding is that the percentage of people who can distinguish significantly increases with training.

 

People should understand that this paper is essentially a systematic review of the literature. People who value data and research should not be dismissive of the analysis.

 

 

Yes I thought the two most interesting things were the training (though not surprising) and the idea longer times between samples improved discrimination. Also longer samples, but the long interval between them was the most interesting. I also note the tests showing some of the most significant discrimination were done in the last few years and did not use dither which I find very puzzling. Puzzling in the sense of why would anyone do the test that way.

 

Meta-analysis has plenty of problems and I don't trust it in the medical field. Still were one to just look at the results it would appear to be better than 99% not chance that hires is audible in the whole group of studies they selected. Using just trained listener studies you have a result that almost meets criteria for finding the Higgs Boson at the LHC.

[mansr]

Yes I thought the two most interesting things were the training (though not surprising) and the idea longer times between samples improved discrimination. Also longer samples, but the long interval between them was the most interesting. I also note the tests showing some of the most significant discrimination were done in the last few years and did not use dither which I find very puzzling. Puzzling in the sense of why would anyone do the test that way.

 

Meta-analysis has plenty of problems and I don't trust it in the medical field. Still were one to just look at the results it would appear to be better than 99% not chance that hires is audible in the whole group of studies they selected. Using just trained listener studies you have a result that almost meets criteria for finding the Higgs Boson at the LHC.

 

The key is "studies they selected." A different selection could easily yield the opposite result. Refer also to the current debate about publication bias whereby studies with negative findings rarely see the light of day, at least in some fields.

 

Now I'll concede that there may in some cases be subtle improvements using sampling rates higher than 48 kHz. What would be truly remarkable is a study indicating audible differences from going beyond 96 kHz.

[esldude]

The key is "studies they selected." A different selection could easily yield the opposite result. Refer also to the current debate about publication bias whereby studies with negative findings rarely see the light of day, at least in some fields.

 

Now I'll concede that there may in some cases be subtle improvements using sampling rates higher than 48 kHz. What would be truly remarkable is a study indicating audible differences from going beyond 96 kHz.

 

Oh, yes, that is the thing about meta- analysis. You see they started with 80 contenders and chose 18. There are real nits to pick with every single one of them. Meyer Moran, while it seems to me it shows the general transparency of AD-DA conversion, as a high res vs normal res test we know some of the hires files weren't. So some unknown number of responses should be thrown out. Yet they included it. Three use non-dithered digital (why and it definitely could be a corrupting influence as all three are comparing 44/16 with higher rate 24 bit). One is tests of 500 hz pulse trains with high harmonic content with filters (why was it chosen though indirect and others dumped out of the list for the same reason?). Oohashi was used though it has been replicated as directly as possible by others and results were a null. Why choose one that you know fails replication? And drop the failed replication as it was in their original list of 80. One didn't falsify the null theory when doing 88.2 vs 44.1 files recorded concurrently on good gear, but did show differences detected when comparing resampled files. That shouts the resampling was audible. Yet all the results from that one were used. I was mistaken they used Kunchur in their chosen tests, but some of the others aren't much better. The earlier two they used were testing with analog filtering of the upper frequencies which doesn't seem quite direct enough for these purposes. So yes real issues with everyone of their 18 I think.

 

I'll give them benefit of the doubt, that they were trying to stack the deck, though I mindful of issues from those they chose. Would like to see some tests to determine if longer samples with rest periods between resulted in better discrimination. It hasn't in any other testing in the past I can recall.

[wgscott]

I review maybe 10 or 15 papers a year, and (usually) 18 grant proposals per year.

 

I found this paper far more impenetrable than anything I have read even remotely related to my field. I can't conclude anything.

 

The idea of doing a "meta-analysis" of an aggregate data set, however, I think is perfectly reasonable. Identification of statistical trends should become more robust, assuming it is done correctly.

 

Just because he is a member of the AES and published this in their journal, doesn't make it the official position of anyone, btw.

[firedog]

Now that's an example of over reach.

 

I guess you mean what you wrote was highly exaggerated....?

[esldude]

I review maybe 10 or 15 papers a year, and (usually) 18 grant proposals per year.

 

I found this paper far more impenetrable than anything I have read even remotely related to my field. I can't conclude anything.

 

The idea of doing a "meta-analysis" of an aggregate data set, however, I think is perfectly reasonable. Identification of statistical trends should become more robust, assuming it is done correctly.

 

Just because he is a member of the AES and published this in their journal, doesn't make it the official position of anyone, btw.

Would you not say these data sets are too different to reasonably aggregate? I get the benefits of aggregation.

[wgscott]

Would you not say these data sets are too different to reasonably aggregate? I get the benefits of aggregation.

 

That was what I suspected.

[The Computer Audiophile]

Now I'll concede that there may in some cases be subtle improvements using sampling rates higher than 48 kHz. What would be truly remarkable is a study indicating audible differences from going beyond 96 kHz.

Wait, did you just move the goalposts? :~)

[mansr]

Wait, did you just move the goalposts? :~)

 

No, I replaced the goalkeeper.

[The Computer Audiophile]

No, I replaced the goalkeeper.

Love it!

[audiventory]

What do you guys think? Does this paper state that high resolution audio is now deemed audible by the AES?

A Meta-Analysis of High Resolution Audio Perceptual Evaluation

 

http://files.computeraudiophile.com/2016/0629/AES-Paper-18296.pdf

 

First problem of such tests is technical problem of creating identical playback apparatus for different resolution.

 

What compared there: apparatus modes - hi-res vs. 44,48/16 or formats?

 

1. 44/16 -> apparatus in 44/16 mode -> check

 

2. DSD -> apparatus in DSD mode -> check

 

3. DVD -> apparatus in hi-res mode -> check

 

May be I missed something, but I don't understood how considered so important part as difference of apparatus?

 

I suppose, apparatus in hi-res PCM or DSD mode should work better with higher probability than in 16/44.

[esldude]

Psychology’s Meta-Analysis Problem | Absolutely Maybe

 

Hmmmmm, do any of the critiques seem to apply to this AES meta-analysis project?

[firedog]

Psychology’s Meta-Analysis Problem | Absolutely Maybe

 

Hmmmmm, do any of the critiques seem to apply to this AES meta-analysis project?

 

Some of them might, but the gist of the article seems to relate to the fact that the populations in psych studies are often very much representative only of WEIRDS (Western, educated, etc) and as such don't reflect the human population as a whole.

 

I'd venture that this is much less of a problem in "audiophile" studies, b/c of the nature of the beast. Nevertheless, the article is pointing out that meta-studies aren't of much use if the individual studies that are used to make them aren't themselves evaluated for the usefulness of the data they produce.

[Fitzcaraldo215]

I think the Reiss paper is a good one and a useful one. For one thing, it reviews many previous studies I was unaware of. I think it also highlights many of the pitfalls that have occurred in testing and which also contribute to many "difference" or "no difference" findings by audiophiles in their own subjective listening.

I think its demonstration of the significance of training listeners in what to listen for is also quite important. Let's face it. Many audiophiles, even allegedly quality conscious ones, just do not know what they are hearing. Not saying I do either. But, often differences in blind tests can be perceived only when the nature of those possible differences are pointed out in advance.

One important issue is whether or not the recordings used in comparisons truly have a hi rez provenance. Mark Waldrep has made a point of this. And, the infamous Meyer-Moran study, probably the most well known and most widely cited, trips up on this issue by using a fair number of remasterings from analog, among other considerable faults biasing it to a "no difference" conclusion. I suspect that quite a few audiophiles have listened to what they thought was "hi rez", but which was really just applying hi rez mastering to analog or RBCD masters. It is not totally surprising that people do not hear significant differences unless the comparison involves material natively recorded and mastered in hi rez. And, the hi rez provenance of many recordings, especially in pop genres, is not always accurately known.

I have done my own anecdotal, subjective, level matched comparisons comparing the RBCD layer vs. the stereo DSD layer of SACDs using classical music. I am quite sure the source for both was the hi rez master recording, but with down conversion for RBCD. I have concluded there is a small but consistently noticeable sonic improvement provided by hi rez. No, it is not the slam dunk, night/day difference marketers and many audiophiles claim. It is an improvement, not a breakthrough.

My opinion proves little as far as others are concerned, but I am happy with my assessment for my own listening purposes. I also put my money where my mouth is by buying native hi rez recordings (in Mch) almost exclusively. No, I generally do not rebuy material in hi rez if I already have the music on CD. No, I would not knowingly buy a hi rez download if it were merely uprezzed from an RBCD master, a reprehensible practice.

Incidentally, I am not a DSD bigot. I generally listen to DSD and other sources via conversion to PCM (at 88 or 176k sampling), if necessary, in order to apply DSP room correction. I still hear the advantage of hi rez, though. And, I am just as happy with many hi rez PCM recordings in sonic terms. I am not sold on the need for ultra hi rez, like PCM > 96k or DSD > 64fs or 32-bit depth PCM. There are sharply diminishing returns.

My good opinion of Reiss' paper is probably biased because it lends some credence to my own findings. Yes, we can nitpick this paper or any other paper to death with arguments, real, hypothetical or just imagined. But, any studies of human listening perception are going to involve noisy data just because people hear and listen differently. There may not ever be a paper that finally "proves" the perceptable advantage of hi rez beyond any doubt. Has there ever been a paper that "proves" vinyl is consistently better than CD or vice versa? No, that battle still rages, as I suspect the RBCD vs. hi rez battle will.

So, I recommend that people read and absorb this well written paper, ignoring some of the more esoteric statistical analysis as necessary. Then, listen for yourself, carefully though, avoiding common mistakes. Reach your own conclusions that are good enough for you. If others wish to disagree, fine, as long as they avoid telling everyone else they are right and all others are wrong.