The walking dead and "blind listening"

[Blake]

On this issue, Tyll Hertsen's recent summary of "Big Sound 2015" sounds very sensible to me overall. It includes subsections on blind and sighted listening. Would be interested in responses:

 

Big Sound 2015 Wrap: What I Learned | InnerFidelity

 

Good post and link Christopher... I read that article as well as the prior posts from that experience and I was about to post it as well.

[mmerrill99]

I think what is missing from or misunderstood in A/B testing is that it is underpinned by the mistaken premise that our auditory perception must be able to act as an instant difference device in order to "prove" that we can hear some differences. This seems to be off the mark to a large extent & is a lot to do with the friction seen in thee discussions. The auditory processing system is primarily concerned with pattern recognition.

 

We know a bit about auditory perception & all the of it points towards the fact that we perceive audio, not as a continuous stream of frequency/amplitude changes but at a much more abstract, higher level of analysis. It appears that we split the impulses coming from the auditory nerve into a number of parallel streams - each stream has a particular, specialised focus. Our auditory perception is the analysis of these different streams, picking out from each what pertains to the auditory objects we are focussed on & combining them into a cogent auditory object. This has already been worked out in great detail for visual processing - M & P pathways. These arrive from different cells in the retina M cells give fast and transient responses, are very sensitive to low contrasts, and are color blind; P cells gives lower, more sustained responses, are relatively contrast insensitive, and can convey information about color. At higher processing levels in the visual pathway these may be recombined in various ways so that the information they convey can be recombined in precise ways according to the demands of a particular task.

 

As we discover more about auditory processing, it appears to operate in some similar ways in that there are two definite pathways (& maybe more) one specialised in spectral content & one in temporal content.

 

So, not nearly as straightforward & one-to-one a relationship between what we often measure (frequency/amplitude) that arrives at the ears & what we perceive via auditory processing. Like in vision, it involves the combination of a multitude of features that occur simultaneously and in temporal sequence. The unique combination of features in the spectral and temporal domain, therefore, characterizes a specific communication sound.

 

Bearing this in mind, when we compare two sounds we mostly compare higher level abstractions of these two sounds, not the low level frequency/amplitude differences expected in quick A/B tests. Furthermore, we are not comparing instantaneous differences - the differences are at a level of abstraction that is not restricted to the timeframe suggested is necessary in A/B testing (to overcome the echoic memory issue). For instance, it seems humans (& most mammals) are highly selective for frequency modulated sweeps - chirps/glides & time-variant frequency modulation.

 

Neuroimaging is the new technique for working out a lot of the unanswered questions about auditory processing & it is a far more reliable & useful technique than the stock auditory testing using perceptual impressions that rely on statistical significant number of trials, etc.

 

Let's hope we make some rapid progress on teasing out the mechanisms underlying auditory processing so that we can begin to better understand how & what we actually hear!

[Sal1950]

" Big Sound 2015 taught me that blind tests are really only useful for professionals evaluating small details, and really shouldn't be used by casual listeners or enthusiast n00bs."

 

He didn't just learn that you don't use a pool of Joe Blows to evaluate high end gear did he?

 

"They would often laugh and say, "No way! If I can't tell the differences in amps with the HE-1000, I'm certainly not going to be able to test the differences between cables.") It is very important for entusiasts not to over-enthusiastically claim huge differences in these areas—it makes objectivists crazy...and rightly so."

 

Yep, because in large numbers of the audible claims they are either imagined or made up for profit motives. If subjected to blind tests they fail and claim the differences are just too subtle to be heard under these conditions. Too Subtle or Nonexistent? We hear every excuse that the subjectives can make up but in the end until you submit to blind tests there is no way for anyone to know if what you say you hear is just too "subtle" to prove or just imaginary.

As the Temptations sang in 1971, "It was just my imagination, running away with me."

LOL

[Teresa]

On this issue, Tyll Hertsen's recent summary of "Big Sound 2015" sounds very sensible to me overall. It includes subsections on blind and sighted listening. Would be interested in responses:

 

Big Sound 2015 Wrap: What I Learned | InnerFidelity

 

Thanks for the link Christopher, here are my favorite quotes from the article.

 

"Similarly, I think when we listen for pleasure to a piece of gear we are neither attracted or off-put by the individual details, but rather, experience the sound as a whole. But the sound, as a whole, is made up of all it's constituent small, but subtle, details. My point here is that while individual subtle details may be difficult to consciously discern, the mind will subconsciously sum the myriad subtle but important details and sense the overall character of the product. It is that experience which allows us to come to personally valid impressions."

 

The above reasons given are why I feel long-term listening under relaxed pleasurable conditions is the only way to know how any audio device presents the music one loves.

 

"I need to add that there is also a time dimension to the relationship with a piece of gear. Over time our mind accommodates to the sound of something. The brain modifies its perception to normalize the sound it hears."

 

I believe this would partially explain why new products sound better with use, it's not just break-in, it is ourselves becoming accustomed to its sound.

 

"Just because someone's carefully formed opinion is different than yours doesn't make one of you right and the other wrong—at least, insofar as relatively good kit goes."

 

I agree.

[davide256]

It's the same for audio as it is for any other scientific technology.

If you can't back up your your opinion with double blind testing and pure accurate measurements it will forever remain just that, YOUR OPINION.

If you say amp A sounds better than amp B and Joe Listener comes to a different conclusion, who's right? How do you prove it?

This is a position that would be laughed out of any scientific investigation accept audio

Just ridiculous!

 

 

Curious. I've never seen double blind used in any technology scientific investigation... it could in many cases be lethal. I have seen it used in psychology and marketing where someone is trying to distill a simple result from complex human perception.

[mmerrill99]

Curious. I've never seen double blind used in any technology scientific investigation... it could in many cases be lethal. I have seen it used in psychology and marketing where someone is trying to distill a simple result from complex human perception.

Yep, & this distillation isn't an easy process - it requires careful experimental setup to account for the complexity of perception along with repeating the tests for a statistically significant number of times - it requires professionals who are experts in the field to set up these experiments if they are to have any scientific validity.

 

The current trend in the use of neuroimaging in auditory research is a step in the right direction.

 

DBT use in medicine require blood, urine or other physiological samples to be taken for measurement & analysis in order to test for the efficacy of the treatment. fMRI, MEG, PET & other neuroimaging tests are now beginning to fulfill this role in the field of perceptual research

[gmgraves]

Why stop there? Think of all the crazies who dismiss all the evidence for divine creation simply because they are unenlightened human beings who pay more attention to the fossil record than they do to the Biblical record, which is the bit-perfect word of God.

 

Think of how much more humane modern medicine would be if we would simply acknowledge that God, rather than natural selection, creates and propagates antibiotic resistance in bacteria.

 

 

Never mind!

[gmgraves]

" Big Sound 2015 taught me that blind tests are really only useful for professionals evaluating small details, and really shouldn't be used by casual listeners or enthusiast n00bs."

 

He didn't just learn that you don't use a pool of Joe Blows to evaluate high end gear did he?

 

"They would often laugh and say, "No way! If I can't tell the differences in amps with the HE-1000, I'm certainly not going to be able to test the differences between cables.") It is very important for entusiasts not to over-enthusiastically claim huge differences in these areas—it makes objectivists crazy...and rightly so."

 

Yep, because in large numbers of the audible claims they are either imagined or made up for profit motives. If subjected to blind tests they fail and claim the differences are just too subtle to be heard under these conditions. Too Subtle or Nonexistent? We hear every excuse that the subjectives can make up but in the end until you submit to blind tests there is no way for anyone to know if what you say you hear is just too "subtle" to prove or just imaginary.

As the Temptations sang in 1971, "It was just my imagination, running away with me."

LOL

 

 

The ear/brain "system" is very sensitive and if one trains oneself to "notice" certain characteristics of recorded sound playback (the legendary "golden-eared" audiophile) one can glean much from listening. However, the ear/brain is not an instrument. It cannot quantify and it cannot "measure or compare" impartially and without passion. Many things affect the listening outcome. These include such things as mood, circumstances (like peer pressure), even health. Also personal belief systems play a large part in what one hears (the so-called "expectational bias"). If one believes that all amplifiers and all DACs and CD players sound alike, then, like with the notorious Arny Kruger, they will all sound alike. This makes even double-blind tests somewhat unreliable (in my estimation) for audio. If many such tests, over a long period of time, all over the world, come up with the same conclusions, then DBT results take on a statistical consensus that makes their findings less likely to be the result of imaginative flights of fancy. But those results are likely still not 100% conclusive. I am reminded of the famous Meyer and Moran ABX test where they found that hundreds of college students couldn't tell the difference between an SACD playback straight-up and the same SACD through a daisy-chain loop of a number of 16-bit/44.1 KHz ADCs and DACs. In fact, their results were that listeners could detect differences only about 37% of the time, far below the level for blind chance. From that result are we to gather that there is no reason to trouble ourselves with SACD/DSD (or any other kind of so-called high-resolution recording/playback methodologies)? Even though if one reads the Meyer and Moran paper, one will see that their procedure was impeccable and the playback equipment used were all of "high-end" pedigree and should have shown clear differences between the samples, no such clear differences were observed, so, in light of what we know to be true, obviously something is wrong here (either with the audiophile community's expectations, or with the results of the study. Even Bob Stuart of Meridian found the Meyer and Moran study to be flawed as have many other industry pundits.

[Sal1950]

Even Bob Stuart of Meridian found the Meyer and Moran study to be flawed as have many other industry pundits.

 

That study was ridiculously flawed from step one. Why in Gods name would anyone set up a test to listen for subtle differences using a bunch of college students rather than experienced trained listeners?

 

If you brought your car in and it was missing on one cylinder I could call why it was down on power instantly just by listening. I wouldn't call in college students in liberal arts to diagnose a engine that was down on power???

[Rexp]

That study was ridiculously flawed from step one. Why in Gods name would anyone set up a test to listen for subtle differences using a bunch of college students rather than experienced trained listeners?

 

If you brought your car in and it was missing on one cylinder I could call why it was down on power instantly just by listening. I wouldn't call in college students in liberal arts to diagnose a engine that was down on power???

I think the opinion of music loving college students is just as valid as the opinion of an Audiophile. I doubt there was a big difference in the sound, most digital players, high res or not sound digital as Michael Lavorgna at AudioStream (a fairly experienced listener) recently admitted when he reviewed the Totaldac d1 tube DAC:

I've been at this AudioStream gig for over four years and it has been my experience that most DACs I've heard fit on a scale from digital sounding to really-good-sounding digital. I listen to my reference Auralic Vega DAC ($3500 see review) most days and to my ears it sounds like really good digital. For the price, I also think that's very special. The totaldac d1-tube-mk2 DAC doesn't sit anywhere on that scale.

When I say digital sounding, one thing I'm referring to is the perception that there's something in-between me and my music. Of late, I've come to picture this barrier as a sheet of glass. Listening through the totaldac late at night in the relative darkness, I can listen in to my music as deeply as I care to go. There is nothing in between it and me except my thoughts, so when I connect to the music and quiet my brain's activity, it's as if space (and time) are defined by the music. This is in distinct contrast to most other DACs I've heard where that same experience is cut short, as if my journey into the music's space (and time) bumps up against a pane of glass. I can 'see' through it, I can mostly ignore it, but it's there. I prefer when it's not.

 

[Sal1950]

As in Digital sounding you do mean as superior sounding to analog/vinyl, correct?

That has been my experience as an audiophile since about 1985

[Teresa]

...DBT use in medicine require blood, urine or other physiological samples to be taken for measurement & analysis in order to test for the efficacy of the treatment. fMRI, MEG, PET & other neuroimaging tests are now beginning to fulfill this role in the field of perceptual research

 

Correct.

 

However I just want to point out that medical DTB's don't require the conscious use any of our five senses.

 

DBTs work in drug testing as the human subjects don't have to make any decisions whatsoever. The subjects either are given the real medicine or a sugar pill. Those who get well taking the sugar pill do so as unconsciously their believe the medicine is real thus their antibodies manage to kill the disease, this is known as the placebo effect. If considerably more people get well with the new drug than with the sugar pill, the drug is considered effective. None of our five senses come into play in this type of test.

[Teresa]

...When I say digital sounding, one thing I'm referring to is the perception that there's something in-between me and my music. Of late, I've come to picture this barrier as a sheet of glass...

 

Thanks, an interesting explanation. What I like about the better well-mastered high resolution PCM and DSD music files is that in addition to sounding more lifelike, is their ability to remove the strident, dry, shrill, cold sound that used to be associated with digital playback and replace it with the smooth, comfortable sound I find with analog reel to reel tapes and to a lesser degree audiophile analog LPs, especially direct to disc ones.

 

...Listening through the totaldac late at night in the relative darkness, I can listen in to my music as deeply as I care to go. There is nothing in between it and me except my thoughts, so when I connect to the music and quiet my brain's activity, it's as if space (and time) are defined by the music. This is in distinct contrast to most other DACs I've heard where that same experience is cut short, as if my journey into the music's space (and time) bumps up against a pane of glass. I can 'see' through it, I can mostly ignore it, but it's there. I prefer when it's not.

 

Sounds like you found a nice sounding DAC.

[mmerrill99]

Correct.

 

However I just want to point out that medical DTB's don't require the conscious use any of our five senses.

Yep & that was my point - neuroimaging is replacing a lot of these crude & difficult to administer testing methods previously used in perceptual testing - mostly to tease out the details & pathways involved in auditory processing.

 

DBTs work in drug testing as the human subjects don't have to make any decisions whatsoever. The subjects either are given the real medicine or a sugar pill. Those who get well taking the sugar pill do so as unconsciously their believe the medicine is real thus their antibodies manage to kill the disease, this is known as the placebo effect. If considerably more people get well with the new drug than with the sugar pill, the drug is considered effective. None of our five senses come into play in this type of test.

Agreed!

[Rexp]

Thanks, an interesting explanation. What I like about the better well-mastered high resolution PCM and DSD music files is that in addition to sounding more lifelike, is their ability to remove the strident, dry, shrill, cold sound that used to be associated with digital playback and replace it with the smooth, comfortable sound I find with analog reel to reel tapes and to a lesser degree audiophile analog LPs, especially direct to disc ones.

 

 

 

Sounds like you found a nice sounding DAC.

I'm actually quoting Michael Lavorgna you can read his full review here: totaldac d1-tube-mk2 DAC | AudioStream

 

But he sums up what I feel about digital pretty well and would love to own the Totaldac!

[Daudio]

I'm actually quoting Michael Lavorgna...

 

Then it would have been nice if you used the double quote characters (") around the quoted text. Or, used the right-most button on the post editor, or even put a blank line between your words and Lavorgna's, or use other techniques to make a clear distinction between your own words and those of another person. That is, if your goal is to communicate and not to confuse.

 

Sorry, I don't mean to pick on you, but the quality of online posts can be pretty terrible sometimes, and the writer may not recognize some of the problems (like no capitalization, no punctuation, run-on sentences, unknown abbreviations, etc, etc.). So consider this a reminder to everyone.

[PeterG]

Thanks for the recommendation--really enjoying the Phillips training on the link!

[gmgraves]

That study was ridiculously flawed from step one. Why in Gods name would anyone set up a test to listen for subtle differences using a bunch of college students rather than experienced trained listeners?

 

If you brought your car in and it was missing on one cylinder I could call why it was down on power instantly just by listening. I wouldn't call in college students in liberal arts to diagnose a engine that was down on power???

 

 

Well, ostensibly, college students can hear better than any audiophile duffers, for all of our supposed "golden ears."

[gmgraves]

Correct.

 

However I just want to point out that medical DTB's don't require the conscious use any of our five senses.

 

 

And excellent point, Teresa! A cogent and important difference, that, until you mentioned it just now, had never occurred to me on a conscious level.

[Audio_ELF]

However I just want to point out that medical DTB's don't require the conscious use any of our five senses.

 

And excellent point, Teresa! A cogent and important difference, that, until you mentioned it just now, had never occurred to me on a conscious level.

 

Except in studies where they do... Think of (for instance) a test of pain medication, that would require a subjective response on the part of the patient. Also we have a lot more than 5 senses ... Neurologists will say at least 9 and up to as many as 21 depending how they are defined - sight, hearing, smell, touch and taste are just simple umbrella terms and miss out things such as balance, sense of temperature, etc.

 

Personally I feel blind testing in various forms has a place, but it's not the start and end of audio testing; and it's worth depends on the situation and what is being tested.

[Jud]

Except in studies where they do... Think of (for instance) a test of pain medication, that would require a subjective response on the part of the patient.

 

Yes, but such tests must then take placebo effect into greater account and are correspondingly less reliable. (It is thought, for example, that placebo effect may account for some or all of the perceived benefit of acupuncture, but the fact that you obviously know someone is sticking needles into you makes this more difficult to determine for certain.)

[Rexp]

Yes, but such tests must then take placebo effect into greater account and are correspondingly less reliable. (It is thought, for example, that placebo effect may account for some or all of the perceived benefit of acupuncture, but the fact that you obviously know someone is sticking needles into you makes this more difficult to determine for certain.)

Acupuncture cured my Sciatica when no amount of Harley Street doctors could, highly recommended.

[Audio_ELF]

Yes, but such tests must then take placebo effect into greater account and are correspondingly less reliable.

I was just pointing out that Teresa's statement is not as clear cut as she would like it to be...

[davide256]

I think its safe to say that setting up a double blind test increases the complexity of testing by an order of magnitude and introduces new error variables. I prefer the real life science method of concensus of testers, stating their test methodology

[Paul R]

Well, ostensibly, college students can hear better than any audiophile duffers, for all of our supposed "golden ears."

 

Ah here is an example of what is actually a problem. Physically, most young college students may actually be able to hear higher frequency sound and hear all frequencies a little clearer. However, on the average, those same kids are much less capable of understanding what they hear.

 

So in a very real sense, they cannot hear anywhere near as well as the old codger who can barely hear a 12K signal, but has decades of experience listening.

 

Like the old story of the the company that hired an engineer to come in and fix an engine. The engineer comes in, listens to the engine, then reports it will $1200 to fix it. The company agrees to pay for the fix, the engineer takes a wrench and tightens up three bolts. Problem fixed. The company bean counter complains to the engineer they are paying him an outrageous sum for tightening three bolts.

 

The engineer replies the cost to tight the bolts was $5. The cost for knowing what bolts to tighten? $1195.

 

Semantics in a way, but a very important distinction. (i.e. the Map is Not the Territory!)

 

-Paul