The walking dead and "blind listening"

[Sal1950]

"How did you miss the reasons I gave why AB tests either blind or sighted do not work."

Those are just the usual excuses, heard em all before, Yawn

 

Thought just maybe I could get you to open your eyes to the other aspects of progress.

No such luck I guess.

Ya'll keep on giving your dough to the snake-oil men. LOL

[sandyk]

Before I purchased the awful sounding Sony CDP-101 (the first Sony CD player) I believed in advertising, reviews and product brochures. Early CDs and early CDs player sounded so ugly, strident, dry, shrill, cold that is was a wake-up to not believe anything until I could hear it in my room, with my system and with my ear brain system. I lost a lot of money as that was back in 1983 when there was no satisfaction guarantees.

 

So yes I only buy what I like and am amazed anyone would advise doing anything different.

 

It is objectivists who would have stifled all audio advancement. It is only complaints from subjectivists that motivate audio designers to improve their products and formats.

 

Likewise.

I was the first private owner of a Sony CDP 101 on Sydney's North Shore, and I had to play discs loaned from the Sony store for a few days until CDs were officially launched.

[Teresa]

Those are just the usual excuses, heard em all before, Yawn

 

Thought just maybe I could get you to open your eyes to the other aspects of progress.

No such luck I guess.

Ya'll keep on giving your dough to the snake-oil men. LOL

 

You have a testing procedure (ABX/DBT) that has never worked in its history, with the exception of extremely large differences. How can you believe in such a fraudulent testing procedure? I gave no excuses but real valid reasons why AB'ing doesn't work with human beings. Are you so close-minded you can't give up your ABX/DBT religion which makes everything under the sun sound statistically the same? Why? What is your agenda? Are you a troll?

 

It is you who is closed to how the world really works, not me.

 

I give zero dough to snake-oil men, as I demand an iron-clad money back guarantee and listen with a wide variety of music in as many listening conditions I can. I have returned for refund more products than I have purchased. I have to be able to enjoy music with whatever it is, and it has to be reasonably priced. Like you I don't buy expensive cables, etc. as I can't afford them. But I don't the gall you do, to belittle and make fun of people who do. I accept that they evaluated them before paying for them. I also never comment on something I have not tried.

 

You attitude is unacceptable IMHO.

[Jud]

The subjective community has tried to discredit ABX/DBT protocols with all kinds of BS.

Bottom line, if you can't prove you hear what you say you hear in a controlled scientific manner it will forever remain just an opinion. Forever an opinion

 

It's interesting that the two times I've quoted or cited actual science (peer reviewed scientific journal articles in this case) to you on these forums, your reaction hasn't been to be curious and ask for more information about the science you claim to support, but to close the door to any further learning by inaccurately calling BS.

 

I agree about the usefulness - the necessity, really - of applying science to advance sound reproduction. That's why I'm curious to learn as much as I can about what the science actually says.

[Sal1950]

Jud, I thought by now you understood my opinion of those articles you posted, I'll repeat mine.

 

Bottom line, if you can't prove you hear what you say you hear in a controlled scientific manner it will forever remain just an opinion. Forever an opinion

[Paul R]

"How did you miss the reasons I gave why AB tests either blind or sighted do not work."

Those are just the usual excuses, heard em all before, Yawn

 

Thought just maybe I could get you to open your eyes to the other aspects of progress.

No such luck I guess.

Ya'll keep on giving your dough to the snake-oil men. LOL

 

Oh horse hockey there dude. Hate to break it to you, but just like all the rest of us, you do not know everything nor are your opinions automatically right. (grin)

 

While in general I agree with you, the dismissive attitude is not science, just a form a bullying.

 

In regard to AB tests, especially in terms of audio, they are not always conclusive, accurate, or even guaranteed to be an accurate measure of what the tester thinks they are testing. That is rather difficult to dispute.

 

As for the reasons the above is true, yes - those are endlessly arguable. But that nether negates nor changes the above statement.

 

-Paul

[Sal1950]

While in general I agree with you, the dismissive attitude is not science, just a form a bullying.

 

In regard to AB tests, especially in terms of audio, they are not always conclusive, accurate, or even guaranteed to be an accurate measure of what the tester thinks they are testing. That is rather difficult to dispute.

 

As for the reasons the above is true, yes - those are endlessly arguable. But that nether negates nor changes the above statement.

 

-Paul

Paul,

True, they are "not ALWAYS conclusive, accurate, or even guaranteed to be an accurate measure of what the tester thinks they are testing"

Neither can they ALWAYS to be dismissed as irrelevant. They are still considered one of the gold standards of scientific tests and the subjective communities constant attempts to discredit them are only maneuvers to avoid having their feet held to the fire and provide some level of proof for their claims. The result is a dis-service to goals we all hold as our desire.

[sandyk]

They are still considered one of the gold standards of scientific tests and the subjective communities constant attempts to discredit them are only maneuvers to avoid having their feet held to the fire and provide some level of proof for their claims.

 

What a load of unmitigated crap !

Subjectivists reject them because they are highly flawed and invariably yield the desired NEGATIVE result the know-it-all E.E.s so desperately desire, to show just how much smarter they are . Even when they do yield POSITIVE results, as 6 separate sessions did with comparison .wav files I provided to well experienced E.E. and technical journalist Martin Colloms, the results are dismissed as they weren't performed the way they wanted them to be performed, ( not near enough samples etc.) or we are accused of lying about the results or the methodology used.

In any event, you simply aren't worth discussing this with further as you have already completely made up your mind that ALL Subjective reports are a figment of their posters' imaginations !

BYE !

[Audio_ELF]

If we're talking about reviews and personal decision making then we are all biased in our opinions if we know what the equipment is, what it costs, etc. In some cases its a knowledge of how other devices from a manufacturer sounds; in others a belief that there must be better sound from more expensive equipment and for other people its a desire for that "bargain" to sound better than its price belies.

 

There are lots of arguments over listening tests and how they should be carried out and I agree, in most cases, strict double blind tests are not the way to go; as commented they put too much pressure on the listeners. However there are cases - such as when you are doing a study to show that a particular change creates an audible difference - where DBT is the only valid scientific method; and the details of how that DBT is carried out is as important as the results which is why I have questioned the results of some so called DBT studies.

 

On the other hand, a casual blind test is easy for many people (especially for example a group test of amplifiers in a magazine or at a HiFi club meet) is easy to carry out and removes much of the subjective bias:

 

Take a group test of amplifiers which is carried out in an evening or over a day. The sighted test has the group sat down and each amplifier is on display, plugged in in turn and each time people listen and give their impressions. Now some people will have a preference to the perceived qualities of Amplifier Brand A over Brand B and C; others will prefer B over C and A. Its easy to remove this bias if you are so inclined, simply remove the amplifiers to behind a cloth or into a separate room. Still listen to each amplifier in turn; if you want to be thorough and have time listen to amplifiers each twice but in a random order (C, A, B, A, C, B); take your time, relax between each listen. Only once you finish should you discuss your opinions.

 

You can even do this yourself with a helpful dealer, ask that instead of him setting each amplifier up in turn, ask that he places a selection of 3 amplifiers for you to test on the rack and he doesn't tell you which you listening to each time, but you have free reign to listen to each as long and as relaxed as you like.

 

Speakers are harder to test this way of course.

 

No this isn't truly scientific, but adds an element of blind listening in. Now some people will still not be satisfied, but ideally all testing / evaluation should be done in this manner. To a professional reviewer, I would even go as far to say: If you want to truly and subjectively test equipment without bias, you should not know its cost or its manufacturer until you have completed your review; though that introduces difficulties which would require time and money to overcome - different device types being easier or harder to evaluate this way.

 

At the end of the day I believe in equipment sounding different, however I also believe that as humans we are swayed by our eyes and other knowledge meaning that sighted listening tests do have flaws.

 

Eloise

[semente]

In my view, the problem with human-sourced info is that our listening abilities are tinted by our musical and sonic culture and guided by our taste; it would be more useful if we could evaluate performance from a observational perspective but in practice I find such "selflessness" virtually impossible to achieve.

As such, subjective reviews are nearly always of little use to others. They are personal accounts of a non transferable experience.

 

But I agree that "the whole point of better sound quality in audio gear is for humans to enjoy listening to recorded music"; what it matters above all is to enjoy your system, or better yet, the way it reproduces your favourite music.

 

R

 

Well said with one exception.

If you enjoy the sound of your system/music with an EQ that tilts the response like a roller coaster that's cool too. Just don't call it High Fidelity.

 

High Fidelity generally refers to an absolute respect for the recorded signal.

And, in theory, one should have High Fidelity as main objective.

 

But this would require that the recording itself was done in such a manner as to provide a reasonably realistic recreation of the original musical event and this is not always (or hardly ever) the case.

Most pop and rock recordings are made in semi-anechoich environment, with the (often monaural) mic positioned where one's ears would never be or the instruments directly connected to the mixing console, the players often performing alone in a boot while listening to a mix of his colleagues' previously recorded parts (so much for a musical event), with the final mixture EQ'ed to taste... "Realism" is not a goal.

And even in "classical" music one one will soon realise that there isn't a standard for "naturally" sounding (timbre- and perspective-wise) recordings, although I tend to think that some engineers intentionally ignore the best practices in favour of techniques that will make their work recognisable (i.e. 2L.no)... Ego gets in the way?

 

So whilst an absolute respect for the recorded signal may make sense for "classical" listeners, its usefulness is less obvious for rock and pop lovers.

 

And we are ignoring the fact that even the best recordings will not sound "real", even when playback by the best systems in the most adequately purpose-built rooms.

 

That said, I do find it strange how some people can put up with some frequency response related aberrations such as tweeters 3 or even 6dB too "hot", a "hard-sounding"/fatiguing exaggeration of the presence region or a huge hole in the mid-bass...

From what I read in most forums, people's focus is generally aiming at accessory qualities like detail/resolution or sonic effects ("air", "soundstage") above timbre and "realism".

 

R

[sandyk]

Take a group test of amplifiers which is carried out in an evening or over a day. The sighted test has the group sat down and each amplifier is on display, plugged in in turn and each time people listen and give their impressions. Now some people will have a preference to the perceived qualities of Amplifier Brand A over Brand B and C; others will prefer B over C and A. Its easy to remove this bias if you are so inclined, simply remove the amplifiers to behind a cloth or into a separate room. Still listen to each amplifier in turn; if you want to be thorough and have time listen to amplifiers each twice but in a random order (C, A, B, A, C, B); take your time, relax between each listen. Only once you finish should you discuss your opinions.

 

Eloise

It's not a good idea to plug them in in turn, as some amplifiers take a considerable amount of time to fully stabilise for their best performance. It also greatly increases the time delay between listening to the various samples..

If the amplifiers are already powered up, quite often the majority of those present will not know which amplifier is actually in use. Neither should they know.

 

Regards

Alex .

[Audio_ELF]

It's not a good idea to plug them in in turn, as some amplifiers take a considerable amount of time to fully stabilise for their best performance. It also greatly increases the time delay between listening to the various samples.

Thats true ... though my meaning was more "plugged into the source and speakers in turn.

 

If the amplifiers are already powered up, quite often the majority of those present will not know which amplifier is actually in use. Neither should they know.

Its been my experience that often in such tests it's often announced "We will now listen to the McIntosh amplifier" (to take a brand completely at random). So long as that announcement is not made you are already into the realm of blind (though not double blind) testing. Ideally you would listen to each amplifier twice (as I suggested).

[gmgraves]

Likewise.

I was the first private owner of a Sony CDP 101 on Sydney's North Shore, and I had to play discs loaned from the Sony store for a few days until CDs were officially launched.

 

Arny Kruger still has his CDP-101 and thinks it sounds fine, and in fact, he told everyone one time (on a usenet audio forum) that it sounds exactly like some five-thousand dollar CD player he was then currently auditioning! I already had suspicions that Kruger couldn't hear, but that bit of news cinched it for me! He also still has a Dynaco Stereo 120 and has told everybody that the unit's notorious crossover distortion notch was inaudible! How this man maintains any credibility at all in the audio world (he's a 'famous' objectivist and the guy who "invented" the ABX comparator) is beyond me.

 

I too had a CDP-101 for about 2 days. It sounded so bad, to me, that I took it back for a refund and bought the little 14-bit Philips/Magnavox player instead. It was slightly cheaper than the Sony (and about as pretty as a piece of hi-fi gear could be) and sounded infinitely better!.

 

[Paul R]

Paul,

True, they are "not ALWAYS conclusive, accurate, or even guaranteed to be an accurate measure of what the tester thinks they are testing"

Neither can they ALWAYS to be dismissed as irrelevant. They are still considered one of the gold standards of scientific tests and the subjective communities constant attempts to discredit them are only maneuvers to avoid having their feet held to the fire and provide some level of proof for their claims. The result is a dis-service to goals we all hold as our desire.

 

Of course AB test results are not always something that can or should be dismissed.

 

To assume they are the "gold standard" in audio, however, is not a valid assumption. They certainly are not the "gold standard" for all audiophiles, as many people find the results to be questionable. (Again, not arguing the reasons they find them questionable, just that they do.)

 

Manufacturers - other than Bose of course - tend to publish the physical specifications of their gear. Measurements are usually from trusted and reliable labs. Those measurements are often backed up by reliable and trusted second opinions, such as John Atkinson and his lab at Stereophile.

 

It is far more likely that subjective results and AB tests do agree. Again, when they do not agree, the reasons can be arguable, but things like listener fatigue, using unfamiliar systems, unfamiliar or inferior sources, and other such questions often have not been adequately answered or even considered by some folks who are quick to declare all subjective data as fantasy.

 

 

-Paul

[Paul R]

Arny Kruger still has his CDP-101 and thinks it sounds fine, and in fact, he told everyone one time (on a usenet audio forum) that it sounds exactly like some five-thousand dollar CD player he was then currently auditioning! I already had suspicions that Kruger couldn't hear, but that bit of news cinched it for me! He also still has a Dynaco Stereo 120 and has told everybody that the unit's notorious crossover distortion notch was inaudible! How this man maintains any credibility at all in the audio world (he's a 'famous' objectivist and the guy who "invented" the ABX comparator) is beyond me.

 

I too had a CDP-101 for about 2 days. It sounded so bad, to me, that I took it back for a refund and bought the little 14-bit Philips/Magnavox player instead. It was slightly cheaper than the Sony (and about as pretty as a piece of hi-fi gear could be) and sounded infinitely better!.

 

[ATTACH=CONFIG]21693[/ATTACH]

 

(*sigh*) Arny Kruger is a really shining example of why AB tests are treated with so much suspicion by so many audiophiles. The guy definitely has an agenda that many people dispute.

[gmgraves]

They are still considered one of the gold standards of scientific tests and the subjective communities constant attempts to discredit them are only maneuvers to avoid having their feet held to the fire and provide some level of proof for their claims.

 

What a load of unmitigated crap !

Subjectivists reject them because they are highly flawed and invariably yield the desired NEGATIVE result the know-it-all E.E.s so desperately desire, to show just how much smarter they are . Even when they do yield POSITIVE results, as 6 separate sessions did with comparison .wav files I provided to well experienced E.E. and technical journalist Martin Colloms, the results are dismissed as they weren't performed the way they wanted them to be performed, ( not near enough samples etc.) or we are accused of lying about the results or the methodology used.

In any event, you simply aren't worth discussing this with further as you have already completely made up your mind that ALL Subjective reports are a figment of their posters' imaginations !

BYE !

 

 

I think that Eloise hit the nail on the head when she mentioned that in science, DBTs rarely ask for an opinion from the testers that requires the use of any of their 5 senses. They are comparing hard data, and hard data leaves little room for doubt. In drug tests, those being tested are examined by physicians who don't know which participants are the controls and which are actually taking the drugs being tested. Since the participants don't know which group they are in either, all they can say (in the way of an opinion) is whether they feel better or not and these opinions are not taken at face value, but are backed-up by a medical examination. This is quite different from asking a room full of listeners as to whether or not they heard a difference between two test components and that difference probably makes all the difference when evaluating the efficacy of DBTs in audio.

 

Again Kudos to Eloise.

[Jud]

Jud, I thought by now you understood my opinion of those articles you posted, I'll repeat mine.

 

Bottom line, if you can't prove you hear what you say you hear in a controlled scientific manner it will forever remain just an opinion. Forever an opinion

 

Yes, you like science except when it disagrees with your opinion, I've got that.

 

The difference between us is that I love science *particularly* when it differs from my preconceived opinion, because it gives me a chance to learn.

 

So you're just going to stick with your preconceived opinions about DBT and audio regardless of what the actual science says, do I have that right?

[Audio_ELF]

I think that Eloise hit the nail on the head when she mentioned that in science, DBTs rarely ask for an opinion from the testers that requires the use of any of their 5 senses.

Actually that was Teressa who noted that, though I commented that in some tests there is an element of "how do you feel" with DBTing of drugs. Having said that, I think there is also an element in the treatment stage where even doctors will ignore all the hard data and rely on their experience, anecdotal evidence and some of their own prejudices in choosing what treatment plan is best for a particular patient...

 

I have mostly been arguing that blind tests have their place in evaluating equipment, but not strict double blind tests just (as far as possible) removing the influence of knowing the brand and cost of a piece of equipment.

 

Eloise

[Teresa]

...Bottom line, if you can't prove you hear what you say you hear in a controlled scientific manner it will forever remain just an opinion. Forever an opinion.

 

I totally agree with you here and will go further, there is no possible way to prove what one person hears to another person. However, to me it does not matter as I accept as a given that all personal audio opinions are just that personal audio opinions and don't need to pretend to be more.

 

Everything except for the small amount of specifications that we can actually measure are scientifically unprovable.

 

Bottom line is, it doesn't matter in the least to me, personal opinions are personal opinions and to pretend they can be more is just silly IMHO.

[Teresa]

They are still considered one of the gold standards of scientific tests and the subjective communities constant attempts to discredit them are only maneuvers to avoid having their feet held to the fire and provide some level of proof for their claims.

 

What a load of unmitigated crap !

Subjectivists reject them because they are highly flawed and invariably yield the desired NEGATIVE result the know-it-all E.E.s so desperately desire, to show just how much smarter they are...

 

I agree, DBT tests are only the gold standard when using objective data obtained from none of the human major five senses, such as measurements, physical exams, blood tests, brain scans etc. Once human beings are asked to make conscious decisions on their own such tests are no longer valid.

[Sal1950]

Yes, you like science except when it disagrees with your opinion, I've got that.

 

The difference between us is that I love science *particularly* when it differs from my preconceived opinion, because it gives me a chance to learn.

 

So you're just going to stick with your preconceived opinions about DBT and audio regardless of what the actual science says, do I have that right?

 

Nope, the real difference is you'l devise any excuse not to have your claims of audibility put to controlled tests. All Cop outs

 

ROTFLMAO, Wind em up and watch em spin, just look at this thread of subjectives on the run from the light.

[Audio_ELF]

Nope, the real difference is you'l devise any excuse not to have your claims of audibility put to controlled tests. All Cop outs

 

ROTFLMAO, Wind em up and watch em spin, just look at this thread of subjectives on the run from the light.

Sal ... Reading above you may get a general feeling of my opinion on blind testing but I will remind you of an adage... An absence of proof is NOT proof of absence. That is that should a proper DBT prove people can tell the difference between (for example) different cables then that does prove that there is an audible difference between them; however because no test has yet shown an audible difference doesn't prove that there is no difference.

[Sal1950]

Sal ... Reading above you may get a general feeling of my opinion on blind testing but I will remind you of an adage... An absence of proof is NOT proof of absence. That is that should a proper DBT prove people can tell the difference between (for example) different cables then that does prove that there is an audible difference between them; however because no test has yet shown an audible difference doesn't prove that there is no difference.

 

LOL Yep that is a good adage.

But that still leaves us with a total lack of proof one way or the other. No blind testing but a completely blind high end community.

[gmgraves]

Actually that was Teressa who noted that, though I commented that in some tests there is an element of "how do you feel" with DBTing of drugs. Having said that, I think there is also an element in the treatment stage where even doctors will ignore all the hard data and rely on their experience, anecdotal evidence and some of their own prejudices in choosing what treatment plan is best for a particular patient...

 

I have mostly been arguing that blind tests have their place in evaluating equipment, but not strict double blind tests just (as far as possible) removing the influence of knowing the brand and cost of a piece of equipment.

 

Eloise

 

 

Sorry for mixing you ladies up. I have always been bad at remembering names is my only excuse. Again, I apologize.

[gmgraves]

Sal ... Reading above you may get a general feeling of my opinion on blind testing but I will remind you of an adage... An absence of proof is NOT proof of absence. That is that should a proper DBT prove people can tell the difference between (for example) different cables then that does prove that there is an audible difference between them; however because no test has yet shown an audible difference doesn't prove that there is no difference.

 

 

IOW, you can't prove a negative. Not finding Atlantis on the floor of the Atlantic ocean does not prove that the continent didn't exist there, but actually finding it will prove that it did.*

 

So, by that logic, a negative result in a DBT, will not prove that there are no differences between components being compared, while a positive result will prove that differences do exist.

 

*Just an example. I suspect the legend of Atlantis is based upon the destruction of the so-called Minoan civilization by the eruption of the volcanic island of Thera (now Santorini) in the Aegean. The eruption destroyed their capital on Thera and the resulting tsunami destroyed the settlements on the other islands and the north shore of Crete.