The walking dead and "blind listening"

[gmgraves]

LOL Yep that is a good adage.

But that still leaves us with a total lack of proof one way or the other. No blind testing but a completely blind high end community.

 

 

Not completely blind. But some audiophiles are rather completely at the mercy of their aural impressions, and don't really care about the reality because they have no technical background and don't really think that they need any. They feel that their ears tell them all they need to know. You just have to accept that.

[Ajax]

If we're talking about reviews and personal decision making then we are all biased in our opinions if we know what the equipment is, what it costs, etc. In some cases its a knowledge of how other devices from a manufacturer sounds; in others a belief that there must be better sound from more expensive equipment and for other people its a desire for that "bargain" to sound better than its price belies.

 

There are lots of arguments over listening tests and how they should be carried out and I agree, in most cases, strict double blind tests are not the way to go; as commented they put too much pressure on the listeners. However there are cases - such as when you are doing a study to show that a particular change creates an audible difference - where DBT is the only valid scientific method; and the details of how that DBT is carried out is as important as the results which is why I have questioned the results of some so called DBT studies.

 

On the other hand, a casual blind test is easy for many people (especially for example a group test of amplifiers in a magazine or at a HiFi club meet) is easy to carry out and removes much of the subjective bias:

 

Take a group test of amplifiers which is carried out in an evening or over a day. The sighted test has the group sat down and each amplifier is on display, plugged in in turn and each time people listen and give their impressions. Now some people will have a preference to the perceived qualities of Amplifier Brand A over Brand B and C; others will prefer B over C and A. Its easy to remove this bias if you are so inclined, simply remove the amplifiers to behind a cloth or into a separate room. Still listen to each amplifier in turn; if you want to be thorough and have time listen to amplifiers each twice but in a random order (C, A, B, A, C, B); take your time, relax between each listen. Only once you finish should you discuss your opinions.

 

You can even do this yourself with a helpful dealer, ask that instead of him setting each amplifier up in turn, ask that he places a selection of 3 amplifiers for you to test on the rack and he doesn't tell you which you listening to each time, but you have free reign to listen to each as long and as relaxed as you like.

 

Speakers are harder to test this way of course.

 

No this isn't truly scientific, but adds an element of blind listening in. Now some people will still not be satisfied, but ideally all testing / evaluation should be done in this manner. To a professional reviewer, I would even go as far to say: If you want to truly and subjectively test equipment without bias, you should not know its cost or its manufacturer until you have completed your review; though that introduces difficulties which would require time and money to overcome - different device types being easier or harder to evaluate this way.

 

At the end of the day I believe in equipment sounding different, however I also believe that as humans we are swayed by our eyes and other knowledge meaning that sighted listening tests do have flaws.

 

Eloise

 

Thoughtful post Eloise and reflects my own views - we are all "conditioned" to some degree .... by our upbringing, environment and experiences .... and IMO we need all the help we can get to mitigate our biases so that we are truly listening to what we hear - not what we want to hear or expect to hear.

[Sal1950]

Not completely blind. But some audiophiles are rather completely at the mercy of their aural impressions, and don't really care about the reality because they have no technical background and don't really think that they need any. They feel that their ears tell them all they need to know. You just have to accept that.

 

Cheers to ya George, you see it from a much better mindset than me.

May the force be with ya.

[mdsn]

IOW, you can't prove a negative. Not finding Atlantis on the floor of the Atlantic ocean does not prove that the continent didn't exist there, but actually finding it will prove that it did.*

 

So, by that logic, a negative result in a DBT, will not prove that there are no differences between components being compared, while a positive result will prove that differences do exist.

 

*Just an example. I suspect the legend of Atlantis is based upon the destruction of the so-called Minoan civilization by the eruption of the volcanic island of Thera (now Santorini) in the Aegean. The eruption destroyed their capital on Thera and the resulting tsunami destroyed the settlements on the other islands and the north shore of Crete.

 

George,

 

You don't understand the concept of burden of proof and you don't understand the argument from ignorance fallacy. I've watched you repeatedly demonstrate your ignorance, but never bothered to engage due to your apparent combination of ignorance and arrogance, but today I'll give it a shot.

 

If someone makes a claim without presenting sufficient evidence the best you can do is reject the claim and explain that they hold a belief that is not rationally justified. When you assert that a claim is impossible, which I've seen you do numerous times, you have now adopted your own burden of proof. How did you determine that a cable claim, or even deity claim, is impossible? If your answer is that it hasn't been proven true, therefore, it's false, or impossible, then that would be an argument from ignorance fallacy.

 

X is true because it hasn't been proven false

X is false because it hasn't been proven true

 

Again, the best you can do is to say that you don't accept the claim and make the point that one is not objectively rationally justified in holding the belief at this point.

 

So, how did you determine that any cable or deity claim is "impossible"? How did you determine that a deity is impossible without a clear definition of any deity?

 

It is possible to prove that a positive claim is impossible if one has clear definitions. For example, one can prove that the Christian God is impossible based on the definitions and attributes given for that God. Same goes for all other gods proposed. It's just a matter of pointing out the logical contradictions. Though, typically this shifting of the burden of proof is not necessary and it's best to just leave it to the claim maker to hold to his burden of proof. Unnecessarily shifting the burden of proof and turning the claim from positive to negative usually leads to an argument from ignorance fallacy on the part of the original claim maker, ie., mistakenly thinking their claim is true and/or justified because it hasn't yet been proven false.

 

And again, even if I cannot show that a claim is false or impossible, I can still show that one is not rationally justified in accepting the claim or belief due to insufficient reason or evidence. So, rather than asserting something impossible, it's best to adopt a position of unconvinced. When you assert that X is "impossible" you have to actually demonstrate that. Further, when you assert that something is impossible without demonstrating that, you impede progress. You're asserting that you have it all figured out, and that no one should ever investigate any aspect of some particular idea, at all. Ever!

 

Do you have evidence that any of the various cable claims are impossible, or just an argument from ignorance fallacy that X is false because it hasn't been proven true? Is this the point at which you shift the focus to cable prices, as you typically do?

 

Not finding Atlantis on the floor of the Atlantic ocean does not prove that the continent didn't exist there, but actually finding it will prove that it did.

 

That's correct. I get the sarcasm, but the statement is actually correct. Are you making the positive claim that Atlantis never existed on the floor of the Atlantic? Or, more appropriately, that you are not convinced, and that one is not currently rationally justified in believing that it did? There is a big difference. Understand burden of proof and the argument from ignorance fallacy.

 

I can't say that Russel's Teapot is not orbiting Pluto at this very moment, but I can tentatively reject the claim due to insufficient evidence and will live my life as if it doesn't exist.

 

*Just an example. I suspect the legend of Atlantis is based upon the destruction of the so-called Minoan civilization by the eruption of the volcanic island of Thera (now Santorini) in the Aegean. The eruption destroyed their capital on Thera and the resulting tsunami destroyed the settlements on the other islands and the north shore of Crete.

 

That is a positive claim. What evidence, reason, or justification do you have for making such a claim? Until justified reason or evidence is presented, I will tentatively reject your claim and live my life as if the claim is false. I cannot say that Atlantis, as described, did not exist, or that it's impossible, unless I present evidence that it did not exist, or that it is somehow logically impossible for it to have existed. And my lack of evidence against it's existence in no way validates the claim that it did exist. However, if you point out the exact location and give specific details we may be able to reasonably conclude the Atlantis claim to be false. Definitions and details are extremely important before concluding something impossible.

 

By the way, you also demonstrate your ignorance and flawed thinking when you make the analogy between supernatural religious belief and subjective cable difference belief. Gods and religions are by definition supernatural, whereas, cables difference claims represent possible unknown unknowns.

[Sal1950]

George,

You don't understand the concept of burden of proof and you don't understand the argument from ignorance fallacy. I've watched you repeatedly demonstrate your ignorance, but never bothered to engage due to your apparent combination of ignorance and arrogance, but today I'll give it a shot.

By the way, you also demonstrate your ignorance and flawed thinking when you make the analogy between supernatural religious belief and subjective cable difference belief. Gods and religions are by definition supernatural, whereas, cables difference claims represent possible unknown unknowns.

 

If you have to open and close your post with name calling and nastiness, anything you have written is totally without relevancy and should be totally ignored. If I had mod privileges here I would delete your post and suspend your account for a month or two to see if I could teach you a lesson.

Go home littleone, your mothers calling.

[mdsn]

If you have to open and close your post with name calling and nastiness, anything you have written is totally without relevancy and should be totally ignored. If I had mod privileges here I would delete your post and suspend your account for a month or two to see if I could teach you a lesson.

Go home littleone, your mothers calling.

George has made it very clear on numerous occasions that referring to someone as ignorant is not name calling at all and should not be taken as insulting. He's frequently referred to others as ignorant and offered his defense for using the term. Look at some of his old posts. Or better yet, ask him. And pointing out flawed thinking is name calling to you?

 

You might want get your thoughts in order before posting next time.

 

By the way, are you George? My post was to George.

 

What should mods do to sophomore members who tell new members to "go home littleone"?

[4est]

If you want to look down on someone, you shouldn't stoop so low beforehand.

If you have to open and close your post with name calling and nastiness, anything you have written is totally without relevancy and should be totally ignored. If I had mod privileges here I would delete your post and suspend your account for a month or two to see if I could teach you a lesson.

Go home littleone, your mothers calling.

[mordante]

Like those who will spend thousands of dollars on a few feet of power cord when they have no control over the miles of wire between the outlet and the power station. ROTFLMAO

 

Why do some zealots like you turn every discussion into a cable topic. Please get of you high horse.

[mordante]

bla bla bla

 

I should have said nothing. Cannot edit the post anymore.

[Daudio]

But that still leaves us with a total lack of proof one way or the other. No blind testing but a completely blind high end community.

 

So you have no Magic Bullet philosophy left to hide your lazy thinking behind. So now out comes the bullying behavior. And you have the gall to call for others to be disiplined by the forum 'power that be'.

 

You would probably be happier in the gasbag forum !

 

Now is a good time to IGNORE your ugly, uninformed, opinions. Bye...

[Sal1950]

Why do some zealots like you turn every discussion into a cable topic. Please get of you high horse.

 

SOME discussions will bring up cable topics cause they most clearly highlight the ridiculousness of a totally subjective approach to audio and it's results. One that refuses to be subjected to any type of ABX/DBT protocols or any other truly scientific examinations.

[Sal1950]

Now is a good time to IGNORE your ugly, uninformed, opinions. Bye...

 

Bye and bless your heart.

[Paul R]

SOME discussions will bring up cable topics cause they most clearly highlight the ridiculousness of a totally subjective approach to audio and it's results. One that refuses to be subjected to any type of ABX/DBT protocols or any other truly scientific examinations.

 

Oh, what would it take to highlight the totally idiotic objective-only approach to audio? Do you really need someone to do double blind ABX tests to determine what gear sounds best to you? Or what kit you like the most?

 

There are some things that *only* subjective testing can tell you. For example, whether or not a piece of kit sounds good to *you* or not. No amount of objective testing will ever be able to answer that question. Nor is there any reason that objective only testing should be able to do so.

 

Your insight, thinking, and opinions are interesting. The taunting and bullying behavior is not.

 

-Paul

[semente]

I don't know if this has been mentioned before but wrote an interesting chronicle on this subject:

 

The Blind leading the Deaf

By J. Gordon Holt, Stereophile 4/1987

 

As the person who "invented" subjective testing, I have followed with great interest the many articles in the mainstream audio press which purport to prove that none of us can really hear all the differences we claim to hear, particularly those between amplifiers. My reaction has usually been: "Why didn't they invite me to participate? I would have heard the differences under their double-blind listening conditions." I could make that assertion with supreme confidence because I had never been involved in any such test.

 

A few weeks ago, however, I was involved in such a test. It happened during Bob Carver's visit to Santa Fé, in response to our report on his M1.0t amplifier, which he claimed to sound identical to a well-known perfectionist tube amplifier. Following our extended auditioning of both amplifiers, John Atkinson and I had declared that they didn't sound the same, so Carver insisted that we prove we were hearing differences, a demand which JA and I felt to be completely unwarranted and unreasonable. (Doesn't he trust us?).

 

Both JA and I report in full on the results of these further tests on the Carver on p.117, but I was anticipating no problems whatsoever. After all, the differences I had heard at one point during my preliminary listening were great enough for me to describe them as "dramatic." Certainly, any "dramatic" differences would be immediately audible under any conditions of comparison.

 

Before the blind tests began, I had been listening "informally" for about an hour to both amps, but attributed the fact that I had not been hearing "dramatic" differences to the additions to the system of the lightweight wiring and the switch which Bob had rigged up to allow instant comparisons. I assumed that, when we went back to hard-wired conditions, those "dramatic" differences would re-emerge. To my surprise and chagrin, they didn't.

 

(...)

 

Read more at The Blind leading the Deaf | Stereophile.com

[mordante]

SOME discussions will bring up cable topics cause they most clearly highlight the ridiculousness of a totally subjective approach to audio and it's results. One that refuses to be subjected to any type of ABX/DBT protocols or any other truly scientific examinations.

 

Subjective listening is the only way. All others are meaningless. Music is about enjoyment. Knowing that something measures well or badly has zero influence on the musical experience.

 

Just look at Stradivarius violins. They do not measure or sound better compared to modern violins but still they hold more attraction. Look at motorbikes, from a subjective point of view a small Japanese Honda would most likely be the best. However many people still like archaic motorbikes like Harley Davidson.

 

Life is about emotional experience and whether that comes from measurements, or fancy cables is irrelevant.

[Jud]

SOME discussions will bring up cable topics cause they most clearly highlight the ridiculousness of a totally subjective approach to audio and it's results. One that refuses to be subjected to any type of ABX/DBT protocols or any other truly scientific examinations.

 

All of the following is solely in my own opinion:

 

It's still early days, and you may have taken this place as being like many other Internet forums, where trading insults is thought to be entertainment, calling other people's attitudes, opinions, or positions "ridiculous" is treated as amusing, and the loudest, most outrageous person wins in the end.

 

I like to think we're not quite so outrageously amusing and entertaining, but instead lean a little bit toward friendly, and when we're at our best, educational and helpful. Hope you'll contribute to that kind of community.

[sandyk]

Yesterday I posted a link in another forum to a C.A. discussion. In the reply this morning it said this :

They are quite rude on that forum aren't they!

 

Perhaps Blizzard won't be quite as lonely soon ?

[Sal1950]

I don't know if this has been mentioned before but wrote an interesting chronicle on this subject:

 

The Blind leading the Deaf

By J. Gordon Holt, Stereophile 4/1987

 

As the person who "invented" subjective testing, I have followed with great interest the many articles in the mainstream audio press which purport to prove that none of us can really hear all the differences we claim to hear, particularly those between amplifiers. My reaction has usually been: "Why didn't they invite me to participate? I would have heard the differences under their double-blind listening conditions." I could make that assertion with supreme confidence because I had never been involved in any such test.

 

A few weeks ago, however, I was involved in such a test. It happened during Bob Carver's visit to Santa Fé, in response to our report on his M1.0t amplifier, which he claimed to sound identical to a well-known perfectionist tube amplifier. Following our extended auditioning of both amplifiers, John Atkinson and I had declared that they didn't sound the same, so Carver insisted that we prove we were hearing differences, a demand which JA and I felt to be completely unwarranted and unreasonable. (Doesn't he trust us?).

 

Both JA and I report in full on the results of these further tests on the Carver on p.117, but I was anticipating no problems whatsoever. After all, the differences I had heard at one point during my preliminary listening were great enough for me to describe them as "dramatic." Certainly, any "dramatic" differences would be immediately audible under any conditions of comparison.

 

Before the blind tests began, I had been listening "informally" for about an hour to both amps, but attributed the fact that I had not been hearing "dramatic" differences to the additions to the system of the lightweight wiring and the switch which Bob had rigged up to allow instant comparisons. I assumed that, when we went back to hard-wired conditions, those "dramatic" differences would re-emerge. To my surprise and chagrin, they didn't.

 

(...)

 

Read more at The Blind leading the Deaf | Stereophile.com

 

Humm

The Carver Challenge Page 4 | Stereophile.com

The Final Achievement

After this last bit of tweaking, where Bob was able to reinstate his 70dB null while driving a very difficult load, we now had what sounded like two absolutely identical amplifiers. No matter what speakers we used, every "difference" we thought we had isolated turned out to be there, in equal quantity, when we swapped amplifiers.

 

 

This time, the listening went on through the whole afternoon and much of the evening, until all of us were listened out. More leisurely listening, refreshed by a good night's sleep, failed to turn up anything. As far as we could determine, through careful comparisons and nit-picking criticisms, the two amplifiers were, in fact, sonically identical. It is a gross understatement to say that we were flabbergasted!

 

After the second day of listening to his final design, we threw in the towel and conceded Bob the bout. He packed up his equipment and limped triumphantly back to his Lynnwood, WA home base. (He had single-handedly hoisted the hefty reference amp onto a table at one point during the proceedings and injured his back.) The question remains whether or not we might have eventually picked up some miniscule but repeatedly audible difference between the amplifiers, had we been able to listen longer?

Somehow I doubt it. We had thrown some of the most revealing tests that we know of at both amps, and they came through identically. Even on the subliminal level—the level at which you gradually get the feeling that one amplifier is more "comfortable" than another—we failed to sense a difference between the two amps.

[Sal1950]

All of the following is solely in my own opinion:

 

It's still early days, and you may have taken this place as being like many other Internet forums, where trading insults is thought to be entertainment, calling other people's attitudes, opinions, or positions "ridiculous" is treated as amusing, and the loudest, most outrageous person wins in the end.

 

I like to think we're not quite so outrageously amusing and entertaining, but instead lean a little bit toward friendly, and when we're at our best, educational and helpful. Hope you'll contribute to that kind of community.

 

Really Jud,

Your point the finger at me.

All I've done is try to express a different opinion on the better ways to progress the SOTA in sound reproduction. There are large numbers of people who believe in a more objective approach, does the voicing of such beliefs scare you so as to stoop unjustly accuse me? I have called no one names, nor insulted anyone personally. Not something I can say for some who disagree with my legitimate opinion.

[christopher3393]

Really Jud,

Your point the finger at me.

All I've done is try to express a different opinion on the better ways to progress the SOTA in sound reproduction. There are large numbers of people who believe in a more objective approach, does the voicing of such beliefs scare you so as to stoop unjustly accuse me? I have called no one names, nor insulted anyone personally. Not something I can say for some who disagree with my legitimate opinion.

 

Well, I think there is much in this statement that is quite reasonably open to question. You use the tactic of shifting to a general insulting statement directed toward subjectively-oriented audiophiles, some of whom are the very people you are presently interacting with. Many here value and rely on both objective and subjective approaches, but you seem to only acknowledge the validity of some purely objective approach, and so seem to lump all others together as foolish subjectivists, meriting your laughter and ridicule. This is insulting, I think. You assume a superior position and an intent to show us the errors of our ways and lead us to the light, which is highly presumptuous. And you do very little to establish your own credibility. You denigrate others responses, some of which were in good faith and indicated some tolerance and patience that your responses do not return in kind.

 

In my opinion, Jud is being quite fair. I think you have earned the responses you have received.

[plissken]

Suitably high quality amplification and speakers, optimised room acoustics, highest quality source material ,( NOT via flawed USB either) and optimised seating for all participants, (no big heads/upper bodies in the way) and to be scientifically valid, numerous repeats without generating listener fatigue which will result in false negatives.

 

But sighted evaluations require none of that.....

[plissken]

There is solid peer-reviewed scientific literature supporting what you say here.

 

I keep seeing this literature referred too but never any links. AES? JAMA?

[plissken]

Sal, do some reading in scholarly sources about echoic memory, particularly:

 

- How long it lasts.

 

- What is remembered.

 

- Masking of recall of earlier sound by later sound.

 

Now ask yourself, given that background science, what can be discerned from an ABX test.

 

Given that sighted testing doesn't improve echoic memory I have to ask how it is people can announce huge deltas of improvement after they have swapped components out.

[sandyk]

But sighted evaluations require none of that.....

 

It depends on what you are evaluating. If you are comparing various media players for example, you also need the highest quality amplification etc.

Sighted evaluations are only of use where there are glaringly obvious audible differences heard.

It's best not to know which item of equipment is being used at the time, and having all items powered continuously to avoid giving a hint, is the best way to perform them, in conjunction with the person making the changes at the rear of the equipment not announcing which item is currently in use.

[Jud]

Given that sighted testing doesn't improve echoic memory I have to ask how it is people can announce huge deltas of improvement after they have swapped components out.

 

You're correct, the difference is not sighted versus blinded. An A/B comparison won't reveal more just because it is sighted. And of course there are the additional problems of potential bias with sighted testing.