Controversy of ABX testing

[goldsdad]

"Oh really, what exactly to you mean by 'insufficient unbiased evidence in support of Michael Fremer or the great Randi'"

 

I meant evidence in support of their differing accounts of their disagreement, not evidence of their knowledge or experience of audio.

 

"I really don't care what you think, and will do my best to avoid saying anything further. Instead I will carry on listening happily to my most excellent Nordost speaker cables, thank you very much."

 

You care so little that you wrote four paragraphs, I see.

 

Yes, you go listen to your cables while I listen to music.

 

 

[wgscott]

24 bit gives you more dynamic range, hence less compression, hence potentially a more realistic reproduction is possible.

 

[wgscott]

Cleanup on Isle Six.

 

[Paul R]

I would say I always allow the possibility of error, but I am not so blind as to ignore the obvious. You honestly don't see that you are building up some fantastic set of objections, rather than even tentatively accepting the simplest explanation?

 

Which is of course, there is a difference people can hear. In speaker cables, in USB cables, in interconnects, in optical cables. The simplest explanation is usually the best explanation, you know.

 

Or, I could say that you are so convinced that it cannot be possible, you refuse to consider any other option.

 

I'm not slamming you Owen, but I am also unwilling to back down and am convinced by rather compelling evidence. If you can produce any evidence, or even any theories that explain why I and many others can hear a difference, other than the obnoxious assertion that I am imaging things, then I am willing to listen, consider, and even be convinced if your evidence is compelling enough.

 

I find it mildly perplexing you don't seem to get that just like any other activity practiced by humans, skill in audio listening is gained by being taught to listen, and practice practice practice.

 

Surely you are a better judge of your audio system today than you were a year ago?

Why deny that in any way?

 

As for the scientific community, question everything, but be aware what one so enthusiastically denies today is very likely to bite one squarely on the ass tomorrow.

 

I keep coming back to the FTL neutrinos as an example. Sure, it could be some kind of systemic error - but in three years with a very bright team trying their best to find that error, nobody has.

 

Surely, most people find it difficult to accept that someone actually is measuring particles zipping along faster than the speed of light in a vacuum, *I* bloodly well do! But nobody (with any sense) is denying it. Simply questioning it. And you can bet other people are out there trying to duplicate the results.

 

Which brings me to the point - go try and duplicate what we can so easily duplicate. If you can great. If you can't, we can move forward from there.

 

-Paul

 

 

[Paul R]

(snort) It is getting deep.

 

I believe you were unable to hear a difference when you changed out a USB cable, right Bill? But wasn't there some cable that you changed out that made a huge difference?

 

-Paul

 

 

[Jud]

...always happy to jump in. :-)

 

You are unwilling or unable to accept any possibility of an illusion when you perceive a difference.

 

Well, that viewpoint is certainly tempting when the personal "evidence" is oh so consistently convincing (i.e., in our personal listening we have consistently heard differences between USB cables). And I imagine it is also quite tempting when the personal "evidence" is consistently convincing in the other direction (i.e., in your personal listening you have never, ever heard differences between USB cables).

 

After all, the alternatives are for us to understand we've doled out good money because we're fooling ourselves, imagining things, falling for some marketing hype; and for you to understand that, having doled out good money, you don't have the ears and/or a sufficiently good system to hear differences that exist. Neither of those alternatives seems very attractive, does it?

 

So no wonder there's some resistance on both "sides" to admitting the possibility of being wrong. Then throw in all the various controversies about tests of the proposition at hand (Has a double blind test of the particular proposition been done? Was the experimental protocol conducive to finding actual differences if they exist? How should differences in acuity between subjects be treated, averaged over the whole or used to determine individual abilities only? Should subjective tests be completely discounted? Is there good test data locked away as proprietary?), and one has an extremely hard task to find proof sufficient to convince either side they've been idiots. :-)

 

Thus it is utterly un-shocking that such controversies may be the closest things we have to perpetual motion machines.

 

[koogook]

I think its funny that people say 'why are people begruding Audiophiles about their hobby', as if this hobby has something only to do with cables?

 

people who sell cables put time and effort into their company and have associated qualities with thier products, for me there is only one key difference for paying more money, and that is warranty. I would pay 10 times more then standard for a quality cable, better moulding and cable 24 carat this and 4 layer titanium shielding with wood if i get a life time warranty. As pulling out pushing back in can wear on the joints, dont get me started on usb and toshlink falling apart. Apart from this, speakers, room then system after source is for me the route of all evil.

 

I would say that Audiophiles are interested in hifi music reproduction, some like in all cases take it to an extreme because they have the inclination to do so, others dont.

To argue an opinion, knowing it is just that and then trying to defend it so passiontely.

 

The pursuit in computer audio is for me 2 fold,

Remote control in my hand all my music, anywhere in the house.

have the best sound at cheaper front end pricing.

 

My issues with sound are my own, but i would say this forget your cable unless its broken and make sure your front and back end of system are the best(not necessarily expensive).

 

we wont be using cables soon anyway

 

[goldsdad]

"You honestly don't see that you are building up some fantastic set of objections, rather than even tentatively accepting the simplest explanation?

 

Which is of course, there is a difference people can hear. In speaker cables, in USB cables, in interconnects, in optical cables. The simplest explanation is usually the best explanation, you know.

 

Or, I could say that you are so convinced that it cannot be possible, you refuse to consider any other option. :)"

 

Paul, I've said numerous times that I accept the possibility that I am not hearing something that exists and that other people such as yourself can hear. Where do I ever say that there absolutely cannot be an audible difference between USB cables?

 

I keep trying to make clear that I'm open to possibilities. I do favour the idea that an audible difference in non-faulty USB cables is illusory and any sonic difference that may exist (since cables cannot be fabricated within infinitely tight tolerance) is likely to be below a humanly detectable threshold. But let me reiterate that I am not convinced of the inability of absolutely everybody to be aware of a genuine difference in sonics arising from different correctly functioning USB cables.

 

"I find it mildly perplexing you don't seem to get that just like any other activity practiced by humans, skill in audio listening is gained by being taught to listen, and practice practice practice."

 

Yet another unfair assessment of my attitude, in my opinion. Where do you get the idea that I don't believe that there is skill, or learned ability, in audio listening? Of course listening ability improves by dedicated practice or simply from life experience.

 

"... FTL neutrinos ... nobody (with any sense) is denying it. Simply questioning it ... "

 

Yes!!! Don't you see that I am the one who is questioning while you are the one denying a possibility, namely imagination of audible differences between correctly functioning USB cables.

 

Going back a few paragraphs, you said:

 

"the obnoxious assertion that I am imaging things"

 

Please tell me that I'm wrong in believing that you've taken offense at my suggesting that there is the remotest possibility of you being mistaken about something.

 

 

[Guest Caner]

"As for the scientific community, question everything, but be aware what one so enthusiastically denies today is very likely to bite one squarely on the ass tomorrow"

 

Do not worry, SCIENCE will be the way to find likely miscalculations down the road to make it correct, nothing else, NOT perceptions, hallucinations or gut feeling.

 

Through history the converse happened almost as a rule though, scince has a way of debunking myths.

 

So be careful what you wish for...

 

As far as training ears I do not buy the argument of "training ears", not even a bit. There are many people who have a good music ear, can discriminate many details better than people with "trained ears", there are musicians in africa uneducated untrained but who might even hear the so claimed difference brought by the cables. So hearing is natural, it is there by default and there is a bell shaped curve on what people can hear probably explained by anatomy rather than "training" and age...

 

By the way I am totally neutral on the cables, since I do not know enough o make a judgement. And yes I am educated enough not to come to a conclusion only by my perception, what I feel today might be totally different tomorrow depending on tons of internal dynamics, stress etc.

 

For people who claim A/B/X testing do not work (and I understand that does not include you); they have the burden to come with a method that can suppot their claims. They will be responsible to find out an alternative since thay are the originator of claims.

 

And I will remain a skeptic (as always) till they do.

 

But that's me...

 

Caner

 

 

 

[Paul R]

LOL! No, I am not offended.

 

I just would not have thought you would be a person so intent on the negative side of things. Of course it is possible that all the differences I and other people here is imaginary, but it is very very unlikely.

 

The first thing I thought of was I was fooling myself into hearing differences, and I would not have dared say anything if I was not convinced otherwise.

 

I think to move forward, you have to get off the negative side and work with the possibility it is real. If it proves otherwise, so be it.

 

 

-Paul

 

 

[Gregor Samsa]

I have been accused of being a moderate on this issue, and suitably chastised for it. Nonetheless, I remain steadfast in my wishy-washyness, and I'll try to explain why.

 

Although I am on the audiophile side of wishy-washyness, I have great reverence for science, and have a research degree. However, this was in Social Science, which, for all the discussion of capacitance, inductance, etc., is what we're talking about here. Human experience and perception, which is subject to countless counfounds. The study that Paul links to shows only a subset of the potential issues that can muddy things. I don't think examples like which pill of identical appearance is most effective, or using HIV+ status as the dependent apply here.

 

ABX testing has the advantage of excellent control of independent variables, the only problem being that no one (I hope) actually listens to music that way, limiting its applicability to the real world. IMO (not always humble) the best you can say is that, if ABX testing does reveal a difference (does anyone have any good examples?), you're probably on to something. If not, there could be a number of explanations, only one of which is that audiophiles are daft.

 

Oh, and if folks think that goldsdad and wgscott are too rigid, go hang out on Hydrogen Audio or AVS forum for a while. They're positively wishy-washy (in the best sense of the word) by comparison.

 

[wakibaki]

I noticed this header back a ways: 'I don't buy this demand for proof thing.'

 

Now, 'irony' is an overworked word in this context, but it is truly ironic that this is the only aspect of the whole issue about which the writer is sceptical.

 

Anything that anyone says purely on the basis of relying only on their ears, he will buy into that, but when I say 'Where's your proof?' he doubts that it's necessary.

 

I wonder if he'd like to exchange that fat cow he's got for this bag of magic beans? You can tell they're magic because people with extraordinary hearing can hear them whispering to each other.

 

@Paul.Raulerson

 

You argue very convincingly Paul, for a sophist.

 

Stop prostituting your talents, because these arguments are not only about audio equipment and you will end up rudderless in life, without a star to steer by.

 

The real problem with all this subjective twaddle is that it undermines the ability of designers to design.

 

Hi-fi design is not art, it is not opinion, it is not style or fashion. It is engineering.

 

It is not philosopy, psychology, physiology or any combination of the foregoing.

 

It is numbers.

 

Not veils, soundstages, detail or texture.

 

When the numbers aren't with you, you lose.

 

None of you subjective guys have ever been able to consistently get the numbers to come out on your side. Never ever.

 

All us design engineers from Edison on down can get the numbers to come out on our side.

 

When the FTL neutrino business is sorted out, it'll be on the basis of numbers.

 

How do you imagine MP3 compression was designed? It was designed using numbers obtained by careful (blind) testing of the population at large.

 

Jealousy will get you nowhere.

 

Wishful thinking will get you nowhere.

 

Sticking your head in the sand will get you nowhere.

 

You'll never have a single scientific discovery to your credit until you sort out your attitudes. All you can lay claim to is contributing to the fog of incomprehension that is new-age anti-intellectualism.

 

[Larry B]

I'm just your garden variety neuropharmacologist with only a dozen or so patents to my name, not nearly as smart as many here. That said, I would like to offer my observations.

 

1. The one invariant is these discussions is that neither side ever convinces even a single member of the opposing side.

 

2. To the best of my knowledge, no "objectivist" has ever snuck into the home of a "subjectivist" in the middle of the night, and removed the subjectivist's favorite interconnect/speaker wires/magic stone/whatever. Despite this, the subjectivists are hell-bent on convincing the objectivists of the folly of their ways, and thereby converting them.

 

3. Again to the best of my knowledge, no subjectivist has ever forced an objectivist to purchase special cables/stones/whatever. Despite this, objectivists are hell-bent on convincing the subjectivists of the folly of their ways, and thereby converting them.

 

4. Subjectivists invariably maintain that they hear differences between product x and product y, and appear to think that the objectivists are telling them that they have not heard said differences. This however is not what the objectivists are saying (or at least, should not be saying). Rather, the objectivists are asking whether the differences are sonic or instead, are due to any of the myriad other factors that influence human perception. The only way we know to distinguish between these possibilities is to hide the identity of the components during the comparative listening session(s).

 

5. The notion that we could be so easily fooled is apparently terrifying to many. I suspect this is because audiophiles are control freaks, and cannot deal with the fact that there are many aspects of our perception that are beyond our control. (As an aside, I wonder why audiophiles don't recoil in horror at so-called optical illuions. Or perhaps they do!)

 

6. Thankfully, bandwidth is cheap, so the same arguments can be repeated at regular intervals, ad infinitum.

 

Larry

 

P.S. Alan: The so-called placebo effect is in most cases far less robust than had originally been believed. If you're interested, I can send you links to some relevant publications.

 

[Paul R]

Wow - some job there of labeling people with an incredibly broad brush. I'm not sure anyone here fits those categories.

 

The easiest way to look at it is not a never ending pub argument, but a slowly evolving understanding.

 

Yes, there are some "audiophiles" who are simply crazy and fooling themselves.

There are also some "objectivists" (as you call them) who are simply crazy and fooling themselves.

 

There are also a few people who are more than crazy, and have a downright evil streak in them. Add in a couple con-artists, and it is easy to see why people start demanding more proof than they can uncover for themselves.

 

I will state again that it is silly - no, plain stupid - to discount the evidence of your ears the way some people do. And it is plain stupid to not spend the effort to uncover some level of objective proof of what your ears tell you, again as some people do.

 

Extremes are tricky places to be. One is best advised to avoid them.

 

-Paul

 

 

[Larry B]

"Extremes are tricky places to be. One is best advised to avoid them"

 

That seems ike a pretty extreme position.

 

 

"I will state again that it is silly - no, plain stupid - to discount the evidence of your ears the way some people do."

 

I agree, and never suggested otherwise. Rather, my view is that the question should be to determine to what our ears (by which I, and I suspect, you, mean the entire human auditory system) are responding; actual sonic differences on the one hand, or something other than sonics.

 

"And it is plain stupid to not spend the effort to uncover some level of objective proof of what your ears tell you, again as some people do."

 

 

On that point, I do not completely agree. Listening to music can be purely for pleasure, and some folks are quite content to not care about the things we're discussing here. I don't think that makes them stupid; just different.

 

Regards,

 

Larry

 

 

 

 

 

[Paul R]

@Larry

 

On that point, I do not completely agree. Listening to music can be purely for pleasure,

 

I agree with that as far as it goes. How about if I qualify that?

 

"And it is plain stupid to not spend the effort to uncover some level of objective proof of what your ears tell you, again as some people do, in particular if they intend to spend money on expensive gear"

 

That's actually what I assumed but failed to clearly state.

 

-Paul

 

 

[Paul R]

@ wakibaki

 

This is the most amusing post I have seen in quite a while. Indeed, you seem to have really put some thought into it. Congratulations.

 

I would point out that I am not as smart as other people here, not by a long shot. I don't do so bad though. (grin)

 

I disagree with you though - if all you see is numbers, you will never grasp the soul of design, and how things fit correctly into their environment. Might build bridges for TexDOT, but you will never be able to design the really striking bridges that make people gasp, and realize what has been built is more than just the sum of the numbers.

 

Of course, I do agree one must be able to do the numbers, and do them correctly as well. But it is far from all there is.

 

The great audio gear of our time all have numbers that add up, but the final touch, the part that makes the gear great and not merely competent, is done with ears and not calculators.

 

-Paul

 

P.S. - Please do not take "you" as in you personally, just in a general sense.

 

[Larry B]

Better. By and large I agree, with the caveat (and I really am not trying to be argumentative) that some folks seem to truly not care. One rule I try to live by is to not tell others how to spend their money unless asked - or unless they are trying to borrow the money from me!

 

Larry

 

[Larry B]

Barrows, you said "I do find it a sorry commentary on the state of human beings that many on this thread cannot believe in their own experiences."

 

The issue is not of believing in one's own experience, but of understanding whether those experience were the result of powerful external factors.

 

Please look at this: http://www.michaelbach.de/ot/mot_feet_lin/index.html

 

We all have the "experience" of seeing the yellow and black bars undergoing saltutory motion. A sorry commentary would be if individuals accept that at face value, instead of delving deeper to understand why we perceive it in that way.

 

So it is - or should be, IMO - with audio.

 

And for the record (no pun intended), I have no interest in telling people what they like or don't like. I do however have an interest in uncovering what factors contribute to our perceptions and observations, especially when those factors are quite different from the supposed explanation. That is simply seeking truth; nothing more and nothing less.

 

Larry

 

[4est]

Larry,

 

I do not see how your optical illusion pertains to this. I believe it is possible to trick me like that in a contrived situation, but not daily in my listening room.

 

[Larry B]

Forrest, I'll begin by saying that I have no interest in tricking anyone. While the analogy was not perfect (analogies never are), my point is that our observations and perceptions can be and often are influenced by many factors; as a result, we can easily reach incorrect conclusions.

 

"I believe it is possible to trick me like that in a contrived situation, but not daily in my listening room."

 

That may well be the case but without careful testing, it is simply conjecture. Again, I am interested in the truth, whatever it may be.

 

And for the record, I too "trust my senses" when choosing audio gear, but I do so with the full knowledge that what I "hear" may be the result of external factors, rather than the actual sonic properties of the gear itself.

 

Larry

 

[Jud]

Forrest, I tend to agree with you.

 

I'm not aware of an auditory illusion that would result in my wife, in blind testing, consistently preferring the sound of Cable 1 over Cable 2 if there is actually no audible difference. Making distinctions *in*consistently, yes (sometimes Cable 1, sometimes Cable 2). But she's been consistent.

 

I could subconsciously be giving her cues as to which one *I* prefer (consistently, and as I've noted previously in this thread, not at all in order of expense), though I guess this would not, technically speaking, be an auditory illusion on her part.

 

For Larry and everyone else interested in illusions, this is one of the better ones I've seen:

 

http://illusioncontest.neuralcorrelate.com/2010/impossible-motion-magnet-like-slopes/

 

[4est]

"...what I "hear" may be the result of external factors, rather than the actual sonic properties of the gear itself."

 

Fair enough, I buy that. I'd have to mostly agree, maybe I am just more confident of my odds or my ability to take myself out of the equation- knowing I'll get there eventually. After all, it is fooling one's self. Where's there to hide...

 

[Larry B]

If your wife distinguished the two cables in a blind test (assuming the test was properly blinded, the resuts reached statistical signficance, etc. etc.) then there is indeed a sonic difference between the two. Lest I be misunderstood, I have never stated that such differences do not exist. (Others make that claim, but not me.) I do however advocate that one be careful in reaching conclusions when careful, unbiased testing has not been performed.

 

Larry

 

[Larry B]

This passage from the Sean Olive article cited above, sums it up nicely:

 

In summary, the sighted and blind loudspeaker listening tests in this study produced significantly different sound quality ratings. The psychological biases in the sighted tests were sufficiently strong that listeners were largely unresponsive to real changes in sound quality caused by acoustical interactions between the loudspeaker, its position in the room, and the program material. In other words, if you want to obtain an accurate and reliable measure of how the audio product truly sounds, the listening test must be done blind. It’s time the audio industry grow up and acknowledge this fact, if it wants to retain the trust and respect of consumers. It may already be too late according to Stereophile magazine founder, Gordon Holt, who lamented in a recent interview:

 

 

“Audio as a hobby is dying, largely by its own hand. As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me..”