Controversy of ABX testing

[cfmsp]

 

"Cases are expensive"

 

and

 

"...especially with passive items such as a bit of wire."

 

 

Yes, even cables needs 'cases', esp. if we consider the dielectric as part of the 'case' and not part of the 'bit of wire'. ;0

 

clay

 

 

PS, lampcord anyone?

 

http://en.wikipedia.org/wiki/Dielectric

 

 

 

[cfmsp]

duplicate post deleted.

 

 

 

 

 

 

[cfmsp]

 

barrows says

 

"...but actual testing should generally be done by third parties: reviewers, and end users. I admit it can be hard to trust "testing" done by some manufacturers..."

 

Agreed, can you imagine the howling from the AB/X test brigade if a manufacturer provided a test result that differed from their own beliefs?

 

Although it is encouraging that Nordost / Chord are doing a type of industry collaborative testing.

 

I wonder if their results will prevent less worthy cable manufacturers from simply saying "what he said", rather than having to prove that their own cables are covered by the Nordost/Chord study.

 

Should be interesting times. Skeptics might want to download the website marketing of suspect cable companies, as the story of some is likely to change overnight at some point. ;0

 

 

 

 

clay

 

 

 

[cfmsp]

 

Sean,

 

Thanks for your reply. Nothing that needs to be said in response, just wanted to thank you. I respect your position.

 

clay

 

 

[Gregor Samsa]

No, they didn't change the power supply, or anything else. Modwright changes the power supply. Ayre changes the power supply. Even OPPO changes the power supply on the SE version. To quote Audioholics, who broke the story: "it's not just using the same parts - they actually stuck the player inside - chassis and all" - and they opened up the box to prove it.

 

If this sounds harsh, well, sometimes the truth is like that. Maybe it will spur Harman to get back to designing and building their own terrific gear. I LOVE my 10 year old Proceed amp. It sounds much better than the very good NAD it replaced. I don't know and I don't care if they measure any differently.

 

[Gregor Samsa]

If I was held responsible for all the decision making where I work, I'd have to resign immediately. But when you append your title to your posts, making it clear you do more than work in the mail room, you do stick your head above the parapet a bit.

 

No personal issues here. I'm a satisfied past customer of Harman. You still make some great stuff. I've heard the ML systems you put in Lexus (Lexi?). Sound terrific for a car.

 

[tonmeister86]

Gregor,

 

I hear you. I include my title/association because a few people here accused me of being dishonest for not including it when I express my opinions in the past. It seems I lose either way -

 

A few years ago we had a cable inventor visit us who claimed he had the greatest sounding cable, and wanted Harman to invest in his invention. He had somehow gotten through to the Chairman of the company so we ended up having to take him seriously and do due diligence regarding the merits of his invention. The inventor told me he had already peddled his cable to all the big name cable companies, and one of the major cable company Presidents told him how great it sounded.

 

I asked him if he or any of the companies he had visited had done any electrical measurements or controlled A/B competitive benchmarking listening tests on his cables. To my surprise, neither he nor the cable companies he visited had done this. So we measured the LCR and frequency response and did A/B listening tests comparing his exotic able to various standard cables we have in our labs. The results from both objective/subjective tests were than no one including him could reliably hear any difference between the different cables. He was baffled that he could clearly hear the differences when we told him which cable was his - but those differences went away once the test was blind.

 

I told him the mind and power of suggestion can play tricks on your perception. In blind taste tests, people preferred Pepsi until they saw the packaging, at which point their preferences switched to Coke. Of course, these cable tests were not about preference where the perceived differences are well above the detection threshold of audibility but rather about whether an audible difference can be reliably detected at all.

 

We told him his cables were well designed in the sense that they had neglible measurable effect on the electrical transfer function between the CD player/amplifier/loudspeaker, much like any other well-designed cables that can be found for the cost of a case of beer. This feedback seemed to greatly disappoint him since he had envisioned something a little more grandeur (note: some exotic cables actually do change the electrical transfer function between your source/amplifier/speaker, making them essentially tonal equalizers - no thank you!)

 

He thanked us for our time, & expertise, and for giving him the first measurements of his cable that he had ever seen. We never heard back from him. Luckily I don't get calls from the new Chairman to do cable invention vetting.

 

 

 

 

 

[Part-Time Audiophile]

And here I thought you were going to say:

 

"And he went on to found Pear Cable ...."

 

[barrows]

Here is a link to some info on the new measurement technology being developed by Nordost and Vertex AQ (I was experiencing a memory error when I previously stated that The Chord Company was involved).

I have not yet read the pdf file presented here, but I suspect it is an introduction to the new measurement technique. BTW, this new measurement technique appears to be equally applicable to electronics.

 

http://www.nordost.com/downloads.asp?offset=14

 

 

 

[BobH]

The indefatigable Mr. Roy Gregory in full flow!

 

I was sort of being swept along by it all up to this point:

 

So, adding the power cord produced a signi?cant reduction in overall error: In fact, that reduction in misplaced samples amounts to 36%. Repeating the process with the Kinibalu support reduces the error by a further 15%, although bear in mind that this result is cumulative – the Kinibalu had less to work on because the power cord had already improved things.

 

Now, being no kind of expert, ths caused much turmoil amongst my aged brain cells! He seems to be implying that untweaked systems are going to be producing lots of misplaced samples. These errant little devils can then be fixed, a bit, by changing to a better power lead OR sticking the black box onto a posh rack. Doing both will be wicked!

 

Now, here we seem to have performance degredation caused by stock power leads AND performance degredation caused by inadequate vibration isolation/absorption BOTH being measured, at the same time, by the same piece of kit!! The implication being that, whatever is 'wrong' with your system, it all comes down to measurable errors in the Time Domain.

 

I don't mind admitting that that concept goes straight over my head!

 

[Shadorne]

They are comparing stuff in the time domain. Of course nobody uses this "new" technique. It is nonsense. We hear frequencies. The time domain shifts (between input signal and the signal reaching your ears) in an ordinary room and from the distance between your ears and from tilting your head 1 mm or shifting a speaker by 1 mm are of course ABSOLUTELY ENORMOUS (were you to subtract the two signals).

 

Thank goodness for evolution. Our brain and hearing systems have evolved to handle these complexities and which is why we do not hear these kind of differences nor are we confused by all this time domain complexity - if we were then we would have become extinct long ago - like this research will be very soon. Piffle.

 

[Part-Time Audiophile]

Or is that Double-Piffle? Not to be confused with (or skipped in favor of) Triple-Dog-Piffle, the ultimate in schoolyard dismissivity.

 

Sorry, Shad. Couldn't help it. It was a biological imperative. Blame Darwin. I do. Often.

 

Anyway, I (for one) think that this is an interesting approach, and something I intuited independently and then opined on some time back. Which is probably why the approach is so interesting. ;-)

 

That said, Shad is right. The sketch (because it's barely a methodology) has at least two fundamental issues to overcome.

 

First, what does a given result actually mean? The article fails to mention (or if it did, it wasn't in boldface, and therefore I failed to read it) that 'measurability' does not entail 'audibility', though I would argue that the reverse is a logical truth. Hence, the Threshold Problem bears here. We're going to need some kind of semantic scale here.

 

Second, I have a problem with the baseline. It appears that they are observing the file via a digital readout pre-playback and comparing that to a playback capture (but then, I am barely literate, so I might have missed the nuances here). My problem is that measuring before and after doesn't really tell you which is more "accurate", though, only that there is a deviation between the two measurements. Why? Well, what's the "correct" baseline? The recorded file or "Real Music"? At a blush, it seems impossible to standardize a captured live sample and rectify it with a playback measurement. There are too many variables associated with the capture chain let alone the playback chain. But a lack of such a baseline means that you have nothing to compare either to -- which is a problem in that it could be that the deviations of a given playback chain actually will better approach what the "real music" sounded like (unlikely though that might be).

 

All in all, I think this is a very clever approach. Not sure if the flaws are fatal. Whatever. It is a darn sight better than the haphazard guesswork we have now.

 

[BobH]

Scot said "Whatever. It is a darn sight better than the haphazard guesswork we have now."

 

No, says I!! What happens then? Some ****** comes along and says 'Here's my wonderous measurements gizmo. It reveals all the secrets of The Dark World and lays bare the truth'. (That Nordost products were the best, all along.)

 

That means that we would all have to forego the cosy world we have now - wherein we, (as individuals), only ever buy absolutely brilliant stuff wot is miles better than the old absolutely brilliant stuff we bought before. To be replaced by the sure certainty that what we just bought is OK, but nowhere nears as good as what we can't ever, ever, afford?

 

Give me BS fields of marketing guff and haphazard guesswork any day of the week! As Jack Nicholson nearly said - "The truth? I can't handle the truth"!!

 

[Shadorne]

I don't need an expert like Sean Olive to know that most speaker designers aim to be time accurate/coherent to within about 2 msecs overall from 20 Hz to 20 KHz. They also aim to maintain an overall smooth phase response with frequency but inevitably some frequencies are delayed compared to others - it is just the nature of the beast - it is what you get with voice coils and crossovers.

 

Now we have cable marketeers trying to compare signals in the time domain and trying to interpret differences like tea leaves and claiming advances in science with audible benefits to boot! This is worse than the priests at the UK Met Office predicting a mild winter after barbecue summer based on some computer model and an irrational fear of plant food (CO2)- it ain't science it is sheer voodoo.

 

FWIW: I saw two crows fly by my bedroom window this morning from East to West exactly at dawn - it must be a sign! Perhaps I'll write a book about it or a technical "peer-reviewed" paper - probably it has something to do with the climate or my health and life expectancy or whatever. In any case, I'll be sure to have my witch doctor stop by and have him remove the bad karma on my audio system and bless the cables with his juju charms ;-)

 

Of course, this would all be simply amusing if it were not so sad. The state of scientific research today is at an all time low - wild unsubstantiated claims like "cold fusion", new wonder drugs, climate thermageddon, and amazing properties of power cords and other bits of wire are just par for the course.

 

[TimDH]

I am a climate scientist, but I've refrained so far from the potshots that have nothing to do with audio. However, everyone should know that after all the ballyhoo from skeptics, 2009 turned out to be the 2nd warmest year on record (tied with one other year).

 

 

 

[Shadorne]

So what is your degree? I took physics, including atmospheric physics. Nothing against your discipline - I found radiative physics to be fascinating. I understand enough to be totally mystified by the hyperbole around climate science - always talk of ecological disaster but no observational evidence to support any of the claims being attributed to human emission of CO2 (soccer mums with SUV's).

 

So not to pick a bone with a laudable area of research - my point was, that in science, just like audio, we lately seem to suffer from a near universal willingness to jump to unsubstantiated conclusions and wild claims. Nobody gave a hoot about cables 50 years ago and 30 years ago we became worried about the next ice age (which will surely come one day - whatever we do)

 

[Audio_ELF]

Reverse the polarity of the neutron flow!

 

Eloise

 

(Sorry - always wanted to say that - and it always woked for the Doctor)

 

[wgscott]

I am but a mere physical chemist, but the telltale sign is fairly straightforward: carbon isotope ratios from CO2 produced from fossil fuels are different from metabolic respiratory sources. From what I have seen, the current effects are already substantially worse than what was predicted, especially coral reef habitats and tundra thawing, not to mention what is happening at the poles.

 

The difference is with cable voodoo, no one will supply evidence to back their claims.

 

[The Computer Audiophile]

Climate similarities to cable?

 

 

The globe is getting warmer and this is measurable.

 

Cables are different and this is measurable.

 

 

Other than that there will never be agreement on causes or if it really matters.

 

It is really great to see some very smart and educated readers commenting.

 

[Shadorne]

I suggest that those of you who have strong beliefs about the significant audible benefits of high priced cables or power cords would benefit from watching this lecture:

 

 

 

 

 

 

 

I think the important point here is that human perception and evaluating equipment in any ordinary room is fraught with issues.

 

And yes we can detect minuscule differences with instrumentation between one cable and another just like we can detect a tiny tenth of a degree rise in global temperatures at an airport weather station - however, the mistake, often made by well meaning people, is to make enormous leaps of faith towards the assumption that this simply must somehow be meaningful overall, audible (in the case of the audiophile) or that we need to stop SUV driving soccer mums from causing climate thermageddon.

 

It is all rather sad that a lot of modern science has become bogged down in voodoo. A kind of dumbing down has occurred. People will believe anything these days before picking up a text book and learning the physics.

 

[souptin]

... reminds me of something from "The Right Stuff" - I think the book rather than the movie, although both are highly recommended in the souptin household, anyway. Quick summary: A British film called "The Sound Barrier" has a British pilot breaking the barrier by "reversing the controls" at a critical moment, but Chuck Yaeger, the American who was in reality the person who broke the barrier was asked to comment on the movie and said (best to pretend it's Sam Shepard saying this) "A pilot who reversed the controls is likely to end up dead". More or less, anyway. Just to confuse matters further, "control reversal" is, or so google tells me, a legitimate description of aircraft behaviour in high speed flight. Not particularly relevant to this topic, but maybe an explanation of why reversing something is such a popular solution in scifi-ish television.

 

On the youtube link posted by Shardone, I agree it's a very interesting presentation and gives good grounds to question some of the extremes of audiophile journalism - eg the infamous dance-able cables and the veil that is almost always lifted. On the other hand, I think it does also illustrate why long term listening (over days, weeks or months) is preferred over the quick switching method: Namely that if you hear a particular detail one one setting, you tend to hear the same details on subsequent tests even if you are radically changing the system setup particularly over a short time scale. Obviously a blind ABX style test could in theory be left set up over an extended period of time, but I seriously doubt if this is done very often. If I want a review of the latest bit of gear that takes my fancy, then I want Chris Connaker to review it right now, dammit!

 

[Part-Time Audiophile]

I'd take this a step farther -- most audio reviews (present company excepted, of course) are utter rubbish.

 

Great video, BTW. Thanks for re-posting that.

 

[wgscott]

It is all rather sad that a lot of modern science has become bogged down in voodoo. A kind of dumbing down has occurred. People will believe anything these days before picking up a text book and learning the physics.

 

A bit of a sweeping generalization.

 

The difference between cable voodoo and modern science is that the latter takes as a pre-condition of being considered scientific the ability to state, unambiguously, under what conditions you would be willing to accept that your hypothesis is wrong. So, to take your climate change example, if it can be shown that there is no statistically significant positive correlation between fossil fuel-derived CO2 emissions and temperature increases, then most if not all proponents of that hypothesis would consider it to be experimentally refuted. Can you state under what conditions you would accept being wrong? If not, you have to ask yourself whose hypothesis is closer to that of the cable voodooers.

 

[BobH]

...if I state, unequivocally, that under all known conditions I am not prepared to accept being wrong? :~)

 

[Shadorne]

WgScott,

 

This appeal to correlation is precisely one of the false assumptions underpinning much of modern voodoo science: the jump to the conclusion that correlation must necessarily imply causation.

 

This is one of the points demonstrated in the lecture - they show how audible improvements such as "better speech intelligibility" may be achieved when one knows what to expect. Just like the amplifier "trick" they mention. In this case, the "knowledge" of what one is hearing (a great amplifier) governs the perception of sound quality. If you believe correlation is causation then you jump to the conclusion that it is the amplifier that causes the audible difference.

 

Anyway - it will be far more interesting to discuss the "Audio Myths" lecture rather than to get sidetracked on the poor quality of science both in media and journals (even though the issues are related). Although, I suspect most people won't bother to watch the lecture as it will challenge their beliefs.