Ars prepares to put “audiophile” Ethernet cables to the test in Las Vegas

[Jud]

Regret that I won't be able to make it to Axpona. I thought it might possibly be in New York as it was a few years ago and I could have gotten there, but I see it's in Chicago.

 

Music: Just a couple of general thoughts, no specific tracks -

 

Gillian Welch, The Harrow and the Harvest

 

Ottmar Liebert, Dune

 

Brian Bromberg, Wood

 

Any of several recordings by Mark Knopfler/Dire Straits

 

Any of several recordings by Morphine

[firedog]

I read and participated in the thread over at the ARS site. The comments there were pretty interesting: first, I was the only one who pointed out that taking the AQ cable to be tested by a competing commercial firm (BJC) is about as ethically bogus and non-scientific as you can get.

 

Second, as the summing of the cable testing, it was repeatedly stated that the Vegas test showed there were no perceptible differences between the two cables. When in fact the article about the test itself clearly showed that the test was basically invalid - at least as far as drawing any general conclusions about cables from it. But many of the posters understood the test results as proving there are no differences between cables.

 

Just goes to show that some of the people who claim to be scientific and objective are no different than some audiophiles: any "test" that backs them up is a good one, and any test that doesn't is rejected.

[mmerrill99]

We are not looking for a consistent ability to do anything. That's not what this particular testing is about. You just went to a foregone conclusion.

 

The simple question is based on what I have seen Journalist/Enthusiasts claim: Can a trained listener with unimpaired range and sensitivity that follow what is considered normal hearing standards able to take well mastered music, on an accepted high resolution system with generally acceptable speakers, amp, DAC, source materials and room treatments statistically pick out a difference in a CAT6A cable from say BJC and a $340 AudioQuest Vodka Ethernet cable? 2nd follow on question is if a meaningful series of choices is made. If difference has indeed been indicated then is there a preference? 3rd follow on question is if there is a preference does it track to the more or lesser expensive cable.

 

What I am getting at is with a well engineered system with brand names of more than respectable gear can Audiophiles who make some claim as to superior hearing deduction skills.

 

How many on said system can take material that they may or may not be familiar with and use that to discern a difference.

 

If the difference is night and day or easily discernible, goal posts not set by the test administrator mind you, then that's all that is needed.

 

If the end result is that there is no significance then in the constraints of various high resolution, well recorded tracks, on equipment that is considered well regarded by the community in general then that is an answer in and of itself.

 

So what would a list of 4 tracks look like?

 

And there is the rub, Paul - he's not looking to do anything of worth other than disprove people who claim they are "trained listeners" & state they can hear "night & day" differences - that's what his testing boils down to.

 

The question of whether there may actually be an audible difference to be found in a properly conducted test is immaterial. Instead of getting actually trained listeners who have proven to be discerning of small differences he takes the statements of individuals & sets out to prove their statements wrong.

 

It's typical of the motivation which then guides the experimental design, full of experimenter bias & guaranteed to return a null result. Completely & utterly worthless on every level.

[Paul R]

We are not looking for a consistent ability to do anything. That's not what this particular testing is about. You just went to a foregone conclusion.

 

if you don't get consistent results, then the test is flawed. That doesn't mean all the results have to be the same, just that you consistently get an accurate result for what you are testing.

 

 

The simple question is based on what I have seen Journalist/Enthusiasts claim: Can a trained listener with unimpaired range and sensitivity that follow what is considered normal hearing standards able to take well mastered music, on an accepted high resolution system with generally acceptable speakers, amp, DAC, source materials and room treatments statistically pick out a difference in a CAT6A cable from say BJC and a $340 AudioQuest Vodka Ethernet cable?

 

Wrong question I would say. First, stats can lie. They are more subject to bias expectation than audio is.

 

Second, since the ethernet cables themselves cannot and do not have any intrinsic sound, what you are really asking is more along the lines of "in this system with this music, can test participants acurrately hear a difference in the output audio if the ethernet cable, and only the ethernet cable is changed.

 

Where's your control? What about if there is no ethernet cable? What about if the cable is not changed? Hows does your test plan take into account these factors and may many more. Each factor of course, is capable of biasing the answer.

 

 

2nd follow on question is if a meaningful series of choices is made. If difference has indeed been indicated then is there a preference? 3rd follow on question is if there is a preference does it track to the more or lesser expensive cable.

 

This is purely perceptual, and even more difficult to ascertain than the first question.

 

What I am getting at is with a well engineered system with brand names of more than respectable gear can Audiophiles who make some claim as to superior hearing deduction skills.

 

How many on said system can take material that they may or may not be familiar with and use that to discern a difference.

 

If the difference is night and day or easily discernible, goal posts not set by the test administrator mind you, then that's all that is needed.

 

If the end result is that there is no significance then in the constraints of various high resolution, well recorded tracks, on equipment that is considered well regarded by the community in general then that is an answer in and of itself.

 

So what would a list of 4 tracks look like?

 

You do realize that to any rational observer, you have changed the basis of what you are testing several times? Are you testing perception or audio hardware? The idea is a little too loosey goosey for me at this point.

 

-Paul

[plissken]

I read and participated in the thread over at the ARS site. The comments there were pretty interesting: first, I was the only one who pointed out that taking the AQ cable to be tested by a competing commercial firm (BJC) is about as ethically bogus and non-scientific as you can get.

 

Then AQ should provide their own slip of paper. I think there would be a lot of egg on face if 20 of their cables were to be tested. They certainly aren't going to pass the CAT7 spec when ratified. The cable that was tested barely made 6A. And it could fail if retested.

 

BJC as far as I'm concerned is of un-assailable reputation. I have zero reason or hesitation to take Kurt and Co and the testing at face value.

 

I'm a BJC customer through and through. The fact that every Ethernet cable that ships from them includes a printed certification sheet and AQ doesn't.... You can reach your own conclusions.

[plissken]

if you don't get consistent results, then the test is flawed. That doesn't mean all the results have to be the same, just that you consistently get an accurate result for what you are testing.

 

Who said there won't be a consistent result? I didn't. We are looking for data. What ever that data would be. We are postulating a question. It doesn't mean the testing results are going to be consistent or is going to conform to the question.

 

Second, since the ethernet cables themselves cannot and do not have any intrinsic sound, what you are really asking is more along the lines of "in this system with this music, can test participants acurrately hear a difference in the output audio if the ethernet cable, and only the ethernet cable is changed.

 

Correct...

 

Where's your control? What about if there is no ethernet cable? What about if the cable is not changed? Hows does your test plan take into account these factors and may many more. Each factor of course, is capable of biasing the answer.

 

In a blind taste test of Chocolate vs Vanilla and the tasting experts where is the control? The test parameters are based on claims. I'm not making the claims. I'm designing a test around those claims and also around the objections that are normally going to thrown up. I'm mitigating the variables of questionable electronics.

 

I'm not attempting to mitigate any persons bias. And it doesn't matter what bias that is (other than against blind testing). What is going to be done how ever is their bias will be eliminated by the testing process. If you do not do well under blinded testing I have one suggestion for you: Don't participate.

 

If you can not, will not, trust your ears then how could you ask anyone else too trust yours.

 

You do realize that to any rational observer, you have changed the basis of what you are testing several times? Are you testing perception or audio hardware? The idea is a little too loosey goosey for me at this point.

 

-Paul

 

If you really believed what you just posted there would have been more from you on the matter. I haven't changed the basis of evaluating 'Easily Discernable', 'Readily Apparent', or 'Night and Day' claims.

 

I can test those with a $160 boom box or $10,000K worth of equipment. Which do you think is going to be more representative of the people that make those claims and the discussion boards that people like you and I congregate at?

[Jud]

 

I'm a BJC customer through and through. The fact that every Ethernet cable that ships from them includes a printed certification sheet and AQ doesn't.... You can reach your own conclusions.

 

One conclusion (which doesn't negate others, such as BJC making a fine product and wanting customers to be sure of that as well) is that this is an aspect of BJC's overall marketing which would very likely be completely lost on AQ's customers.

[esldude]

Ah, you mean an subconcious registering of a difference.

What percentage of blind tests would you say this has been demonstrated - a subconscious differentiation was demonstrated (not talking about fMRI results) compared to the number of null results that are caused by people not knowing what to focus on & shifting their attention to different aspects of the sound trying to search for a "conscious" difference?

 

Only if you can design a blind test where the tester is unaware of what's being tested or better still unaware that it is a test, is there any likelihood of this being a successful strategy for testing.

 

This also comes down to the motivation of the participants - some people will just admit to themselves that they don't hear a difference (after a couple of tries) & purposely stop listening - just hitting random choices. How do you separate out these results from the subconscious listeners?

 

BZZZZT! Wrong again.

 

I have no idea of the percentages. I have read of tests where what I described was the case. I have taken or seen friends do them and experienced such results where you think you aren't hearing a difference, and results show you are. Re-testing lends credence to it being a solid result and again results are over the 95% confidence level. It is of course exactly different than what most audiophiles expect. They expect to hear differences consciously that still test at near random levels. Not to sub-consciously get good results while saying, " I don't know, sounds the same to me".

 

In some cases test takers know what is being tested for and to focus on and in others they didn't. As for someone giving up and just hitting randomly there is no way to control. It appears even when they think they are hitting choices randomly they often aren't. In any case, results either point to a likely difference or fail to indicate one. As the tests are to see if a difference is perceived that is all you can do with them in that context.

[plissken]

One conclusion (which doesn't negate others, such as BJC making a fine product and wanting customers to be sure of that as well) is that this is an aspect of BJC's overall marketing which would very likely be completely lost on AQ's customers.

 

So AQ customers don't want to know if their Ethernet cables possibly are so in the margin that they may fail certification?

 

What are you saying about the average AQ customer Jud?

[mmerrill99]

BZZZZT! Wrong again, strike two.

 

I have no idea of the percentages. I have read of tests where what I described was the case. I have taken or seen friends do them and experienced such results where you think you aren't hearing a difference, and results show you are.

So you are asking us to believe your anecdotal evidence that it is "not uncommon"? If you had some statistical evidence to back up your statement we might be talking in a way that can bear analysis but otherwise I'm afraid, it's conjecture that this method is an effective way of consistently discerning differences. But I see a way forward in what follows.

 

Re-testing lends credence to it being a solid result and again results are over the 95% confidence level. It is of course exactly different than what most audiophiles expect. They expect to hear differences consciously that still test at near random levels. Not to sub-consciously get good results while saying, " I don't know, sounds the same to me".

 

In some cases test takers know what is being tested for and to focus on and in others they didn't. As for someone giving up and just hitting randomly there is no way to control. It appears even when they think they are hitting choices randomly they often aren't. In any case, results either point to a likely difference or fail to indicate one. As the tests are to see if a difference is perceived that is all you can do with them in that context.

Yes, if the first blind test is used as a qualifying test, then I agree - only those who show they can differentiate the differences should be allowed through to the "actual" blind test - they have proven themselves to have the ability to differentiate this difference on this equipment, at this time. The "actual" blind test is the one where the statistics are recorded & analysed - this "qualifier" test is not used in stats.

 

Then we can agree but I don't see much difference between what I suggested & the above - except maybe your proposal is more stringent

[wgscott]

he takes the statements of individuals & sets out to prove their statements wrong.

 

That, is essence, is what it means to test a hypothesis.

 

It's typical of the motivation which then guides the experimental design, full of experimenter bias & guaranteed to return a null result. Completely & utterly worthless on every level.

 

No. It is typical of any scientific test of a hypothesis. You formulate it as strongly as possible, and then you try to refute it.

 

Ars fell short because they did not formulate it as strongly as possible. It has nothing to do with experimenter bias. Any decent experiment will be immune from experimenter bias and participant bias, ipso facto.

[plissken]

That, is essence, is what it means to test a hypothesis.

 

 

 

No. It is typical of any scientific test of a hypothesis. You formulate it as strongly as possible, and then you try to refute it.

 

Ars fell short because they did not formulate it as strongly as possible. It has nothing to do with experimenter bias. Any decent experiment will be immune from experimenter bias and participant bias, ipso facto.

 

I've been trying to tell them. It may be better received by another however.

[plissken]

Yes, if the first blind test is used as a qualifying test, then I agree - only those who show they can differentiate the differences should be allowed through to the "actual" blind test - they have proven themselves to have the ability to differentiate this difference on this equipment, at this time. The "actual" blind test is the one where the statistics are recorded & analysed - this "qualifier" test is not used in stats.

 

Then we can agree but I don't see much difference between what I suggested & the above - except maybe your proposal is more stringent

 

Agreed. Even a broken clock is correct twice a day.

 

What happens if your screening suggestion leaves you with a zero participant pool to retake the test again to show consistency and repeatability?

[Jud]

So AQ customers don't want to know if their Ethernet cables possibly are so in the margin that they may fail certification?

 

What are you saying about the average AQ customer Jud?

 

Nothing different than saying Mercedes drivers are likely less interested in a printed certificate that their vehicles meet certain minimum automotive specifications than, for example, Chevy owners might be. Mercedes and AQ owners firmly believe their products are *better*. They're not terribly interested in certifications of compliance with minimum standards; likely the reaction if told of noncompliance would be to say that spec must not be important for audio anyway - right?

[mmerrill99]

That, is essence, is what it means to test a hypothesis.

Ok, so he's not interested in whether there is an audible difference between cables which can be differentiated - he's just interested in proving it's not "night & day" so therefore can & probably would use the $160 boombox. So if we are all agreed that this is his hypothesis that he is testing let's not try to pretend otherwise!

 

No. It is typical of any scientific test of a hypothesis. You formulate it as strongly as possible, and then you try to refute it.

 

Ars fell short because they did not formulate it as strongly as possible. It has nothing to do with experimenter bias. Any decent experiment will be immune from experimenter bias and participant bias, ipso facto.

So the strong hypothesis is that any "night & day" difference can be differentiated by a cheap boombox. Excellent- go at it

[mmerrill99]

Agreed. Even a broken clock is correct twice a day.

 

What happens if your screening suggestion leaves you with a zero participant pool to retake the test again to show consistency and repeatability?

You have no result - maybe your pool of listeners was simply undiscerning, untrained listeners? Maybe your equipment was insufficiently revealing. If you bring your experiment to Axpona with a boombox I can guarantee your result.

 

As I said before - if you want to do perceptual testing find out how to do it & stop playing make-believe like you are doing something scientific & of any value!

[plissken]

Nothing different than saying Mercedes drivers are likely less interested in a printed certificate that their vehicles meet certain minimum automotive specifications than, for example, Chevy owners might be. Mercedes and AQ owners firmly believe their products are *better*. They're not terribly interested in certifications of compliance with minimum standards; likely the reaction if told of noncompliance would be to say that spec must not be important for audio anyway - right?

 

So you are saying that Mercedes owners aren't interested in the Car and Driver and other like publications data plots. Braking, Acceleration, Slalom, G Pad, horsepower, torque curve, long term driving results?

 

That they aren't interested in crash testing results?

 

One the second part, if I read between your lines correctly, then we are thinking along the same lines in regards to an AQ customer. Which is funny and sad in it's own rights.

 

Ethernet is an exacting engineered product. I'm not sure it says anything positive about the persons that would pass that off.

[plissken]

Ok, so he's not interested in whether there is an audible difference between cables which can be differentiated - he's just interested in proving it's not "night & day" so therefore can & probably would use the $160 boombox. So if we are all agreed that this is his hypothesis that he is testing let's not try to pretend otherwise!

 

So the strong hypothesis is that any "night & day" difference can be differentiated by a cheap boombox. Excellent- go at it

 

I'm right here. You can ask. The issue is it's already been explicitly stated what is being tested and why it's being tested.

 

Now that 8th grade science subjects are being explained to you I'll rest my case as to your failure to understand what is being theorized.

[wgscott]

"Under what conditions would you be willing to accept that your hypothesis is wrong?"

 

Unless both "sides" are able to do this, there is no point in even conducting a perfect experiment.

 

For the believers, is there any set of potential conditions under which you would be willing to accept being wrong? (If not, it is a tacit admission of irrational metaphysical/religious/superstitious belief.) If so, under what conditions would you accept being wrong?

 

For the non-believers, does it have to be a statistically significant majority of listeners who pass an A/B/X test, or would one listener, who repeatedly can pass an A/B/X test under a variety of conditions, in a statistically significant manner, be enough to accept being wrong?

[plissken]

You have no result - maybe your pool of listeners was simply undiscerning, untrained listeners? Maybe your equipment was insufficiently revealing. If you bring your experiment to Axpona with a boombox I can guarantee your result.

 

As I said before - if you want to do perceptual testing find out how to do it & stop playing make-believe like you are doing something scientific & of any value!

 

You are being corrected about the basics of Theory and Hypothesis but you want to preach to me about perceptual testing? Good luck with that.

 

There is no such thing as zero data in these types of testing. If no one can distinguish? Guest what? I STILL have data. I STILL have #'s.

 

It's just not the #'s YOU want. That isn't what this testing is about. It's not about the #'s I want. I'm designing a test that is evaluating someone else's claims.

 

Those claims are steering the design process. Now that there is commentary from Audiostream it's potentially more data to incorporate and points of contention to mitigate. It's hopefully data that is usable to tighten up the testing gear to better test their claims.

[wgscott]

I'm right here. You can ask. The issue is it's already been explicitly states what is being tested and why it's being tested.

 

Now that 8th grade science subjects are being explained to you I'll rest my case as to your failure to understand what is being theorized.

 

 

Arguing with Creationists, Birthers and other conspiracy theorists is similarly frustrating/entertaining.

[mmerrill99]

You are being corrected about the basics of Theory and Hypothesis but you want to preach to me about perceptual testing? Good luck with that.

 

There is no such thing as zero data in these types of testing. If no one can distinguish? Guest what? I STILL have data. I STILL have #'s.

 

It's just not the #'s YOU want. That isn't what this testing is about. It's not about the #'s I want. I'm designing a test that is evaluating someone else's claims.

 

Those claims are steering the design process. Now that there is commentary from Audiostream it's potentially more data to incorporate and points of contention to mitigate. It's hopefully data that is usable to tighten up the testing gear to better test their claims.

Again you prove your completely out of your depth

I'll leave you to your ineptitude yet again - you again prove you are beyond learning.

[plissken]

For the non-believers, does it have to be a statistically significant majority of listeners who pass an A/B/X test, or would one listener, who repeatedly can pass an A/B/X test under a variety of conditions, in a statistically significant manner, be enough to accept being wrong?

 

Bingo!

 

Does anyone not realize how huge this would be. To find that one consistent individual would open up an entire branch of audio theory, research, measurement, and theory.

 

There would be another data point to base an entire study the effects of differing Ethernet cables on audio reproduction. This would have potentially huge yields for mastering studios, high end audio companies that use TCP/IP in consumer settings, live audio venues, the list goes on.

[plissken]

Again you prove your completely out of your depth

I'll leave you to your ineptitude yet again - you again prove you are beyond learning.

 

Is there an ignore feature here? I can't stand attempting dialog with people that failed 8th grade science.

 

**I found it**. My SNR just improved by 3dB.

[Jud]

So you are saying that Mercedes owners aren't interested in the Car and Driver and other like publications data plots. Braking, Acceleration, Slalom, G Pad, horsepower, torque curve, long term driving results?

 

That they aren't interested in crash testing results?

 

One the second part, if I read between your lines correctly, then we are thinking along the same lines in regards to an AQ customer. Which is funny and sad in it's own rights.

 

Ethernet is an exacting engineered product. I'm not sure it says anything positive about the persons that would pass that off.

 

Crash tests, of course. It's in Mercedes' ads, and not accidentally. They know their market.

 

Now tell me how many Mercedes owners know the C&D skidpad numbers for their cars? How many would switch to a Chevy, Honda or Lexus on learning their skidpad numbers were better? Does Mercedes show C&D skidpad numbers in their ads or print them on certificates for their customers? Of course not. Again, they know their market.

 

So why exactly is it AQ owners merit criticism for the same sorts of attitudes?

 

Regarding "critically engineered:" Which do you think best meets the definition of critical engineering, whether an Ethernet cable used in a home audio system meets the Cat 7 spec, or the performance of the family car on a skidpad?