Is Audiophiledom a confidence game?

[lucretius]

8 hours ago, wgscott said:

The burden of proof falls on those wishing to refute the null hypothesis.

 

Unfortunately, there are different interpretations of the "null hypothesis".

 

A. Null hypothesis could imply a statement that expects no difference or effect. 

B. OTH, the null hypothesis could simply imply a commonly accepted statement.  In this case, it doesn’t necessarily mean that the statement is null itself. (Perhaps the term should be called the “nullifiable hypothesis” as that might cause less confusion).

 

For case A, the burden of proof is always on someone rejecting the "null hypothesis".  Case B is much more fuzzy!

[jabbr]

15 minutes ago, wgscott said:

Maybe an example would help:

 

Null hypothesis:  files with identical checksums sound identical regardless of their history.

 

Non-null hypothesis:  files with identical check-sums can sound different depending on their history.

 

Seems very straightforward but... here is the thing. "Null hypothesis: files with identical checksum" will have distinct electrical playback i.e. noise pattern, and furthermore: every playback will have a distinct electrical pattern. At some level I could easily construct an amplification system that highlights these differences (greatly amplify the noise). By by null hypothesis. Not a good null hypothesis to show what I assume you actually mean.

[wgscott]

The fact that you have to explain how such differences would arise and could somehow maybe be made audible tells me it is a better null hypothesis than the alternative hypothesis.

[wgscott]

27 minutes ago, lucretius said:

 

Unfortunately, there are different interpretations of the "null hypothesis".

 

A. Null hypothesis could imply a statement that expects no difference or effect. 

B. OTH, the null hypothesis could simply imply a commonly accepted statement.  In this case, it doesn’t necessarily mean that the statement is null itself. (Perhaps the term should be called the “nullifiable hypothesis” as that might cause less confusion).

 

For case A, the burden of proof is always on someone rejecting the "null hypothesis".  Case B is much more fuzzy!

 

Most people would recognize "God exists" as not being a null hypothesis, even though in the US something like > 90% claim to believe in God.

[jabbr]

8 minutes ago, wgscott said:

The fact that you have to explain how such differences would arise and could somehow maybe be made audible tells me it is a better null hypothesis than the alternative hypothesis.

 

Eh. The real question is whether such electrical variations persist in a predictable fashion. I'd say that the differences probably exist but are random and impersistent across systems. If you look at electrical changes rather than "SQ" this could be modeled in an entirely electrical fashion e.g. SPICE. That question can be answered with physics.

[jabbr]

11 hours ago, plissken said:

 

Could you, one time, just one time, find an error in the testing method? This rebuttal of 'it's not peer reviewed' is tiresome.

 

You're smart enough to either tear down Archimago's method or repeat it and post contrarian results.

 

That was you asking me to comment on Archimago, right?

 

Regarding your own proposal, unless you care to write it down in a coherent and unambigous fashion i.e. don't change it each time I respond, I will take your last post:

10 hours ago, plissken said:

You are missing the part where you start.

 

Archimago has a instrumented measurement approach. My approach is to de-bias.

 

Ok so do it. How many evaluators do you need to get statistically significant results. More than one. How many? You have 23 cables... that's a lot of combinations. No statistical discussion. Also there may be a number of different switches and endpoints and DACs. You need to take that into account. The way I see it is that, suppose you are going to offer each subject $2000 to participate in your trial, the number of subjects you would need will start ballooning out of control. This will be a difficult study to do.

 

Much easier to do measurements. My advice would be to start by measuring leakage currents in various configurations. Once you find a configuration that has a high leakage current you could then do human tests to see if they are audible. Much better use of your resources.

 

I'm glad to see that you aren't asking for people to listen to recordings because I would have a dead stop objection that the recording process strips out any differences. I would want the recording accuracy to be 5-10x that of the DAC and unless you have a really crappy DAC then that would be out of the question.

[rando]

1 hour ago, jabbr said:

John’s a highly esteemed designer. Not a measurement expert though — we can measure truly amazing levels of small signals. Think satellites able to listen in on ground conversations. The problem is that we don’t know how to properly interpret what we are measuring and that requisite equipment may be prohibitively expensive.

This came as close to anything I've read here to explaining the current predicament.  We could have an amazing wireless protocol for audiophile listening if the world was a better place.  There are more than a few other audio advances that are leaps beyond cables the common person has no access to.  Until our next step forward is granted the scrum of people butting up against that wall and making larger and larger claims is inevitable.  

 

It really gets to be a dull subject very fast.  Maddening only if you let it get to you.  Though I doubt the munificence of noting how unhappy people in this hobby are was sincere.  It was a valid response from someone pushed too far.  Or at least a sane mind inclined towards a positive bent could interpret it that way.  

 

[plissken]

16 minutes ago, jabbr said:

Ok so do it. How many evaluators do you need to get statistically significant results. More than one. How many? You have 23 cables... that's a lot of combinations.

 

I said my approach could theoretically support any number of cables due to the way a LAG is setup. 

 

Here is my offer: 

 

 

 

Quote

Also there may be a number of different switches and endpoints and DACs. You need to take that into account. 

 

I'm not testing switches, endpoint, DAC's. I'm testing like constructed, in spec, averaged priced vs high cost boutique. I don't believe the interconnected equipment matters in this regard since electrically speaking the cables look the same from a connection viewpoint.

 

 

Quote

 

Much easier to do measurements. My advice would be to start by measuring leakage currents in various configurations. Once you find a configuration that has a high leakage current you could then do human tests to see if they are audible. Much better use of your resources.

 

Archimago has already done instrumented analysis. I'm interested in the human perception.

 

If one doesn't discount all the sighted evaluations then you can't discount if I were to go out to someones setup and with biases controlled differences fail to manifest. This has the upshot of obviating your need for a statistically rigorous test. 

 

Quote

 

I'm glad to see that you aren't asking for people to listen to recordings because I would have a dead stop objection that the recording process strips out any differences.

 

So I can rid my system of leakage currents showing up in the DAC output by adding and additional AD loop? 

 

Any objection to using one 90 meter generic cable as a control vs the typical 1-3 meter you see WW, AQ, ChordCable, Kimber, Nordost sell?

[lucretius]

48 minutes ago, wgscott said:

 

Most people would recognize "God exists" as not being a null hypothesis, even though in the US something like > 90% claim to believe in God.

 

You would think, but in my ealier years I have come across many who would not accept this. And while you may think the above a good example, I would argue that "God" is not a valid concept and should not form part of any hypothesis (i.e. the statement "God exists" is neither true nor false, rather it's simply giberish -- it has no real meaning).   

[pkane2001]

5 hours ago, Teresa said:

That is the assumption I believe is wrong. My friend, audio designer John Curl has stated many times that we can only measure a small percent of what we can hear. Measurements get you a competent design, selection of quality parts by listening is what gets you a product people want to hear music through.

 

It's different to say that we can't measure a certain perceived quality of the sound than to say we can't measure difference between two audio signals. I may not know how 'transparency' or 'liquidity' can be measured because that's a perceived quality. But I do know how to determine if two ethernet cables produce an effectively different electrical output at the DAC, and so does John.

 

Speaking of, I recall you've received a Lush USB cable from Peter. Did that make a difference in your system?

 

[jabbr]

2 hours ago, plissken said:

If one doesn't discount all the sighted evaluations then you can't discount if I were to go out to someones setup and with biases controlled differences fail to manifest. This has the upshot of obviating your need for a statistically rigorous test. 

No no no noooooooo......

 

You are trying to move the marble. Your claim is that sighted individual reports are biased. Ok, but I have already stated that I will "tag" a report with a score that determines how much I believe it. If you don't want to do statistics then your listening test gets the same value as a stereophile article. Sorry. No statistics == anecdotal report in my book.

 

You just can't do human testing that anyone will scientifically believe without statistics. You could never get it published without statistics, heck I'd hope that most university professors wouldn't allow a passing grade on a lab (regarding human subjects) without a statistical discussion. IIRC my college drosophila lab for genetics required statistics. @Ralf11?

 

You've complained that I keep saying this is hard, well these are some of the reasons ... let me reiterate that the DAC-ADC-DAC loop is a full stop deal breaker as far as making me believe anything unless your ADC has 5-10x the resolution as your DAC. Non negotiable. 

 

I am not being "difficult" in order to give you a hard time. I spend and have spent a great deal of time over decades deciding how to characterize data and what data to believe. I have long ago decided what it takes to sway me and these (scientific) rules apply to all science not just what you are trying to do.

[plissken]

36 minutes ago, jabbr said:

You are trying to move the marble. Your claim is that sighted individual reports are biased.

 

No I'm not. I'm relying on the statements where people are saying "I see all these reports about the audibility of Ethernet so I can't discount them even though I know it shouldn't make a difference". Yes I am claiming sighted bias to be a problem. And anyone that says they don't discount what people SAY they heard sighted can not, de facto, discount what a person hears when their sight bias is controlled for. 

 

You can't have your cake and eat it too. 

 

Quote

 

Ok, but I have already stated that I will "tag" a report with a score that determines how much I believe it. If you don't want to do statistics then your listening test gets the same value as a stereophile article. Sorry. No statistics == anecdotal report in my book.

 

Here's one statistic for you: over the years of offering this via direct email exchange at [email protected] I've had 32 people vacillate and back down. 

 

I've finally had one taker and that is for the first weekend this November. I'll let everyone know how it goes. The participant sent two cables in: One DIY CAT6a and they cryo treated it (it failed 5e) and an Audioquest Vodka 3 meter (It was marginal 6 and BJC said they wouldn't ship a cable that tested out like it did). 

 

 

Quote

You've complained that I keep saying this is hard, well these are some of the reasons ... let me reiterate that the DAC-ADC-DAC loop is a full stop deal breaker as far as making me believe anything unless your ADC has 5-10x the resolution as your DAC. Non negotiable. 

 

No, I've complained that you've been remiss on point out the short comings. I'm not doing instrumented testing so the resolution of the listener is the only thing that matters. 

 

Quote

 

I am not being "difficult" in order to give you a hard time. I spend and have spent a great deal of time over decades deciding how to characterize data and what data to believe. I have long ago decided what it takes to sway me and these (scientific) rules apply to all science not just what you are trying to do.

 

I never thought you were. But I felt like I had to drag out of you your misgivings. To get the needed numbers you sometimes have to start one at a time. 

[esldude]

38 minutes ago, jabbr said:

snip....

You've complained that I keep saying this is hard, well these are some of the reasons ... let me reiterate that the DAC-ADC-DAC loop is a full stop deal breaker as far as making me believe anything unless your ADC has 5-10x the resolution as your DAC. Non negotiable.      snip.........

 

 

I understand your thoughts here.  And when possible the measuring ADC if you will needs to be better.  I do think if you have no other options it can narrow down the range in which things could matter. 

 

In any case, out of curiosity, what parameters would you accept as 5x better in an ADC?  5x less distortion, noise, bandwidth, what all would you need to make you happy with such a test?

[Ralf11]

12 hours ago, Audiophile Neuroscience said:

 

Yes but how do you tell?

 

blind testing & statistical analysis - did you read the thread???

[jabbr]

14 minutes ago, esldude said:

In any case, out of curiosity, what parameters would you accept as 5x better in an ADC?  5x less distortion, noise, bandwidth, what all would you need to make you happy with such a test?

 

I’m throwing that out — the current DACs have such high resolution DSD512, DSD1024 that it would be unreasonable. Perhaps you could do something very exotic with RF transistors & logic but you know, why??? Better to skip the loop. Actually easier to do good measurements.

 

(Not going to get into bitdepth etc because it would have to be SDM)

 

This is similar to: if you are going to measure a signal bandlimited to X, what us the required minimum resolution of the scope ...

[Ralf11]

How many evaluators do you need to get statistically significant results. More than one if you are attempting to produce a replicable sample of the entire popn. (e.g. you are a manf.).  And some manf. do these studies, tho more should.

 

If you have 23 cables then you might want to use sparse matrix procedures a la Box, Hunter, et al.

 

But I only care about me - 20 tests is fine, maybe even 10, depending on the outcome.  For a consumer the only real issue is WAF, which might better be expended on the sports car.

[Ralf11]

Jabbr - I've seen medical school profs. do things w/o statistics.  OTOH, I'm old and it was long ago - tho just one state over from you.  I thought OMG like thoughts, and "this guy probably sends reviews back to journal editors in red pencil" but as a very junior faculty I kept my big mouth shut.

 

Yes, statistics are very powerful and easy to do on your PC.  people still use PC's right?  maybe they have stat. packages for phones now...

[plissken]

21 minutes ago, esldude said:

I understand your thoughts here.  And when possible the measuring ADC if you will needs to be better.  I do think if you have no other options it can narrow down the range in which things could matter. 

 

In any case, out of curiosity, what parameters would you accept as 5x better in an ADC?  5x less distortion, noise, bandwidth, what all would you need to make you happy with such a test?

 

All you need is an ADC that can capture a source and upon A/B playback you would fail to detect the original and the copy. 

[Ralf11]

10 hours ago, Audiophile Neuroscience said:

 

sex acts with ear, why not

 

You realize that Konrad Lorenz got a Nobel Prize for having a jackdaw copulate with his ear, right?

[Samuel T Cogley]

14 minutes ago, Ralf11 said:

 

You realize that Konrad Lorenz got a Nobel Prize for having a jackdaw copulate with his ear, right?

Best.

CA.

Quote.

Ever.

[esldude]

14 minutes ago, jabbr said:

 

I’m throwing that out — the current DACs have such high resolution DSD512, DSD1024 that it would be unreasonable. Perhaps you could do something very exotic with RF transistors & logic but you know, why??? Better to skip the loop. Actually easier to do good measurements.

 

(Not going to get into bitdepth etc because it would have to be SDM)

 

This is similar to: if you are going to measure a signal bandlimited to X, what us the required minimum resolution of the scope ...

Well I was thinking about the opposite approach.  Is what I had in mind doing 8th generation copies.  Every parameter was at least 8x worse.  Is that audible versus the pristine original.  Didn't seem to be.  

 

Now that doesn't definitively show something 8x better rather than worse wouldn't be an improvement.  But we are already into the realm where some physical constraints of air and sound won't allow that level of improvement.  

 

I never posted the files as it seems people here mostly like to make claims from uncontrolled listening rather than actually listen to something.  I kept doing additional generations of copies.  What eventually made them obviously different is when the noise floor was reaching audibility.  That isn't a surprising result.  

 

 

[esldude]

4 minutes ago, Samuel T Cogley said:

Best.

CA.

Quote.

Ever.

I concur. 

[Teresa]

3 hours ago, pkane2001 said:

Speaking of, I recall you've received a Lush USB cable from Peter. Did that make a difference in your system?

 

Perhaps? However, I'm not willing to comment yet. I let it burn-in and am spending two weeks listening to music through it. This Sunday night I will be taking the Lush out and putting back my inexpensive Dynex USB cable for a week or so and then go back to the Lush again. So it will be another 2 or 3 weeks before I am willing to comment.

 

Read more about it in this post.

[vmartell22]

11 minutes ago, Teresa said:

 

Perhaps? However, I'm not willing to comment yet. I let it burn-in and am spending two weeks listening to music through it. 

 

Read more about it in this post.

 

Burning-in a USB cable? oh god   [v puts popcorn in microwave and start hitting refresh...]

 

 

v

 

[jabbr]

1 minute ago, vmartell22 said:

 

Burning-in a USB cable? oh god   [v puts popcorn in microwave and start hitting refresh...]

 

 

v

 

In her post she writes about "newness bias" which is a great term. I've similarly found that some items I initially liked, lost their lustre over time ... if this is burn-in then burn-in can make stuff worse also