Blind listening at Schiit

[vmartell22]

 

Jason gets into the fray:

 

https://www.head-fi.org/threads/schiit-happened-the-story-of-the-worlds-most-improbable-start-up.701900/page-3798#post-15486474

[Bill Brown]

Wow.  Interesting.  And impressed with his honesty regarding the very small differences and dismissing some of the more radical subjectivists' beliefs re. output devices, cables, etc.

 

More common ground emerging......(?)

 

Bill

[vmartell22]

The Schiit guys have always striked me as "reasonable subjectivists" - which is totally cool, given how the put literally their money where their mouth is. Not sure if I am completely off base here but have always tought that if Yggdrasil was made by any of the "high end" manufacturers, it would a $15K product instead of $2K.

 

 

[PYP]

So, this guy is talking Schitt and you believe him?  Ok, had to say that.  This is really interesting stuff.  Thanks for the link.  I think the only blind listening I have done was with speakers and the difference, and my preference, was very obvious.  

 

My own experience that you need to live with a new piece of equipment for months to truly appreciate it fully aligns well with this part of his post:  Human hearing seems to be more integrative than differential, so those small differences between components may be magnified over time, and therefore seem larger and more important than during rapid switching.

[fas42]

Interestingly, some of the Schiit people have picked up on the Auditory Scene Analysis concepts ... https://www.audiosciencereview.com/forum/index.php?threads/schiits-jason-stoddard-on-blind-testing.11716/page-4#post-338778

[Audiophile Neuroscience]

When is a Schiit test not a test and how Schitt is the test?

 

Well first kudos to the Schiits for delivering a more balanced approach to the Blinding debate.

 

They are suggesting blind listening over blind testing . Their (as well as mine and others) premise is that tests are anxiety provoking and they suggest therefore blind "listening" may be less anxiety provoking. I'm not sure I was completely convinced with the distinction:

 

"listening to hidden audio components in a low-stress environment. Afterwards, you might pick a favorite, comment on any differences you might hear, or try to identify the product. But there’s no forced choices, no time limit, no pressure…and “there is no difference” is a totally cool answer."

 

To the extent that it is just preference based - Preference and difference are related but not equivalent.  What if A and B were chocolate vs Vanilla ice cream. There can be strong difference but no particular preference either way.

 

They are really asking to discern a difference and I doubt saying "chill out, its okay to fail" will make people less anxious.

 

Anxiety is well known to influence test performances and is one of the reasons conventional audio blind testing has been challenged. You remove one bias (expectation bias) but introduce another in the form of an inter-dependent variable (anxiety). Hawthorne effect (Observational bias) may also be relevant.

 

Even successfully removing one bias does not guarantee that the test is somehow now magically validated. Failure to tell a difference may be due to some other methodological design flaw. This is notwithstanding that sighted listening will produce false positives due to obvious "tells". The fact that nothing else has (seemingly) changed when the tell is removed would appear, prima facie , to equate with the tell (brand, appearance, whatever) being the *only* reason a difference could have been heard. This conclusion only follows if you can demonstrate the test is valid and reliable *in the first place* so that it will pick up on all possible true positives and produce no false negatives (and those measurements cited) ; and that you haven't introduced other confounders.

 

Blind tests are fine but all tests must be cited. Maybe it sounds catchier as, Non-sighted tests must be cited 🤔😁. Tests of test measurements aka calibration against a gold standard of known accuracy.

 

Only small differences or indeed no differences will tend to be yielded in tests that are biased to false negatives (Type 2 error). Everything will supposedly sound the same. There is widespread subjective belief that blind tests are unreliable and there is some evidence to support a systematic bias  towards a high risk of concluding that audible differences are inaudible (type 2 error).

 How Conventional Statistical Analyses Can Prevent Finding Audible Differences in Listening Tests

Leventhal -Type 1 and Type 2 Errors in the Statistical Analysis of Listening Tests