Jump to content
IGNORED

Does BIAS affect audio test results?


Recommended Posts

The short answer is yes BIAS will affect the audio test result.

 

But it’s not that easy that those BIAS testes therefore should be seen as some kind of litmus paper for SQ. All type of subjective SQ tests* also have an effect on the results, because the test methods force us to listen and come to conclusions about the quality of a product in an (for many) unnatural way. The tests themselves have to be done very different to how many people normally listen to and evaluate audio gear** to be scientifically correct. It’s not so much the stress IMO, as it is about the many repeat and short time to hear how it sounds like, compared to then listen for longer time in a familiar system.

 

BIAS tests are good if we for example want to know if a particular cable can sound different to another cable. Not so good if we also want to know which cable people prefer the SQ of. So in other words BIAS tests can be done for “Discrimination testing” but not for “Preference testing”. Threshold test are not for audio gear, it’s for testing the human hearing.

 

*A/B test, ABX test, DBT, blind testes etc.

 

**Familiar system, room, many well-known but different sounded records.   

Link to comment
2 hours ago, pkane2001 said:

 

None of the studies I mentioned so far were used for discrimination testing: all were testing for listener preference. And none required short samples and fast switching during the test. As SAM said, this is an often used argument against blind testing, but perhaps that's an issue only with those who don't know how blind tests are conducted (and I don't mean just subjectivists here, objectivists often have their own misconceptions). This is why I'd like to have this discussion where actual findings and facts can be discussed and referenced instead of generally used but unsubstantiated arguments.

 

Testing for listener preference is one thing, it can be done by A/B test, ABX test, DBT, blind testes etc.

 

To test for BIAS is another thing. How would we know if the test group or some in the test group really liked one gear over another or if it was because of confirmation bias?

 

All type of subjective tests have an effect on the results, not only blind tests. 

 

I see that you find my reasoning to be misconceptions and unsubstantiated arguments so I will not post here anymore.  

Link to comment
  • 3 weeks later...
  • 2 weeks later...
5 hours ago, pkane2001 said:

Your main objection seems to be that longer term audio evaluation is more sensitive than short term to small differences. That's an often brought out hypothesis, but I've yet to see any sort of objective evidence to support it. Can you cite some studies that demonstrate this increased sensitivity? By the way, your hypothesis in no way invalidates blind testing, it simply proposes a method of doing it.

 

 

That is correct my problem have never been about testing SQ blind, I have done it myself a few times. The problem is the methodology commonly used in all type of tests to determine sound quality of different gear.

 

The biggest problem is that these tests are not conducted in the way that I and many audiophile listen and evaluate audio gear sighted. To be able to hear subtle difference between audio equipment I need to be familiar with the room, audio system as well as the recordings. A blind test or a sighted A/B test is only difficult to do if the point is to achieve statistical significant proof by many fast repeated switching of gear.

 

Our Auditory memory is very short so a test there they change gear every 5-30 second test won’t let us hear the SQ just the change between the gear, and to some degree their overall sound signature.

 

To properly evaluate any audio gear we need to be able to first “calibrate” to the sound and then compare it to our long-term memory of real none recorded sound. Echoic memory or other short time memories are not valuable for this task. Then I test two audio gear in my stereo I will compare how the bass, drums, the guitar, piano, voices etc. sound to how they normally sound like in real life and to do that I need to use my long time memory. Even when I compare two audio gear I will use my memory of how those instrument sound compare to both the other audio gear and the references, the non-recording memories I have.

 

I believe that to understand why most audio tests are flawed we need to understand how we hear and compare sound. Here is a start.

 

“Each type of memory is tied to a particular type of brain function. Long-term memory, the class that we are most familiar with, is used to store facts, observations, and the stories of our lives. Working memory is used to hold the same kind of information for a much shorter amount of time, often just long enough for the information to be useful; for instance, working memory might hold the page number of a magazine article just long enough for you to turn to that page. Immediate memory is typically so short-lived that we don’t even think of it as memory; the brain uses immediate memory as a collecting bin, so that, for instance, when your eyes jump from point to point across a scene the individual snapshots are collected together into what seems like a smooth panorama.”

 

https://brainconnection.brainhq.com/2013/03/12/how-we-remember-and-why-we-forget/

Link to comment
18 hours ago, fas42 said:

 

Yes, decide what you want to measure ... IME many ambitious rigs are about developing various types of tonality seasoning, and then of course subjectivity is everything in the assessment of what one hears. But "sound quality" is not subjective, in my book - it is the degree to which there is no significant audible adulteration of what's on the recording by the playback chain - how one assesses is by listening for faults in the sound, clearly evident misbehaviour of the replay.

 

To use the dreaded car analogy, 😜, most audiophiles compare by saying things like, I prefer the MB ambience over the BMW variety ... I say, I note that car 1 develops a slightly annoying vibration while accelerating, at a certain engine speed; whereas car 2 doesn't. Therefore, car 2 is the better quality car.

 

Sound quality is not subjective per se, but the listening evaluation normally is.

 

I thought tonality seasoning was your thing 9_9.  

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...