Jump to content
IGNORED

Does BIAS affect audio test results?


Recommended Posts

12 hours ago, fas42 said:

 

My experience has shown that the circumstances that blind tests are conducted in will be enough to completely undermine the optimisation that the hobbyist has applied to his particular situation. IOW, the blind tests never actually test what they purportedly are attempting to reconcile - the abyss in understanding between the two camps remains just as wide and deep as it ever has; and will remain so, until greater overall understanding evolves ...

 

 

You've obviously had a huge amount of experience conducting blind tests, Frank! Can you please share any of the results and procedures here, so we can try to reproduce your findings?

Link to comment
11 hours ago, pkane2001 said:

 

You've obviously had a huge amount of experience conducting blind tests, Frank! Can you please share any of the results and procedures here, so we can try to reproduce your findings?

 

Paul, what I mean here is that the organisation of the gear so that a blind test can be carried out will often suffer from the observer effect - will take require a bit of care to ensure this is not the case.

Link to comment
31 minutes ago, fas42 said:

 

Paul, what I mean here is that the organisation of the gear so that a blind test can be carried out will often suffer from the observer effect - will take require a bit of care to ensure this is not the case.


That is true. Setting up a blind test can be hard, especially with speakers and headphones.

Link to comment

This head-fi thread doesn't rise to the level of a real study (or meta-study), but summarizes a lot of the blind tests and attempts at blind tests, especially ABX, that have been tried. These mostly demonstrate that the audiophile-reported huge differences between cables, DACs, amps, and various hi-res formats are not so huge or obvious when proper bias controls are put in place:

 

https://www.head-fi.org/threads/testing-audiophile-claims-and-myths-the-original-compilation.769887/

 

Quote from the conclusion section:

 

"If hifi is all about sound and more specifically sound quality, then we should, once the other senses have been removed be able to hear differences which can be verified by being able to identify one product from another by only listening. But time and again we cannot."

Link to comment

This is an article from The Institute of Engineering and Technology, a publication subscribed to by over 140,000 professional engineers according to their own stats:

 

https://eandt.theiet.org/content/articles/2011/11/audio-and-technological-mythology/

 

A short quote that agrees with the other studies shared in this thread:

 

Confirmation bias plagues the audio business. The brain is only too ready to accept that making a small change to an audio system – such as plugging in a new cable – results in a perceptible difference in performance.

 

The article goes into a few of the audio myths, such as AC power cords, burn-in for electronics, bi-wiring, analog vs digital, etc., in more detail.

 

Link to comment
10 hours ago, pkane2001 said:

The article goes into a few of the audio myths, such as AC power cords, burn-in for electronics, bi-wiring, analog vs digital, etc., in more detail.

 

It should be obvious to many that The Institute of Engineering and Technology, a publication subscribed to by over 140,000 professional engineers has their own set of expectation biases . 

Quote

The article goes into a few of the audio myths, such as AC power cords, burn-in for electronics, bi-wiring, analog vs digital, etc.,

 

It's more of the usual " if we can't measure it, then it's not there"  syndrome that you also appear to subscribe to .😋

 

BTW, this area of the forum is supposed to be about Objective proof, and is not intended to be another vehicle for attacking what Audiophiles report about your so called " Audio Myths"  , many of which should be quite provable if properly conducted DBT sessions were performed.

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment

Paul's doing an excellent job of developing tools that help with understanding what might be happening when the SQ is below par - I applaud what he has contributed to the community! 👍

 

Eventually, methods will be developed so that numbers can be applied, relatively straightforwardly, to what people are hearing - I'm always looking out for any serious research into getting closer to this goal; now and again something looks promising, but then it disappears into obscurity - there is little overall interest, it seems; far easier to keep beating the "it's all in their heads!" drum, 😜.

Link to comment
3 hours ago, sandyk said:

 

It should be obvious to many that The Institute of Engineering and Technology, a publication subscribed to by over 140,000 professional engineers has their own set of expectation biases . 

 

It's more of the usual " if we can't measure it, then it's not there"  syndrome that you also appear to subscribe to .😋

 

BTW, this area of the forum is supposed to be about Objective proof, and is not intended to be another vehicle for attacking what Audiophiles report about your so called " Audio Myths"  , many of which should be quite provable if properly conducted DBT sessions were performed.

 

Considering all the studies I've shared are about subjective evaluation and biases involved in doing them, measurements are not involved. So, perhaps you're showing just a bit of your own bias in trying to attack me on things that I explicitly didn't bring up or say in this thread? And even going so far as to diagnose a "syndrome" that I apparently "subscribe to" that has no bearing on this discussion?

 

Link to comment
6 minutes ago, pkane2001 said:

So, perhaps you're showing just a bit of your own bias in trying to attack me on things that I explicitly didn't bring up or say in this thread?

NO !

If you hadn't felt the need to post the part I mentioned, I wouldn't have felt the need to respond.

 You were demonstrating your own prejudices by saying that.

Quote

The article goes into a few of the audio myths, such as AC power cords, burn-in for electronics, bi-wiring, analog vs digital, etc.,

 

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment
Just now, sandyk said:

NO !

If you hadn't felt the need to post the part I mentioned, I wouldn't have felt the need to respond.

 You were demonstrating your own prejudices by saying that.

 

 

Alex, enough. You're not adding anything to this discussion but smoke. I didn't mention measurements. I shared an article which explicitly mentioned myths and audiophile biases. Their words, not mine. What I posted was a description of what's in the article. I didn't make it up. You don't like it? Argue with facts and examples in the article and not with ad hominem attacks .

Link to comment
23 minutes ago, pkane2001 said:

 

Alex, enough. You're not adding anything to this discussion but smoke. I didn't mention measurements. I shared an article which explicitly mentioned myths and audiophile biases. Their words, not mine. What I posted was a description of what's in the article. I didn't make it up. You don't like it? Argue with facts and examples in the article and not with ad hominem attacks .

 

 Your post #56 was deliberately provocative , Anti Subjective, and presented no facts or corroborating measurements  of your own.

 

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment
On 6/17/2020 at 10:32 PM, pkane2001 said:

This head-fi thread doesn't rise to the level of a real study (or meta-study),

 

Agreed (premise 1) ^^^

 

 

On 6/17/2020 at 10:32 PM, pkane2001 said:

but summarizes a lot of the blind tests and attempts at blind tests, especially ABX, that have been tried.

 

Agreed (premise 2) ^^^

 

On 6/17/2020 at 10:32 PM, pkane2001 said:

 

 These mostly demonstrate that the audiophile-reported huge differences between cables, DACs, amps, and various hi-res formats are not so huge or obvious when proper bias controls are put in place:

 

 

Conclusion ^^^?

See premise 1

Sound Minds Mind Sound

 

 

Link to comment
17 minutes ago, Audiophile Neuroscience said:

 

Smoke? Perhaps.  But maybe bias is also reflected in 'mirrors'...and yes that includes me too 🙂

 

 

Of course everyone's guilty of bias. That's what the studies so far clearly illustrate. Despite this, there is a way to move forward, and that's one of the main reasons for this thread:

 

1. Help recognize that our preferences often are affected by things that have nothing to do with sound that are often subconscious and not under our control

2. Figure out a way to remove as many biases as possible from affecting the outcome of a preference test

Link to comment
47 minutes ago, Audiophile Neuroscience said:

 

Agreed (premise 1) ^^^

 

 

 

Agreed (premise 2) ^^^

 

 

Conclusion ^^^?

See premise 1


Read the individual tests and their results. The conclusion was not mine, it's a summary of the comments in the post I shared. His final statement is:

 

The clear conclusion is that ABX testing does not back up many audiophile claims, so they become audiophile myths as they show cables do not inherently change sound. Any change in sound quality comes from the listeners mind and interaction between their senses. What is claimed to be audible is not reliably so. 

...

If hifi is all about sound and more specifically sound quality, then we should, once the other senses have been removed be able to hear differences which can be verified by being able to identify one product from another by only listening. But time and again we cannot.

 

You and Alex seem eager to read bias into my own posts. And while I'm sure I'm biased, here, I'm just trying to share information that I found useful. Considering the post is a summary and an analysis (with references) of various blind tests undertaken by the audio community, including manufacturers and retailers, this is an important data point in this discussion.

 

 

 

Link to comment
5 minutes ago, pkane2001 said:

 

Of course everyone's guilty of bias.

 

Maybe, at least potentially

 

 

5 minutes ago, pkane2001 said:

That's what the studies so far clearly illustrate.

 

 

The 'studies' (like the Head FI thread or opinion piece you linked) don't clearly demonstrate anything beyond opinion, IMO....and biased conclusions

 

 

5 minutes ago, pkane2001 said:

Despite this, there is a way to move forward, and that's one of the main reasons for this thread:

 

see post 64

 

5 minutes ago, pkane2001 said:

 

1. Help recognize that our preferences often are affected by things that have nothing to do with sound that are often subconscious and not under our control

 

Sometimes, if borne out by actual real experiments

 

5 minutes ago, pkane2001 said:

2. Figure out a way to remove as many biases as possible from affecting the outcome of a preference test

 

Yes , if done in an unbiased way that is methodologically valid

Sound Minds Mind Sound

 

 

Link to comment
2 minutes ago, pkane2001 said:

Read the individual tests and their results.

 

None that I read seemed convincing to me. I am happy to reconsider on a case by case basis.

 

 

2 minutes ago, pkane2001 said:

 

The conclusion was not mine, it's a summary of the comments in the post I shared.

 

Who cares apart from the individual forum posters.

On 6/17/2020 at 10:32 PM, pkane2001 said:

This head-fi thread doesn't rise to the level of a real study (or meta-study),

Sound Minds Mind Sound

 

 

Link to comment
Just now, Audiophile Neuroscience said:

 

None that I read seemed convincing to me. I am happy to reconsider on a case by case basis.

 

Who cares apart from the individual forum posters.

This head-fi thread doesn't rise to the level of a real study (or meta-study),

 

Considering that I didn't claim anything about their validity, you can stop arguing with me about them.  But perhaps you can comment on the other 15+  (published and peer-reviewed) studies posted in this thread that demonstrate similar findings? 

Link to comment
Just now, pkane2001 said:

 

Considering that I didn't claim anything about their validity,

 

Then why mention them?

 

Just now, pkane2001 said:

  But perhaps you can comment on the other 15+  (published and peer-reviewed) studies posted in this thread that demonstrate similar findings? 

 

Pick any one you like and happy to discuss

Sound Minds Mind Sound

 

 

Link to comment
Just now, Audiophile Neuroscience said:

 

Then why mention them?

 

 

Pick any one you like and happy to discuss

 

Why? Because they are all actual data points summarizing the results of many attempts by audiophiles trying to perform less biased listening tests. Most, if not all, were performed by audiophiles who believed they could hear huge differences, and yet failed to hear them without visual confirmation, i.e., in a blind test. These results are easily predicted by the other, more serious and controlled studies already cited, so they seem to fit into the overall pattern of this thread: biases have a strong effect on preferences in uncontrolled listening testing.

Link to comment
3 minutes ago, pkane2001 said:

 

Why? Because they are all actual data points

 

Yet you said they were not necessarily valid. Are we supposed to be swayed by invalid data points?

 

3 minutes ago, pkane2001 said:

These results are easily predicted by the other, more serious and controlled studies already cited, so they seem to fit into the overall pattern of this thread: biases have a strong effect on preferences in uncontrolled listening testing.

 

If easy, pick one and show us how easy it is

Sound Minds Mind Sound

 

 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...