Should blind testing discussion be banned on CA? POLL

[PeterSt]

Originally Posted by Superdad

If you read a dozen reviews on Yelp where the people said the food and service at a restaurant was terrible and made them sick, will you demand photos, doctor's notes and lab reports before you believe that they had a bad experience? Of it a dozen people report that an item they bought on Amazon was cheaply made crap, will you discount that without documented proof?

 

The restaurant : No.

The Amazon : Yes.

[mmerrill99]

Sorry, assuming you are human, your ears can fail you too.

 

I never said it had to be all or nothing. Stop putting words in my mouth. It can be both. When the two disagree which one are you going to think is right? Well it will depend on the circumstances. Maybe weasel words to you, but more like someone who knows it is rarely black or white.

Without knowing the specificity of either test, it's a toss of the coin which is right because we don't know the specificity of either sighted or blind test - in a BT we don't know the level of false negatives in the result - in a sighted test we don't know the level of false positives in the result. So why would you favour one over the other?

[PeterSt]

Both are wrong anyway.

[Jud]

Without knowing the specificity of either test, it's a toss of the coin which is right because we don't know the specificity of either sighted or blind test - in a BT we don't know the level of false negatives in the result - in a sighted test we don't know the level of false positives in the result. So why would you favour one over the other?

 

Things are rarely so simple. I'm absolutely prepared to believe there are many circumstances in which blind testing is very useful. Whether there are situations where it covers up actual differences audible in other circumstances is an area in which I'm interested but am unaware of any rigorous work.

[PeterSt]

What a saltless discussion. Something is missing. Dunno what.

[mmerrill99]

Things are rarely so simple. I'm absolutely prepared to believe there are many circumstances in which blind testing is very useful. Whether there are situations where it covers up actual differences audible in other circumstances is an area in which I'm interested but am unaware of any rigorous work.

Oh, btw, just some recent material I came across which gives some insight into this - it's from a textbook "Sensory Evaluation Techniques", the educational standard in the field of sensory science. & it's about what's called in academia, "duo-trio balanced reference mode test" which I believe ABX testing falls into this category. Anyway, what it has to say at the bottom of Page 73 is this:

 

6.4.3.

Select, train & instruct the subjects as described under section 6.3.3. As a general rule, the minimum is 16 subjects, but for less than 28, the beta error is high. Discrimination is much improved if 32, 40, or a larger number [of subjects] can be employed."

 

[beta error is a statistical error in testing when it is concluded that something is negative when it is actually positive. Beta error is often referred to as "false negative".]

[PeterSt]

Discrimination is much improved if 32, 40, or a larger number [of subjects] can be employed

 

Like 48. Just counting.

[sbgk]

Is there such a thing as post traumatic blind test discussion disorder ? I think we've been sent to purgatory, not trithio.

[PeterSt]

That's what I meant. Maybe not justified though.

Won't do it again.

I think.

[mav52]

What a saltless discussion. Something is missing. Dunno what.

 

Yep, Again a thread that goes down the rabbit hole.

[YashN]

Maybe getting some inspiration from a forum which has been there and done that so to speak can help:

 

Certain areas of the Asylum have been designated as DBT (Double Blind Testing) free zones. In these designated areas, the topics of DBT and ABX are strictly off limits. Any post related to these topics will be subject to deletion.These rules cut both ways:

 

• Pro-DBT posts are not allowed.
  
• Anti-DBT posts are also not allowed.
  

Why are DBT discussions not allowed?

Quite simply, the reason is that these topics rarely spark a productive exchange. While a vast majority of Asylum inmates are firmly in the middle ground, the topics of DBT and ABX tend to force polarization and quickly degrade into death spiraling flame wars.

 

[TABLE=width: 590]

[TR]

[TD]H.[/TD]

[TD]DBT-free zones: In the interest of harmony, certain areas (Cables) have been designated as DBT-free zones. Discussion of DBT and ABX, either for or against, is strictly prohibited in these areas. A DBT-free zone notice is posted at the top of these forums.[/TD]

[/TR]

[/TABLE]

 

Even implementing this is a complicated matter.

 

The misuse of DBT/ABX by people who have no interest at all in actually pursuing the scientific exploration of audiophile subjects is all too common.

 

They use it as a crutch to stifle discussions or to disparage people who are actually doing investigative work and or producing solutions (c.f. the recent disparaging words towards John by Fitzcaraldo).

 

Many of these people do not even have proper credentials in either Science or Engineering, but they pose under the guise of the same to appear knowledgeable.

 

Some of these people will go so far as to ask for ABX testing ad nauseam, but when presented with the same, will resort to saying the ABX logs must then be doctored in some way (c.f. the relevant thread(s) on that HA forum with Amir).

 

This said, there probably are some people who are genuinely qualified and interested in new knowledge, in research, and in progress, and to these looking for and presenting their own measurements and DBT or ABX results could be very important.

 

So, maybe segmenting is a good way of doing it, but I do not see this as easy to do.

 

I don't think there's an easy answer to this, but maybe Chris can get some inspiration from what AA did.

[YashN]

Things are rarely so simple. I'm absolutely prepared to believe there are many circumstances in which blind testing is very useful.

 

Indeed, if we're looking to scientifically determine close cases and have statistically significant results, it is extremely useful.

 

But bear in mind the issue here isn't whether the concepts and methods are useful.

 

The issue is preventing an influx of trolls messing up the forum.

[YashN]

WTF ?

 

Dennis, you have hijacked your own thread ! It is not about "how cool DBT's are and the competition is crap", it is about the acceptance/not of banning a specific thread subject. Says so in the title !

 

So why are you trolling us with this off topic post about "my side is better then yours" ? Are you trying to influence the poll results ?

 

Ah well...

 

Some people can't help themselves... :-P

[Jud]

One of the references beanbag found mentioned finding false negative rates using *other forms of testing*. That's the thing that most intrigues me, because it seems to me to determine the inherent false negative rate of even very good implementations of a given test protocol, you need a different test. One doesn't calibrate an instrument to itself, e.g., you don't measure your DAC's jitter rates with the clocks in your DAC.

 

I'm also interested in how the various auditory phenomena being tested might play into this. Are there phenomena (loudness and frequency, for example) that are less subject to false negatives, and other phenomena less familiar from everyday experience ("smearing" of transients or any other audible indications of ringing, for example) that are more so? Or is it that we by and large really can't hear any audible results of filter ringing, and blind tests are simply telling us this fact?

 

Anyhow, this is diverging from the question of whether DBT should be discussed here. I continue to think the answer is yes, so people can continue to find resources bearing on questions like those I've raised here, and on topics of interest to others, such as results of DBT of particular products.

[YashN]

Let it go?

 

Maybe we can Frozenroll the offenders with a never-ending loop of this:

 

[YashN]

No it should not be banned, but it also should not hijack threads where people are discussing listening experiences and differences they say they do hear.

Some DBT proponents can be like rude guests who crash a wine tasting party

 

What I've seen that helps in some threads (but despite that, you still get the fun-downers) is to mention either in the title or the OP that the thread isn't looking for feedback on DBT or other similar stuff if that's the case.

[Jud]

Indeed, if we're looking to scientifically determine close cases and have statistically significant results, it is extremely useful.

 

But bear in mind the issue here isn't whether the concepts and methods are useful.

 

The issue is preventing an influx of trolls messing up the forum.

 

If your definition of "troll" is "anyone who wants to discuss ABX," then I suppose that's a big problem. I on the other hand think overuse of the t-word is at least as big a problem.

 

Seems to me it's easy enough not to participate in a thread that's headed somewhere you think unprofitable; easy enough to react quite reasonably or, again, simply ignore it when someone brings up DBT in a thread in which you're already participating. Too often the first reflex is to argue, and there goes another thread "down the rabbit hole" as mav52 says.

 

Try not arguing, not bringing up the "t-word," not responding in kind if someone is uncivil. They'll stick out like a sore thumb, and either they'll go away because no one is playing along, or their behavior will be so evident by contrast to everyone else's it will be easy and justified to ban them.

[Jud]

What I've seen that helps in some threads (but despite that, you still get the fun-downers) is to mention either in the title or the OP that the thread isn't looking for feedback on DBT or other similar stuff if that's the case.

 

Yes, that's very effective. Chris does give thread starters leeway in that regard, and I've used it in some threads I've started to tell people in no uncertain terms that the direction they're taking is unwelcome.

[YashN]

If your definition of "troll" is "anyone who wants to discuss ABX," then I suppose that's a big problem. I on the other hand think overuse of the t-word is at least as big a problem.

 

That's not it, and I believe I have made it clear in this and other threads:

 

They use it as a crutch to stifle discussions or to disparage people who are actually doing investigative work and or producing solutions (c.f. the recent disparaging words towards John by Fitzcaraldo).

 

In other threads, I have mentioned how some people will ask questions, and when pointed to the resources where they can get more information or even answers to their questions, will still reply in various negative ways, until they stoop down to insults.

 

This shows these people aren't really interested in learning more.

[YashN]

Try not arguing, not bringing up the "t-word," not responding in kind if someone is uncivil. They'll stick out like a sore thumb, and either they'll go away because no one is playing along, or their behavior will be so evident by contrast to everyone else's it will be easy and justified to ban them.

 

They won't go away if you continue to cajole them since they crave the attention, so if you continue to engage with them, it makes the forum less enjoyable for all other people.

 

Hence the advice: 'Don't feed the trolls'. They need to be called out and dealt with.

[YashN]

I've used it in some threads I've started to tell people in no uncertain terms that the direction they're taking is unwelcome.

 

Good idea.

[Boris75]

If your definition of "troll" is "anyone who wants to discuss ABX," then I suppose that's a big problem. I on the other hand think overuse of the t-word is at least as big a problem.

 

Seems to me it's easy enough not to participate in a thread that's headed somewhere you think unprofitable; easy enough to react quite reasonably or, again, simply ignore it when someone brings up DBT in a thread in which you're already participating. Too often the first reflex is to argue, and there goes another thread "down the rabbit hole" as mav52 says.

 

Try not arguing, not bringing up the "t-word," not responding in kind if someone is uncivil. They'll stick out like a sore thumb, and either they'll go away because no one is playing along, or their behavior will be so evident by contrast to everyone else's it will be easy and justified to ban them.

 

Many thanks for your post. I fully support your veiled call to resist the temptation to call people of a different opinion "trolls."

[wgscott]

"We are such stuff as dreams are made on, rounded with a little sleep".

 

Besides, horses have hoofs, not wheels. Or is it hooves?

 

It is a term for bicycles.

[Jud]

That's not it, and I believe I have made it clear in this and other threads:

 

They use it as a crutch to stifle discussions or to disparage people who are actually doing investigative work and or producing solutions (c.f. the recent disparaging words towards John by Fitzcaraldo).

 

 

I agree that's a good example.

 

May I suggest that calls not to feed trolls might appropriately be done in PMs, so that accusations don't clog up the on-topic discussions.

[mav52]

I agree that's a good example.

 

May I suggest that calls not to feed trolls might appropriately be done in PMs, so that accusations don't clog up the on-topic discussions.

 

Good idea if people would actually do it.... But many would rather point out to the masses how important and smart they are rather than an attempt to smooth things over. But if that PM doesn't work then there is always a full on assault to beat down a poster openly, which is how we got to this mess anyway.