Should blind testing discussion be banned on CA? POLL

[petaluma]

His name is Jud not Jesus:)

I don't think he want to be crucified to save the CA community.

 

Alfe, damn I was hoping he could turn water to wine and I could purchase some. On a more somber note your statement about someone being crucified for attempting to do something positive seems to be a pretty common problem. At least its discussed a lot.

[Jud]

While I can't, in principle, disagree with anything that you say, I'm enough of a realist to know that intelligent people with strong opinions are often going to express them in a manner that reflects that, especially when they disagree. I'm sure that you have experienced the same regularly as an attorney. Ironically, in my experience, the civil lawyers were far less civil than the criminal lawyers. However, there is no question that the more egregious conduct can be toned down considerably, if not eliminated.

 

It might do well for us to reflect on the following:

 

"We cannot reason our way out of our basic irrationality. All we can do is learn the art of being irrational in a reasonable way."

 

-Aldous Huxley (Island)

 

I am so going to steal that quote....

[Jud]

His name is Jud not Jesus:)

I don't think he want to be crucified to save the CA community.

 

alfe, did you happen to look at the Location under my avatar?

[Jud]

Jud, +1, are you interested in leading a team to create a new user code of conduct sticky/document/thread????? Might help at least the new folks.

 

1 - It's Chris's site.

 

2 - Strangely enough, I like the way he runs it.

 

3 - I *very* seriously doubt a new user code of conduct would help (would we even agree on what it should say?). After all, everyone supposedly already knows the Golden Rule, and that's probably the most succinct and to the point code of conduct I know of.

 

4 - If Chris wanted additional moderators to help share some of that burden, I don't know whether I'd have the time and energy to do a decent job of it. But I'll leave the door open to him contacting me (or anyone else he cares to) about that if/when he wants. Completely up to him, as it should be.

 

5 - On this site as any other forum, we have all got exactly two choices: (a) Whether or not to act in a way that helps create the forum we want; and (b) whether to stay or leave. That's it. I'd stop looking to anyone else - Chris, a user group, a written code - to somehow provide option ©, "Magically save us from ourselves."

[petaluma]

1 - It's Chris's site.

 

2 - Strangely enough, I like the way he runs it.

 

3 - I *very* seriously doubt a new user code of conduct would help (would we even agree on what it should say?). After all, everyone supposedly already knows the Golden Rule, and that's probably the most succinct and to the point code of conduct I know of.

 

4 - If Chris wanted additional moderators to help share some of that burden, I don't know whether I'd have the time and energy to do a decent job of it. But I'll leave the door open to him contacting me (or anyone else he cares to) about that if/when he wants. Completely up to him, as it should be.

 

5 - On this site as any other forum, we have all got exactly two choices: (a) Whether or not to act in a way that helps create the forum we want; and (b) whether to stay or leave. That's it. I'd stop looking to anyone else - Chris, a user group, a written code - to somehow provide option ©, "Magically save us from ourselves."

 

Fair enough

[Jud]

Fair enough

 

Sorry if that sounded negative or harsh. I laud the sentiment it comes from but don't think it can resolve matters.

[Paul R]

There is no doubt a DBT can return false results. Repeating them to confirm a result is done for that reason. The very nature of 95% confidence is that some 5% can be wrong due to randomness. There are other things needed to make them good. They can be wrong due to other factors.

 

EDIT to add: now come to think of it, I find most sighed listening comparisons to have two faults. One they blow actual differences out of proportion. The classic minor barely heard (though perhaps musically important) difference being called night and day. Two there is always a certainty expressed. To do much questioning of the sighted listening quickly results all to often in the heated hurt reply that you shouldn't call into question what another person did or didn't hear.

 

So how about some confidence levels on those sighted listening conclusions? 90% sure whatever seems they are always stated as if it is 100% certainty. Equally maybe descriptions of it being 2% better instead of all these night and day important differences.

 

I don't say only DBT's can be used to evaluate equipment. Remember I said it was a last resort usually. Sighted listening can reveal genuine differences. Is it as valid as DBT? Well depends on your purposes and the size of the differences. In my opinion in order of discriminating ability, you have sighted listening, level matched sighted listening A/B comparisons, measurements, and finally if something is too complex or not certain of being audible the test via DBT. Even the DBT can be informal and barely better discrimination or it can meet full ITU recommendations being quite rigorous.

 

Would you believe I totally agree with you?

 

And boy, did you bring up the 800# Gorilla in the room - namely the actual magnitude of changes that are often described somewhat too enthusiastically. (grin)

 

In terms of DACs, Amps, Preamps, and Players, so far as I can tell, the differences are small in terms of absolute magnitude, but weigh heavily in personal preferences. Which is, perhaps, as it should be.

 

All the brouhaha over Blind vs. Sighted testing is just basically noise, especially when you get to the obnoxious levels like say, occurs regularly on certain other forums. What's the point anyway? If one is interested in other people's opinions about a piece of music or a new piece of kit, well then - DBTs are not going to tell you very much.

 

And everyone will have a slightly different threshold on what they will accept too, of course. For example, I am pretty sure my opinion agrees with yours on $10,000 speaker cables. I am not so sure we agree on subtle differences, like between two DSD Dacs.

 

In some cases, there are no differences I can identify, at least by ear.

 

-Paul

[petaluma]

Sorry if that sounded negative or harsh. I laud the sentiment it comes from but don't think it can resolve matters.

Jud, no offense taken. My thoughts are as I posted earlier that it's Chris's site and as a businessman its entirely up to him how he runs his business. If we don't like it we go somewhere else, there are no shortage of Internet audio sites available....

 

No worries man.

[kumakuma]

Would you believe I totally agree with you?

 

And boy, did you bring up the 800# Gorilla in the room - namely the actual magnitude of changes that are often described somewhat too enthusiastically. (grin)

 

In terms of DACs, Amps, Preamps, and Players, so far as I can tell, the differences are small in terms of absolute magnitude, but weigh heavily in personal preferences. Which is, perhaps, as it should be.

 

All the brouhaha over Blind vs. Sighted testing is just basically noise, especially when you get to the obnoxious levels like say, occurs regularly on certain other forums. What's the point anyway? If one is interested in other people's opinions about a piece of music or a new piece of kit, well then - DBTs are not going to tell you very much.

 

And everyone will have a slightly different threshold on what they will accept too, of course. For example, I am pretty sure my opinion agrees with yours on $10,000 speaker cables. I am not so sure we agree on subtle differences, like between two DSD Dacs.

 

In some cases, there are no differences I can identify, at least by ear.

 

-Paul

 

Your point about the magnitude of differences is well made.

 

For example, I believe, perhaps mistakenly, that my system/ears would prevent me from getting the same dramatic improvements that others get when they change something in their system.

 

As a result, I spend most of my audio budget in the way that brings me the highest return: new music or upgrades to the music I already own.

 

Does this make me more of a music lover than those who spend their money on linear power supplies and expensive cables?

 

No, it just means that I have decided to spend my money in the way that I feel provides the biggest bang for me.

[esldude]

Well you are right as always Jud, thanks. And Dennis has never made a personal attack on me--just my ilk.

 

 

I guess I am just a bit confused about what Dennis and others are after, and wonder why they are in the hobby if they are so afraid to trust their ears. It is not as if I have some sort of special dog hearing, and based on so many anecdotal reports it is hard to fathom why they are still so skeptical.

 

In yesterday's post I used the example of near-universal praise for Miska's Poly-sinc family of filters. Hardly a controversial thing (in that what it does is readily measurable), but Dennis' reply was "I would like to see some more evidence. As this is the kind of thing people report which is marginal and often turns out not to be so."

 

Anyway, I am okay with Dennis having his views and me mine. And I bet we could enjoy a beverage and some good tunes together. But you are right, I need to not take this personally.

 

Best to all,

--Alex C.

 

Yes Alex, I am sure we could enjoy good music and beverages together. I don't think we would have to work hard on it all either. Pretty sure it would just be simply enjoyable.

 

As for trying to explain what I am after it likely would only cause some other people to get pi$$ed off. So I'll just skip that at the current time.

[sandyk]

I also don't really like to accept 95% or 2 sigma results. You get much better concrete results that won't disappear upon retrial if you stick with at least 3 sigma or 99.7% confidence levels. I then of course get accused of moving goalposts or trying to make the acceptance level too stringent

 

You keep ignoring the fact that after a certain number of tests, everything starts to sound the same, even when it isn't.

You would need to do a long series of shorter tests to avoid the fatigue/overload factor if you insist on 99.7% confidence level. That kind of rigorous testing would cost REAL money and a lot of time to implement properly.

We are humans, NOT computers ! After even a short series of tests you may begin to wish the whole damn ordeal was over, which certainly wouldn't help with accurate results.

You don't appear to get it either, that short segments do not affect the emotions. Music is about emotion.

That's why many prefer fewer ,but much longer segments under relaxed conditions.

[Hifidelit]

Aaaaah.....interesting example using the subjective experience of wine tasting.

 

And correct me but...isn't it exactly what happened where the fancy pants abilities to distinguish premium wine were exposed for nonsensical posturing?

 

Listen Alex, I get it....really I do. The mystique of tweeking is held in the highest of reverence for you and the success of your business. It's really a niche' market and shrinking quickly. Anything you and others who share your position can do is certainly a worthwhile effort. Noble work certainly recognized by our host who permits you to tag your signature with a link to your company.

He doesn't permit us to tag our signature - he requires we have the tag in our signature, presumably so that any posts can be taken in the context of maybe selling the product under discussion.

 

The advertising part is a little unclear to me as I linked to a product once in a reply and the post was deleted but the master licence thread started by JRiver is OK it seems.

[Daudio]

Jud,

2 - Strangely enough, I like the way he runs it.

 

Ok, but then you like the trolls, or, at least don't seem to be bothered by their posts. Not exactly typical of the average CA member !

 

 

3 - I *very* seriously doubt a new user code of conduct would help.

 

What happened to your optimism ? I see a very different outcome.

 

eg: New user starts posting troll like stuff, someone points him to the Posting Guidelines document, indicating what they are doing wrong. User either reads it, realizes their mistake, and self-corrects, or continues ugly behavior, ignores warnings, gets reported to Chris, and gets banned.

 

The point is, most of the problems get nipped in the bud, and offenders have no excuse. Not so easy to do that now.

 

 

5 - On this site as any other forum, we have all got exactly two choices: (a) Whether or not to act in a way that helps create the forum we want; and (b) whether to stay or leave. That's it.

 

a) That attitude is fine amongst ourselves, but doesn't seem to have any effect on the prot's and trithio's. Or the partisans that jump in and amplify the 'troll effect'. I prefer a balance between optimism and reality.

 

b) I think I have heard that one before: "Love it, or leave it". Not my favorite

[Allan F]

And boy, did you bring up the 800# Gorilla in the room - namely the actual magnitude of changes that are often described somewhat too enthusiastically.

 

It may be the '800# Gorilla in the room' to you and little more than a mouse to someone else. IMO, differences are described too enthusiastically to no greater an extent than doubters choose to minimize the magnitude of differences while criticizing those who describe them formerly.

 

There is no standard for describing the relative magnitude of sonic differences heard. It is, by its very nature, subjective and personal. What I perceive as a very significant difference, you may perceive as relatively minor and vice versa. What is common and important is the fact that we both hear a difference. The priority that the difference will be given in making any potential equipment choices will also be subjective and personal, Some will base it on the magnitude of the perceived difference, others may simply consider the fact that they hear a difference, while another group might choose a cost/benefit strategy.

[mayhem13]

You keep ignoring the fact that after a certain number of tests, everything starts to sound the same, even when it isn't.

 

You don't appear to get it either, that short segments do not affect the emotions. Music is about emotion.

That's why many prefer fewer ,but much longer segments under relaxed conditions.

 

So glad you posted these two very importent viewpoints. It might aid some in understanding how you arrive at your conclusions and revere your listening skills.

 

Thank you.

[Superdad]

Yes Alex, I am sure we could enjoy good music and beverages together. I don't think we would have to work hard on it all either. Pretty sure it would just be simply enjoyable.

 

Thank you Dennis. I truly agree.

[alfe]

So glad you posted these two very importent viewpoints. It might aid some in understanding how you arrive at your conclusions and revere your listening skills.

 

Thank you.

 

I wonder which is hurting more the prostate or you:)

[Superdad]

Your point about the magnitude of differences is well made.

 

For example, I believe, perhaps mistakenly, that my system/ears would prevent me from getting the same dramatic improvements that others get when they change something in their system.

 

As a result, I spend most of my audio budget in the way that brings me the highest return: new music or upgrades to the music I already own.

 

Does this make me more of a music lover than those who spend their money on linear power supplies and expensive cables?

 

No, it just means that I have decided to spend my money in the way that I feel provides the biggest bang for me.

 

 

It may be the '800# Gorilla in the room' to you and little more than a mouse to someone else. IMO, differences are described too enthusiastically to no greater an extent than doubters choose to minimize the magnitude of differences while criticizing those who describe them formerly.

 

There is no standard for describing the relative magnitude of sonic differences heard. It is, by its very nature, subjective and personal. What I perceive as a very significant difference, you may perceive as relatively minor and vice versa. What is common and important is the fact that we both hear a difference. The priority that the difference will be given in making any potential equipment choices will also be subjective and personal, Some will base it on the magnitude of the perceived difference, others may simply consider the fact that they hear a difference, while another group might choose a cost/benefit strategy.

 

 

Good points gentlemen. And being as our systems/rooms/musical tastes and priorities are likely most all in different states and on different tracks, I do try to remind myself that changes which might be very significant in my system may barely register for others. So I try to temper use of superlatives unless something is so blatantly different that I am confident everyone who makes the same comparison will hear it that way.

 

But for me, if I hear a difference, that difference is either better or worse, regardless of magnitude. Only once in a great while do I compare two things (s/w or h/w) and end up finding both 'A' and 'B' to be a compromise and wish for something in between or imagining something a lot better. (Of course the nature of the journey is that you don't know what better sounds like until you hear it!) This is setting aside the instances were I don't hear any difference at all--a result that I am not at all afraid of.

[sandyk]

So glad you posted these two very importent viewpoints. It might aid some in understanding how you arrive at your conclusions and revere your listening skills.

 

Thank you.

As usual, the dose of sarcasm that most expect from you.

Have you ever participated in a series of blind tests, with say 20 different pairs of comparison tracks, but 3 lots of them with the same files generated and renamed by somebody else's computer from the original supplied 20 pairs of different tracks, using a program specially written for this test? (a total of 60 comparisons) If you haven't , then you don't have any idea about how tedious and hard it is to do.

Especially, if none of the comparison tracks sounds quite the same as the original file inserted as a reference to better help in identifying any differences. (i.e. a total of 90 comparison tracks)

[mayhem13]

I wonder which hurting more the prostate or you:)

 

See a doctor. It might be serious.

[gmgraves]

Pretty simple.

 

Yes, it should be banned.

 

No, it should not be banned.

 

Note this is a public poll.

 

 

This is a public forum and censorship should never be encouraged. Societies lose their freedoms when one point of view seeks to quell all other points of view that don't agree with the majority or the official policy.

[esldude]

You keep ignoring the fact that after a certain number of tests, everything starts to sound the same, even when it isn't.

You would need to do a long series of shorter tests to avoid the fatigue/overload factor if you insist on 99.7% confidence level. That kind of rigorous testing would cost REAL money and a lot of time to implement properly.

We are humans, NOT computers ! After even a short series of tests you may begin to wish the whole damn ordeal was over, which certainly wouldn't help with accurate results.

You don't appear to get it either, that short segments do not affect the emotions. Music is about emotion.

That's why many prefer fewer ,but much longer segments under relaxed conditions.

 

Another straw man Alex. I have ignored none of the things you said I ignored. For instance I like at least 30 samples and were it me I would not do more than 15 at once. Short they would be. If you find longer and fewer per session works better you can do them that way. Yes it takes time. And 99% confidence is not 99% correct. For instance out of a test of 20 samples it would only be two more correct choices or 17 out of 20. For thirty you only need 24 out of 30.

 

This is one of the frustrating things. Someone said I was miffed that others don't so clearly agree with me. That does not bother me all that much. What does is repeatedly, over and over and over people pull out the same old complaints which both are not true and have been explained away dozens of times. If you really are discussing something, instead of supplying knee jerk complaints for something you have already decided you don't like then you wouldn't see posts like this.

 

I have done it both ways, and I can assure you for me the results are better shorter. Why is it okay for people to not believe me in my ABX experience while saying I can't know what someone hears and doesn't hear? If you do it both ways you would know which works better for you. I am guessing you aren't going to do it either way. That is fine too, there just isn't much upside to trotting out the old complaints to score points or whatever you think it accomplishes.

[mayhem13]

As usual, the dose of sarcasm that most expect from you.

Have you ever participated in a series of blind tests, with say 20 different pairs of comparison tracks, but 3 lots of them with the same files generated and renamed by somebody else's computer from the original supplied 20 pairs of different tracks, using a program specially written for this test? (a total of 60 comparisons) If you haven't , then you don't have any idea about how tedious and hard it is to do.

Especially, if none of the comparison tracks sounds quite the same as the original file inserted as a reference to better help in identifying any differences. (i.e. a total of 90 comparison tracks)

 

Yes, I heard you Alex. ....one must be emotionally vested in the musical performance to be able to be objectively comparative........thanks.

 

Carry on.

[sandyk]

This is a public forum and censorship should never be encouraged. Societies lose their freedoms when one point of view seeks to quell all other points of view that don't agree with the majority or the official policy.

 

Well said.

[sandyk]

I am guessing you aren't going to do it either way.

 

Dennis

You are assuming that I have never attempted this kind of thing before.

See post 166.

Alex