Bits Is/Ain't Bits: A Modest Proposal

[alfe]

BTW, Coke & Pepsi varies their sweetness levels depending on the ethnic group and climate:

 

White skin and closer to the North, less sweetness.

 

Dark skin and closer to the South, more sweetness.

 

I don't believe nobody already made a research like this regarding music taste (and other issues when listening to music). I hope there would be an 'universal truth' regarding music listening, but it's only a hope.

 

Cheers!

 

Roch

 

The ganja man is segregationist?

[elcorso]

The ganja man is segregationist?

 

Never, but I'm saying the truth. I have a cousin who worked with Coke International, by that time I was a Coke addict (no jokes please), when I noticed sweetness differences from country to country I asked him and this was his answer.

 

Ganja farm is closed for maintenance in this our dry season

 

Cheers!

 

Roch

[Paul R]

I would say that that choice is a bit more complex than simple taste preferences. Advertising, cool factor, peer-influences, mood. Plus, a crave for drinks may vary quite wildly during as little as 5 minutes: from water to whisky and all in between. Very hard to do a controlled test for such things.

And nowadays focus groups and other similar tests for new products are losing ground and are replaced with big data, crowds science and so on. A brave new world. Not so relevant for CA I'd say. If you apply that stuff to audio we all end up with youtube, mp3 and bose/beats

 

 

 

For Daudio

Yes I did thank you.

 

Actually, no. All they wanted was a taste preference. The results were very skewed by a single simple factor. Not really anything more complex than that, and a very clear example of why blind testing is not and should not be the ultimate standard to depend upon.

 

A good tool to use, but not the "gold standard" by any means.

 

The same is not true in areas like drug testing, where DBTs are just about the perfect tool. Perfect because humans are not required to provide their sensory evaluations to find out if the drug has had the predicted effect.

 

-Paul

[Superdad]

BTW, Coke & Pepsi varies their sweetness levels depending on the ethnic group and climate:

 

White skin and closer to the North, less sweetness.

 

Dark skin and closer to the South, more sweetness.

 

I don't believe nobody already made a research like this regarding music taste (and other issues when listening to music). I hope there would be an 'universal truth' regarding music listening, but it's only a hope.

 

 

Well there is West Coast jazz and East Coast jazz...

[kumakuma]

Never, but I'm saying the truth. I have a cousin who worked with Coke International, by that time I was a Coke addict (no jokes please), when I noticed sweetness differences from country to country I asked him and this was his answer.

 

Ganja farm is closed for maintenance in this our dry season

 

Cheers!

 

Roch

 

Another difference is that Coke uses different types of sugar in different countries. For example, they use corn syrup in the U.S. and cane sugar in Mexico.

[YashN]

 

OK, a small sample:

 

"The prevalent atitude on CA seems to be that science is useless when it comes to audio. Many people seem to be convinced that audio components are somehow build by wizards with sticks and clay. And particularly DBTs are hated with a passion worthy of a better cause.",

"The only stressful-DBT I can think of is one where the test subjects are those self proclaimed golden ears. They have something to prove and a lot to lose. Especially if they are TAS editors or such"

"Or how about this mature 'answer': laa la laaa laaaaa ."

"Was it that hard to post a link for everyone?" (how about just "thanks for the link")

"Nice anecdote. There are surely another million like this. Especially coming from insiders like him with big green stakes in the game"

"In any case, I so dont buy that lousy 'theory' about night & day differences that magically dissapear on DBTs".

"The circle goes round and round but in audioland it almost invariably ends up somewhere in the stoneage."

"Let me tell you this short anecdote. It begins with 'your worst nightmare is true'. And it abruptly ends there"

"Just hope the typical audiophile spends less time talking with skulls"

"Hopefully you at least have fun writing all those funny phrases."

"Do all objectivists get banned sooner or later here?!"

There's lots more but you get the idea...

 

Sigh, another aggressive, unfriendly comment

 

PM this whole crap from trithio to Chris. Should be treated exactly as Prot.

[Superdad]

…But you see how it goes. Two people who just say, "hey can you hear this if I don't tell you which is which?". Well now that is too technical for our crowd. And doing the full ITU is too expensive and difficult….

 

Sorry Dennis, I am going to call you out on this one. Over a year ago you and I had the following exchange:

 

 

Dennis, I guess the final questions I have for you are:

 

a) If you were present and helped administer a blind test, how many people and what percentage of identifying a difference would you require to accept the test as overwhelming proof that there are differences we can hear for which measurements don't yet exist?

 

b) If you were present and administered a blind test to just one individual, and that individual could correctly identify (change versus no change) 95% of the time, would you accept that as overwhelming proof that there are differences we can hear for which measurements don't yet exist?

 

If you answer yes to "b)", then in my room, with my system, and my music, I would be happy to be that individual any day of the week!

I do require that:

1) I be allowed to go A/B/A/B/A--even if all the "Bs" are really the same as "A", but most of the time I will not need to;

2) I say when to start and stop the tracks and which track to play;

Other than those two things I am happy to be literally blindfolded (keep the blindfold off my ears please).

 

Last requirement is that you publish the results. Where will you be flying in from?

 

--ALEX

 

To which you replied:

 

Well the simple formula for 95% confidence on ABX testing is n/2+ sq.root of n. Example: 20 trials what is 95% confidence. 10+ sq root of 10=13.16. So 14 or more correct means there is only a 5% chance of a lucky set of guesses.

 

Of course that does mean you have that 5% chance of a lucky guess. Yes, one in 20 times you just guess lucky on average. So one prefers if testing one person to have that person repeat at least once to greatly reduce the likelihood it was a lucky guess situation.

 

As to how many trials, you really don't fill out a representative distribution at less than 30 trials. So what would I accept? 30 trials with 21 correct on two runs of the test for one person.

 

Now with multiple people you can pool the results. How many trials is enough? The more the better. I would at least consider it highly suggestive if two runs of 30 trials resulted in 21 or more correct. Say 3 people do 10 trials each, and repeat that again.

 

Now it isn't practical for me to visit you in California for such a test. But doing one would be good. Perhaps someone closer can take part or proctor it. Do the test for yourself with local friends. Would be educational for you all and for anyone looking in if you make public your results regardless of what those results turn out to be.

 

I would suggest not playing music too loudly. An average level of 75 db or so seems the level you are most likely to have good results over a long series of test trials. More than that and your ears tire out over time and your acuity goes down.

 

So forgive me if I am skeptical that you are actually lowering the goal posts to "Hey can you hear this if I don't tell you which is which?"

 

 

I am still not sure why the validity of listening comparisons escapes some people:

You guys do realize that the vast majority of high-fidelity products are designed by engineers for whom both measurement and listening are key interactive and iterative processes during development, right?

In fact, for many seasoned engineers, the selection of topology as well as passive and active parts is done more relevantly and expeditiously in the listening studio than on the bench. The selection of a resistor composition for a critical circuit position or of the capacitor construction of crossover capacitors will yield entirely irrelevant data with even fairly sophisticated measurement gear, but 10 minutes comparing 2 or 3 candidates in an amp or speaker will yield a clear winner or point the way for further research.

Happens every day. Ask virtually any practicing audio engineer about this. Heck, even the guys at Texas Instruments (hardly a bastion of fringe audiophiledom) have, on numerous occasions, used listening tests to provide feedback in opamp development.

 

I could cite specific examples--including blind tests of 4 minutely internally different capacitor construction elements (solder, end-spray, voltage/thermal oven time) conducted by vastly different speaker builders on opposite sides of the world coming to the same conclusion--but you guys would just dismiss them as "anecdotes."

So spare us the new lines that you guys are just looking for reasonable evidence of differences revealed from modest tests.

[Jud]

I could cite specific examples--including blind tests of 4 minutely internally different capacitor construction elements (solder, end-spray, voltage/thermal oven time) conducted by vastly different speaker builders on opposite sides of the world coming to the same conclusion--but you guys would just dismiss them as "anecdotes."

 

I believe trithio did mention something about independent identical impressions/results, so perhaps there's an audience for such information. I don't want to presume to speak for anyone else, though. trithio, Dennis, anyone else who considers him- or herself as having the "bits is bits" position - would this interest you, or would it have insufficient indicia of reliability for you?

[kumakuma]

PM this whole crap from trithio to Chris. Should be treated exactly as Prot.

 

Perhaps we could just throw him in the ocean and see if he floats.

 

7 Bizarre Witch Trial Tests

[Allan F]

Perhaps we could just throw him in the ocean and see if he floats.

 

Why pollute the ocean?

[esldude]

Sorry Dennis, I am going to call you out on this one. Over a year ago you and I had the following exchange:

 

 

 

 

To which you replied:

 

 

 

So forgive me if I am skeptical that you are actually lowering the goal posts to "Hey can you hear this if I don't tell you which is which?"

 

 

I am still not sure why the validity of listening comparisons escapes some people:

You guys do realize that the vast majority of high-fidelity products are designed by engineers for whom both measurement and listening are key interactive and iterative processes during development, right?

In fact, for many seasoned engineers, the selection of topology as well as passive and active parts is done more relevantly and expeditiously in the listening studio than on the bench. The selection of a resistor composition for a critical circuit position or of the capacitor construction of crossover capacitors will yield entirely irrelevant data with even fairly sophisticated measurement gear, but 10 minutes comparing 2 or 3 candidates in an amp or speaker will yield a clear winner or point the way for further research.

Happens every day. Ask virtually any practicing audio engineer about this. Heck, even the guys at Texas Instruments (hardly a bastion of fringe audiophiledom) have, on numerous occasions, used listening tests to provide feedback in opamp development.

 

I could cite specific examples--including blind tests of 4 minutely internally different capacitor construction elements (solder, end-spray, voltage/thermal oven time) conducted by vastly different speaker builders on opposite sides of the world coming to the same conclusion--but you guys would just dismiss them as "anecdotes."

So spare us the new lines that you guys are just looking for reasonable evidence of differences revealed from modest tests.

 

Two different contexts and purposes. You were making claims I thought needed excellent proof for me to be convinced. You asked what would be convincing to me. I described that. It was not a simple blind test with two people making a few comparisons.

 

The later simpler test described is something I called a sanity check. Simple quick, and effective. Not so effective as the more rigorous method. Not enough to confirm some highly improbable idea. But enough to let someone say, "hey does this really make sense, can I really hear what I think I am hearing". As audiophiles over time convince themselves of things outlandish doesn't even fully describe, if more did a simple sanity check they might stay more grounded in reality. Or at least be a bit more skeptical of always trusting what they think they hear. Yes I consider that a bad practice.

 

So if you fail a super simple informal blind test does that mean there is nothing to it? It means the effect isn't large. Similarly as I just described you can't trust just sighted listening either as you can reach faulty conclusions when you get near the edges of perception. It would prevent a common description as some highly improbably things being night and day. Something truly night and day will get picked up in such a simple little test. Again, not a fully definitive test, just a sanity check.

[kumakuma]

Why pollute the ocean?

 

I guess you're right. So why don't we just burn him right now? Seems to be the consensus of the room.

[Jud]

Perhaps we could just throw him in the ocean and see if he floats.

 

7 Bizarre Witch Trial Tests

 

[video=youtube;zrzMhU_4m-g]

[esldude]

[video=youtube;zrzMhU_4m-g]

 

I think you have hit upon the solution Jud. We need the wisdom of Monty Python in these forums. No one can reason like those fellows. That scene (and many others) still crack me up after all these years since first seeing that movie.

[Superdad]

But Jud, I do expect trithio weighs more than a duck. Let's devise a DBT (Duck Blind Test) to determine if it is so!

 

Now if you will excuse me, I have to go feed this rabbit we brought home from Caerbannog. He's already devoured two sacred cows today!

 

[esldude]

But Jud, I do expect trithio weighs more than a duck. Let's devise a DBT (Duck Blind Test) to determine if it is so!

 

Now if you will excuse me, I have to go feed this rabbit we brought home from Caerbannog. He's already devoured two sacred cows today!

 

[ATTACH=CONFIG]16676[/ATTACH]

 

Do we need to invoke some watery tart flinging a scimitar about to choose and annoint a forum super moderator?

[Daudio]

"Originally Posted by Superdad Dennis,...

 

a) If you were present and helped administer a blind test, how many people and what percentage of identifying a difference would you require to accept the test as overwhelming proof that there are differences we can hear for which measurements don't yet exist?

 

b) If you were present and administered a blind test to just one individual, and that individual could correctly identify (change versus no change) 95% of the time, would you accept that as overwhelming proof that there are differences we can hear for which measurements don't yet exist?

If you answer yes to "b)", then in my room, with my system, and my music, I would be happy to be that individual any day of the week!"

 

You asked what would be convincing to me. I described that. It was not a simple blind test with two people making a few comparisons... The later simpler test described is something I called a sanity check. Simple quick, and effective.

 

Don't hide behind an excessively rigorous test, it's inappropriate for that situation anyway, go with your 'sanity check'

 

 

So if you fail a super simple informal blind test does that mean there is nothing to it? It means the effect isn't large. Similarly as I just described you can't trust just sighted listening either as you can reach faulty conclusions when you get near the edges of perception. It would prevent a common description as some highly improbably things being night and day. Something truly night and day will get picked up in such a simple little test. Again, not a fully definitive test, just a sanity check.

 

Ha ! You going on and on about how he is going to fail, sounds like a push poll to me. (opps, off topic)

 

So, what would you say if he didn't fail ? That is the question and the challenge !

Step up ! Put your money where your mouth is !! (figuratively )

 

You doubted yourself once before. You tested yourself and changed your mind then. It could happen again. - Inquiring minds want to know

[Allan F]

I guess you're right. So why don't we just burn him right now? Seems to be the consensus of the room.

 

Not a good solution either; burning would result in air pollution. Besides, show some compassion. There is no need to cause physical pain. IMO, simply ignoring him is still the best approach.

[kumakuma]

Not a good solution either; burning would result in air pollution. Besides, show some compassion. There is no need to cause physical pain. IMO, simply ignoring him is still the best approach.

 

You are probably right. It is the best for the environment as well. I'm not sure it will make some of the folks here happy though...

[trithio]

Actually, no. All they wanted was a taste preference. The results were very skewed by a single simple factor. Not really anything more complex than that, and a very clear example of why blind testing is not and should not be the ultimate standard to depend upon.

 

A good tool to use, but not the "gold standard" by any means.

 

The same is not true in areas like drug testing, where DBTs are just about the perfect tool. Perfect because humans are not required to provide their sensory evaluations to find out if the drug has had the predicted effect.

 

-Paul

 

I guess no one told those taste preference people that 'taste is a matter of taste' . It could have saved them a lot of trouble!

My only point is that you cannot fault DBTs for not being able to measure matters of taste. Nothing else can. Apart from taste . You might just as well say DBTs are useless cause they cannot decide the color of the black energy.

The blame in you example lies entirely with the matters of taste themselves. And with those smartasses who thought they can measure and quantify that kind of thing.

Crowds science does work there but only up to a point. And works only at a macro level and with more generic matters like "do americans like burgers more than pizza". That one is actually very easy in principle: just gather all sales data on those and run a simple algorithm on it. The result is however going to be useless for any individual. That's why I said in a previous post that crowds & bigdata science cannot be applied to audio or be much useful to any of us. At least not today.

Like it or not, there are still no better that DBT tools. In spite of all the limitations. And the entire witch burning story.

By the way, I should be free tomorrow, might as well get it done

[trithio]

http://www.theregister.co.uk/2014/04/14/high_fidelity_pure_audio_launch_event_elton_john_goodbye_yellow_brick_road/

These are apparently the new highres bits in town. Is anyone at least moderately excited about yet another disc-based tech with yet another 'unbeatable' DRM scheme!?

[alfe]

Audio fans, prepare yourself for the Second Coming ... of Blu-ray

These are apparently the new highres bits in town. Is anyone at least moderately excited about yet another disc-based tech with yet another 'unbeatable' DRM scheme!?

 

DVD audio with higher storage capacity.

[YashN]

Why pollute the ocean?

 

Save our Oceans, please.

[Paul R]

Grin - your own logic is faulty. Think about it - using DBTs to try and measure audio without understanding all the factors that go into why a person - a specific person - would prefer one sound over another is well - as silly as the Pepsi Challenge. The slightest factor, say a loudness differential, can skew the results dramatically. That's before you start talking preferences or physical hearing differences or experience or familiarity.

 

DBTs are just not - by a long shot - the "best" tool we have for evaluating sound. I agree they can be useful, but not universally better than sighted listening tests and/or putting a piece of equipment on a test bench.

 

-Paul

 

 

I guess no one told those taste preference people that 'taste is a matter of taste' . It could have saved them a lot of trouble!

My only point is that you cannot fault DBTs for not being able to measure matters of taste. Nothing else can. Apart from taste . You might just as well say DBTs are useless cause they cannot decide the color of the black energy.

The blame in you example lies entirely with the matters of taste themselves. And with those smartasses who thought they can measure and quantify that kind of thing.

Crowds science does work there but only up to a point. And works only at a macro level and with more generic matters like "do americans like burgers more than pizza". That one is actually very easy in principle: just gather all sales data on those and run a simple algorithm on it. The result is however going to be useless for any individual. That's why I said in a previous post that crowds & bigdata science cannot be applied to audio or be much useful to any of us. At least not today.

Like it or not, there are still no better that DBT tools. In spite of all the limitations. And the entire witch burning story.

By the way, I should be free tomorrow, might as well get it done

[trithio]

Grin - your own logic is faulty. Think about it - using DBTs to try and measure audio without understanding all the factors that go into why a person - a specific person - would prefer one sound over another is well - as silly as the Pepsi Challenge. The slightest factor, say a loudness differential, can skew the results dramatically. That's before you start talking preferences or physical hearing differences or experience or familiarity.

 

DBTs are just not - by a long shot - the "best" tool we have for evaluating sound. I agree they can be useful, but not universally better than sighted listening tests and/or putting a piece of equipment on a test bench.

 

-Paul

You wish

 

Some things in audio are a clear matter of preference. Like tube vs SS. Which preference by the way was tested in some major DBTs. On average tubes won with the most novices and SS with the most experienced listeners.

 

But most things audio are actually not a matter of preference. Almost everyone can hear that 1% THN sounds better (all other things equal of course). And DBTs are quite often used to find differences. There is precisely _zero_ taste involved in such a test. So no, I dont see much similarity between audio DBTs and pepsi's somewhat funny taste challenge. Which wasnt even supposed to be much of a science experiment but a clever marketing ploy.