Article: My Lying Ears

[Sal1950]

Science applied - Home

 

I'll side with the industries two single most influential people: Toole and Olive

 

I agree let's not just talk about AB/X testing. There are other testing protocols out there to blind and thus prevent bias.

 

The article writer finally figured out the pursuit of audio and it was to finally submit to getting rid of pre-conditioning and accepting it for the data driven endeavor that it is.

 

Blinding isn't handi-capping. It's isn't contrived listening. It isn't even stressful. What is stressful is thinking you can hear differences in everything instead of thinking about being excited to determine what you can and can't hear and going about it in a intellectually honest manner. The only stressed people I ever see in a blind scenario is the person that knows at their core belief is a seed of conspicuous self delusion.

 

+1 Jud your subjective religion is just plain wrong and does nothing but a disservice to progress in the field of audio. All your fancy wordsmithing will never change the facts.

[Sal1950]

Empirical evidence is often what drives scientific investigation. To suggest that it has no place is, IMO, akin to sophistry.

Scientific investigation is what is called for, not unsupported opinions.

[miguelito]

Miggy this is too complicated for me

I know...

 

Could you explain this at my level - the cromagnon subhuman level ?

Simply out you should be able to measure it in the signal. That's it. If your input USB data has lots of jitter, the receiver chip will have to do a lot of work to fix it. The more work it needs to do the more current it draws from the power bus. Since power busses are not zero impedance this shows up elsewhere. As for single tone vs square, single tone in one frequency. Square is all frequencies - it's possible that higher frequencies get affected more hence the edges of the square would be deformed.

 

Makes sense?

[miguelito]

In a sense, there are. They're called confirmation bias.

Such thing exists. There's also real effects. It is also a bias to decide there can be no effect without testing for it.

 

We could also induce whatever differences are found by other means and let listeners determine which ones are audible. Then it would be possible to address those problems directly rather than making random stabs in the dark.

Consider the jitter example. I would say it's not the first thing one would've looked for and yet it turned out to be an important factor in sound quality - a reproducible important factor.

 

So my take is: if it consistently changes the sound then it must be measurable - what to measure might not be obvious but once isolated one should be able to add/remove the effect and get reproducible results. This is a clearly defined scientific method.

 

Now ultimately it always boils down to perception in a way since it is listening by which we decide what matters and what doesn't. That doesn't make it any less valid.

 

I liked the original article bc it said that just perception can lead you astray, but science (ie starting from an objectively flat response) plus a bit of tuning would render much better results than a random walk.

[Sal1950]

Big +1miguelito

[Jud]

+1 Jud your subjective religion is just plain wrong and does nothing but a disservice to progress in the field of audio. All your fancy wordsmithing will never change the facts.

 

Way to completely miss the point, Sal.

 

- I agree with Miguel (if you've been paying attention, you may have seen that).

 

- When you posted, what did I do? Say subjective impressions were king? As John Wayne once said, "Not hardly."

 

- What I did was ask *you* for some scientific evidence. You say it's not scientifically valid in the absence of an A/B/X test. Fine, as long as A/B/X tests themselves are a scientifically valid way of testing whether people can hear differences between two audio samples. So where is your scientific evidence in support of the validity of A/B/X testing?

 

- In other words, I'm being *more* demanding of objective evidence than you are. I need objective evidence the test is valid, while you apparently are willing to blindly - religiously? - accept its validity without such evidence.

[Sal1950]

Way to completely miss the point, Sal.

 

- I agree with Miguel (if you've been paying attention, you may have seen that).

 

- When you posted, what did I do? Say subjective impressions were king? As John Wayne once said, "Not hardly."

 

- What I did was ask *you* for some scientific evidence. You say it's not scientifically valid in the absence of an A/B/X test. Fine, as long as A/B/X tests themselves are a scientifically valid way of testing whether people can hear differences between two audio samples. So where is your scientific evidence in support of the validity of A/B/X testing?

 

- In other words, I'm being *more* demanding of objective evidence than you are. I need objective evidence the test is valid, while you apparently are willing to blindly - religiously? - accept its validity without such evidence.

 

ROTFLMAO. I got to hand it to you, you're very talented at writing this BS up.

[miguelito]

If it can't be measured, or at least proven with a properly conducted A-B-X listening test, it will always remain strictly an opinion. An opinion which has no place in any scientific investigation..

I think it depends on whether it is possible to spot a difference in A-B-X testing.

 

Let me refer to Chris's original article and the headache listening to his first iteration produced. I bet you a quick A-B-X test would not have spotted that he had a frequency peak up there. In fact maybe a quick test would have make him prefer the case with a peak because it sounds more "airy" in comparison. Clearly the opposite result.

 

The concept of "blind testing" is something I advocate, but clearly there are more than one way to do that. Like running your system with A for a week, then B for another week, then X for the next week.

 

I personally would say that I cannot spot specifics in a quick switch situation (unless it's blatantly obvious of course). In fact, most of the time I could not tell you why I prefer one setup over another, I generally realize I enjoy it better - that's it. And it takes me some time to realize this. In some cases after very many back and forths I will be able to focus on what it is that makes the difference, but that's rare.

[Superdad]

Why does our popular REGEN keep getting dragged into every subjectivist/objectivist argument? What it does has been both measured and heard, so why the controversy? Go pick on magic bricks or something.

 

(Don't worry, I'm not that naive. I'm just too tired tonight to weigh in with more substantive thoughts on the issue.)

[Sal1950]

I think it depends on whether it is possible to spot a difference in A-B-X testing.

 

Let me refer to Chris's original article and the headache listening to his first iteration produced. I bet you a quick A-B-X test would not have spotted that he had a frequency peak up there. In fact maybe a quick test would have make him prefer the case with a peak because it sounds more "airy" in comparison. Clearly the opposite result.

 

The concept of "blind testing" is something I advocate, but clearly there are more than one way to do that. Like running your system with A for a week, then B for another week, then X for the next week.

 

I personally would say that I cannot spot specifics in a quick switch situation (unless it's blatantly obvious of course). In fact, most of the time I could not tell you why I prefer one setup over another, I generally realize I enjoy it better - that's it. And it takes me some time to realize this. In some cases after very many back and forths I will be able to focus on what it is that makes the difference, but that's rare.

 

In Chris's case if he had done measurements from the very beginning there never would have been the mystery of the headache. That was exactly the point Chris was trying to teach us. Proper procedure would have been to design the speaker to first measure correctly and then tweak by ear to his satisfaction. It was believing he could depend solely on his ears and not following a correct scientific design procedure that sent him down a rabbit hole. ;-)

[mansr]

Why does our popular REGEN keep getting dragged into every subjectivist/objectivist argument? What it does has been both measured and heard, so why the controversy?

 

Nobody has proved that the measured differences are the cause of the perceived sonic improvements or even suggested a solid mechanism by which they could be.

[miguelito]

In Chris's case if he had done measurements from the very beginning there never would have been the mystery of the headache. That was exactly the point Chris was trying to teach us. Proper procedure would have been to design the speaker to first measure correctly and then tweak by ear to his satisfaction. It was believing he could depend solely on his ears and not following a correct scientific design procedure that sent him down a rabbit hole. ;-)

The point I am trying to make is the considering 1 minute A, 1 minute B, 1 minute X as the end-all be-all scientific method is misinformed and wrong. I think this is Jud's point as well.

[Jud]

Nobody has proved that the measured differences are the cause of the perceived sonic improvements

 

True.

 

or even suggested a solid mechanism by which they could be.

 

False.

[Jud]

ROTFLMAO. I got to hand it to you, you're very talented at writing this BS up.

 

Then just read what Miguel is writing, since it's the same thing as "this BS," and you seem to be a lot happier reading it from him.

[Jud]

 

The concept of "blind testing" is something I advocate, but clearly there are more than one way to do that. Like running your system with A for a week, then B for another week, then X for the next week.

 

 

I like that you put "blind testing" in quotes, because the object is not to be unaware of what one is listening to, but the removal of bias. Blinding is of course one effective way to do that.

 

A couple of ways to try to eliminate bias other than A/B/X:

 

- Let people listen sighted to something like speaker voicing or player software (where there are no visual cues), calling the choices something innocuous like "A" and "B." Do this with multiple test subjects. Have them fill out answers to questions about how the voicings sound, differences between them (if any), and preferences (if any). Don't allow the subjects to communicate with each other before filling out the answers. See if there is a consensus, greater than would be expected as a result of chance, about not only preferences but particular characteristics of sound.

 

- This is a blinded test: Use well recorded monaural music to feed one channel with choice A and the other with choice B. (To try to eliminate room effects, trials could be randomized between right and left channels, and/or headphones might be used.) This eliminates audio memory. Because with monaural music the "stereo effect" (sense that there is one source between the speakers rather than sound coming from each of them) depends on matching the two channels, this is extremely good at identifying differences.

 

As you say, Miguel, identifying preferences doesn't necessarily establish which of two choices is better from an accuracy viewpoint. Once the fact of an aural difference is established, we can try to identify the cause of that difference in the signal and attempt to measure it to determine which of the different choices is being more faithful to the original.

[miguelito]

As you say, Miguel, identifying preferences doesn't necessarily establish which of two choices is better from an accuracy viewpoint. Once the fact of an aural difference is established, we can try to identify the cause of that difference in the signal and attempt to measure it to determine which of the different choices is being more faithful to the original.

Yes. I would steer clear from the term "accuracy" though. The human ear and what "sounds live" is not necessarily flat frequency response. But any difference that one can repeatably hear should be measurable.

[mmerrill99]

 

or even suggested a solid mechanism by which they could be.

False.

Well, I see that the mechanism JS suggested is argued by John Westlake not to exist

 

- the crux of his argument is that there is no ramping up mechanism in the USB PHY that is needed to deal with signal integrity issues & therefore no extra current draw, no extra noise generated from the PHY. It's difficult to find detailed information on the PHY at this level of operation & what is the most prevalent method in the field at the moment, in order to evaluate which theory is correct.

 

 

JW claims that most USB PHY's now operate using an oversampling, clockless data recovery scheme which "When you oversample the input Data, SI Rise & fall times absolutely has no effect on internal circuit conditions, you just need the "Eye Opening" wide enough to recover the data, nothing internally has to "work harder" under poor SI conditions. With USB 2.0 there is no dynamic element unlike USB 3.0 which used a quite intelligent "active" receiver compensation circuits (like HDMI)."

 

Can anyone throw some light on this?

[Sal1950]

I like that you put "blind testing" in quotes, because the object is not to be unaware of what one is listening to, but the removal of bias. Blinding is of course one effective way to do that.

 

A couple of ways to try to eliminate bias other than A/B/X:

.

 

There is no need to re-invent the wheel, unless of course your trying to influence the outcome

Double-blind tests are the gold standard in every field of science!

[Superdad]

Nobody has proved that the measured differences are the cause of the perceived sonic improvements or even suggested a solid mechanism by which they could be.

 

For many years people said the same thing about jitter--but now everyone knows better.

[mansr]

For many years people said the same thing about jitter--but now everyone knows better.

 

The effect of jitter is readily measured at the analogue output. Maybe the effects of the REGEN are too, but so far nobody has been able to show it.

[plissken]

For many years people said the same thing about jitter--but now everyone knows better.

 

Actually people speculated until someone put together a properly setup test to eliminate bias and use the tried and true scientific method showed there was an audible difference.

 

Blinded testing is just a measurement tool like a scope, protocol analyzer, microphone, what have you.

 

I've never seen an audiophile get all lathered up over a mic and pink noise or frequency sweeps.

[plissken]

For many years people said the same thing about jitter--but now everyone knows better.

 

What finally happened though is someone setup scientifically rigorous testing and showed that jitter was measurable and blinded listeners could pick it out.

 

Blinded testing is a measurement tool just like any other. One should get no more bothered about removing bias than using a measurement mic.

[Jud]

What finally happened though is someone setup scientifically rigorous testing and showed that jitter was measurable and blinded listeners could pick it out.

 

I would be very interested in reading about blinded testing of jitter, if you have any references at all you can give, or even a recollection of where and when you might have read about it.

 

Blinded testing is a measurement tool just like any other. One should get no more bothered about removing bias than using a measurement mic.

 

Sure. On the flip side, anything that would make the mic less effective would be a concern, and similarly for anything that would affect the ability of the human listener to discriminate.

[Jud]

Double-blind tests are the gold standard in every field of science!

 

That's actually not correct.

 

Forum member wgscott (an actual scientist in real life) tried to make that point, among others, over at the Hydrogen Audio forums and was pretty well hounded out of them as a result. (I can't remember whether he was banned, or self-banned after talking-tos from HA admins.) And if I were to peg wgscott on an objectivist - subjectivist scale of members of this forum, he'd be well over on the objectivist side, so it isn't that he was trying to sell them some subjectivist claptrap. It's just that he wanted to mention what the actual role of double-blind tests in science is, and they felt he was questioning something that should be unquestionable.

 

A/B/X is not the only way to do double-blinded tests, and it may be far from the best way, depending on what you are testing for.

[mansr]

I would be very interested in reading about blinded testing of jitter, if you have any references at all you can give, or even a recollection of where and when you might have read about it.

 

Lots of references here: http://www.thewelltemperedcomputer.com/KB/BitPerfectJitter.htm