OK, I tried hydrogen audio. It didn't go too well.

[sullis02]

The Powers That Be - TPTB for short.

 

Well, for one, by using the "gasbag forum" instead of it's proper name, I inject a small amount of humor and more importantly, avoid giving the place free advertisement by putting the name into search engines.

 

Humor is a funny thing. So subjective. And typing 'HA' would hardly fire up the search engines.

 

One might wonder why one goes to such lengths not to. Did you have a Terrible, Horrible, No Good, Very Bad time at HA?

[Paul R]

Humor is a funny thing. So subjective. And typing 'HA' would hardly fire up the search engines.

 

One might wonder why one goes to such lengths not to. Did you have a Terrible, Horrible, No Good, Very Bad time at HA?

 

Just the opposite actually, I found some information I needed and made some good (online) friends from there. Doesn't mean I buy into the blowhard party line there. And "gasbag forum" just fits it so well, and on so many levels.

 

I do understand though. Humor is quite subjective, one has to develop the sense to recognize and appreciate it.

 

By the way, what makes you so sure that an ABX test can confirm or deny a listening experience?

 

That is a lot like saying that because you do not have any suspensions on your driver's license, you must be a really good driver. Or because you own a home you must be really good at managing money.

 

Like IQ, those may be indicators, but they really do not prove much of anything. Same is true of the results of an audio ABX test. It may be indicative, but it proves not much at all. (shrug)

 

-Paul

[sullis02]

Hi dennis, yes i fully agree except I think "fooled themselves" has the wrong connotations especially in a scientific setting where one does not have to have an expectation bias or axe to grind. Attribution of causation is as you know a very complex area.

 

Instead of 'fooled themselves accidentally', what? 'Convinced themselves wrongly'? 'Mistakenly believed'? In a scientific paper on audibility of difference, where the existence of expectation bias, and the need to nullify it, is a given, I'm not even sure the semantic issue would come up...standard biases would be understood as such.

 

I think IMO the whole testing for differences in musical perception is poor, as it is commonly done. There would need to be a "gold reference" standard testing procedure, for which I an unaware, that compares results to audio ABX DBT.

 

There are published standards from the IEC. There are of course also more than protocol type, depending on what you are testing for. Search this page for the word 'subjective' and you'll see some

Broadcasting service (sound)

 

There's also a large literature as well as 'standard texts'. Zwicker & Fastl is one such reference

Amazon.com: Psychoacoustics: Facts and Models (Springer Series in Information Sciences) (9783540231592): Hugo Fastl, Eberhard Zwicker: Books

 

Neither guarantee that a given researcher will follow 'best practice'. Though no paper will get published without some sort of blinding protocol.

[sullis02]

Just the opposite actually, I found some information I needed and made some good (online) friends from there. Doesn't mean I buy into the blowhard party line there. And "gasbag forum" just fits it so well, and on so many levels.

 

I do understand though. Humor is quite subjective, one has to develop the sense to recognize and appreciate it.

 

No, that's not *quite* the problem here.

 

 

By the way, what makes you so sure that an ABX test can confirm or deny a listening experience?

 

It can 'confirm' that a difference was heard, to a statistical level of 'certainty' at least. At can't 'deny' one so much as fail to provide any strong statistical support for it. And in point of fact, science never *absolutely* 'proves a negative'. But science doesn't really care about that, and neither do I. And I daresay, in your daily life, neither do you. It's theoretically non-zero possible that all the air in your room could bunch itself up into a corner, leaving you gasping. But I doubt you worry about it.

 

 

That is a lot like saying that because you do not have any suspensions on your driver's license, you must be a really good driver. Or because you own a home you must be really good at managing money.

 

Like IQ, those may be indicators, but they really do not prove much of anything. Same is true of the results of an audio ABX test. It may be indicative, but it proves not much at all. (shrug)

 

That's a lot of analogies, but no real critique. Why can't audio ABX prove 'much at all'?

[sullis02]

Double blind test is not exact tool itself. For avoiding delusion it must be performed correctly.

Many people believe in DBT. Need check every condition of this test.

 

What is an 'exact tool'? One that yields 0 or 100% probability answers?

 

How exact a tool is the standard operation procedure of audio hobbyists -- sighted listening?

[sullis02]

Well blind audio testing differs from blind medical testing in that objective verification of results is possible in some cases. The cancer disappeared or it didn't. The infection went away or it didn't. In audio you are relying on self reporting in a sense.

 

There is certainly once case where it can be as definitive in audio: the 'phantom switch'. In this instance, the subject believes he is comparing two different things, but in fact, is being presented only one thing. It can be most illuminating.

 

 

As very well done blind audio testing does get us near to the physical limits of possible sound

 

It gets us *to* the limit.

 

it looks plenty good enough when all is said and done. But if you could understand the brain and get objective verification without the interceder of the person choosing it might convince some not convinced. It also would allow testing not directly possible now. Plus if it were well enough developed it would be a lot nicer, more efficient and comfortable than the regimen of blind testing.

 

Yet it is, in the end, the subject who consciously 'reports' that they did or did not hear a difference, and it is *those* reports that drive all the Audio Debates. A change in blood flow in the brain does not prove the person 'heard' something consciously. And that puts you back at square one. Even Oohashi found this to be a muddle...*physiological* evidence not correlated to *self-reported* evidence.

 

 

 

Perhaps you are forgetting how upsetting it is to so clearly hear a difference in say cables. Only to have it collapse and disappear when blinded in the test. Not hard to see how someone has such a bizarre experience to think the test itself must have interferred.

 

Worse, I have accidentally fallen prey to a 'phantom switch'. I swore I was hearing a difference between two recordings I'd made, then realized that I hadn't actually even switched between the two.

 

If anything, those experiences should give lie to the subjectivist mantra 'trust your ears' like nothing else.

 

If you could get something like brain scans or other methods to just quietly monitor activity and detect differences it would be another step in convincing someone. I am all for finding ways to convince without someone having to become an EE, research scientist and blind testing afficionado to get it. Although in a minor sense that is what it took for me.

 

Inferring causes and effects from brain scans is a quite a fraught area. I don't think you;'d get the certainty you think. You need statistics to interpret brain scans too.

[Paul R]

No, that's not *quite* the problem here.

 

Actually, yes, it is exactly the problem.

 

It can 'confirm' that a difference was heard, to a statistical level of 'certainty' at least. At can't 'deny' one so much as fail to provide any strong statistical support for it. And in point of fact, science never *absolutely* 'proves a negative'. But science doesn't really care about that, and neither do I. And I daresay, in your daily life, neither do you. It's theoretically non-zero possible that all the air in your room could bunch itself up into a corner, leaving you gasping. But I doubt you worry about it.

[/Quote]

 

Confirm how exactly? Positive results do not actually confirm that a difference exists, only that the test subject heard a difference. Which according to the vast number of theories bandied about, could be because of expectation bias or some other equally unlikely reason.

 

Not hearing a difference is, again, suggestive. But all it does confirm is that the test subject did not hear a difference. Which according to an equally unlikely pack of theories bandied about, could be because of any number of other reasons.

 

You seriously do not see the issue here? In fact, neither sighted listening tests nor blind listening tests as advocated in ABX style testing can tell us much of anything more than the listener himself can, without all the testing. Put them together in a reasoned and well designed test, and you might start to get some answers. Positive, negative, who cares?

 

Either side by itself is incomplete, and indeed, there may need to be a third, fourth, or even fifth type of testing involved to get answers any better than a person listening in a sighted environment will provide. That one sounds different from that other one over there, etc.

 

 

That's a lot of analogies, but no real critique. Why can't audio ABX prove 'much at all'?[/Quote]

 

Or it is simply that you didn't think deeply enough about the examples? Both examples are contentious, both are hard to prove. The merger mention of IQ can start riots in the street, and yet IQ s the single most effective predictor we have for success at this time.

 

Real differences, exemplified by analysis of IQ scores and correlated with other factors provides suggested conclusions that are utterly unacceptable to many people, and result in automatic denial. Truth, fact, and the scientific method not withstanding.

 

Perhaps you might think on that a little more, and examine how the examples given are both similar to and different from the controversy fueled so well by pseudoscience over on the gasbag forums in regards to what is and is not audible.

 

-Paul

[mmerrill99]

A negative control is same-vs-same, which is built-in to audio DBT. Positive control -- some level difference that *should* just be audible -- is advisable for complete rigor but needs work to determine it for each listener.

I have never seen these controls used in the typical audio DBTs that we see on forums. The controls are hidden anchors used within the test to examine the whole test (including it's participants) & any tendency towards false positive or false negative results. It's really not about rigour & more about proving the validity of the test & it's participants to be capable of hearing to the level of acuity warranted by what is being tested. After all, if you have untrained listeners who don't know what certain distortions sound like then the likelihood is that a null result will be the outcome.

 

The thing is, though, in most cases what is happening (for DBTS on HA , where there has been an historical focus on lossy codecs) is not testing some universal proposition, but instead this:

 

1) listener listens to two files, sighted, and thinks he hears a difference.

2) fires up ABX software to compare them

3) ABX does or does not confirm the difference

 

This is what most of us do, if we ABX. We aren't testing whether it's possible for someone else, somewhere, to hear a difference (an academic study). We are testing our immediate experience-- was what we just heard (sighted), probably real or not? That doesn't need training, or positive controls -- we are simply repeating our listening , only this time, making it blind. The result tells us whether we, at our current level of training, are likely to have heard a real difference.

That's not what I see happening on HA - by & large, positive ABX results are met with suspicion & lately accusations of gaming the results ( & calling into question the honesty of the poster) have been used to dismiss such results. Not really the benign, open-minded atmosphere you are portraying.

[esldude]

Hi dennis, yes i fully agree except I think "fooled themselves" has the wrong connotations especially in a scientific setting where one does not have to have an expectation bias or axe to grind. Attribution of causation is as you know a very complex area.

 

 

 

 

not 100% sure what is meant here in relation to audio perception DBT

 

 

 

I think IMO the whole testing for differences in musical perception is poor, as it is commonly done. There would need to be a "gold reference" standard testing procedure, for which I an unaware, that compares results to audio ABX DBT. The latter could then be refined in terms of methodology to better match results of the gold standard. In medicine this happens all the time such as with measuring things like insulin sensitivity.The gold standard is a complicated research tool/test procedure but there are many 'field' or clinical tests that have been used as a practical alternative.These latter tests have been validated to whatever extent with the gold standard.

 

I agree about fooling themselves being a bad term. Yet I have discovered none that are effectively better. I would preface fooling themselves by saying I don't mean they were gullible or anything. Simply subject to certain perceptive problems that all humans have. Yet, listening under poorly controlled conditions, mistakenly atrribute quality differences to something not real, and then finding out they were not real. Fooled themselves, made a mistake, whatever.

 

I think Sullis has already described the controls as I would have. Negative control is playing the same exact signal and seeing if you get something other than 50/50 results. If you do get one sided picks it indicates a problem with your method or setup or other conditions. Positive control is something non-obvious yet known to be audible. A .5 db level difference is not noticed as a loudness change, but as a quality difference quite reliably. If that signal gets a null result then your test conditions are not good enough to uncover real, but small differences.

 

Common complaint that ABX is not good for testing musical differences. It is demonstrably very good at determining sound quality differences. Differences that are measurable in other ways. Musical perception of two signals known to be exactly the same in those other measuable ways reliably get null results as you would expect. People claim to hear was is not there by any other measure, and ABX also shows no difference. Which should we believe? I am crudely simplifying, but we have three ways to determine if something sounds different. Two of those ways agree almost always, the third seems to return near random results and half the time does not agree. Again which should we believe?

[sullis02]

Actually, yes, it is exactly the problem.

 

Actually, that's just, like, your opinion, man.

 

Confirm how exactly? Positive results do not actually confirm that a difference exists, only that the test subject heard a difference. Which according to the vast number of theories bandied about, could be because of expectation bias or some other equally unlikely reason.

 

Your beef seems to be with scientific standards of proof. Also some serious misunderstanding, since DBT (of which ABX is one species) exists in order to circumvent expectation (and other) cognitive biases. Given that, I'm not really sure there's any productive value in arguing further.

[sullis02]

I have never seen these controls used in the typical audio DBTs that we see on forums. The controls are hidden anchors used within the test to examine the whole test (including it's participants) & any tendency towards false positive or false negative results. It's really not about rigour & more about proving the validity of the test & it's participants to be capable of hearing to the level of acuity warranted by what is being tested. After all, if you have untrained listeners who don't know what certain distortions sound like then the likelihood is that a null result will be the outcome.

 

Yes, but you are leaving out the other part: what does that say about the untrained listener's claim that they DID hear a difference? Do you find it likely to be true?

 

This is the real world: audiophiles claiming , in all sorts of situations, that they heard a clear difference between A and B. Claiming, in effect , to *already* possess enough training to hear that difference. Do we need to 'train' *them* further to test that proposition? Do we need elaborate controls? If the 'night and day' differences they claim , are not supported in an ABX, do you conclude that teh ABX must bee flawed?

 

(And btw, the recent Meridian test of digital filter audibility, touted as 'proof' , did not include all the controls you want.)

 

 

 

That's not what I see happening on HA - by & large, positive ABX results are met with suspicion & lately accusations of gaming the results ( & calling into question the honesty of the poster) have been used to dismiss such results. Not really the benign, open-minded atmosphere you are portraying.

 

Ah, I guess you're talking about the amazing Amir and his positive ABX's of redbook vs hi rez. (In fact, positive ABXs pass through HA fairly often, with little pushback -- again, you must recognize that historically the focus there was lossy codecs).

 

The issues there are not just his rather offputting personality, and his endless diversionary arguing, but whether 'Amir heard a difference' means 'what audiophiles report when they claim Redbook and hi rez sound different, is what Amir heard". Amir claims a high level of ear-training, and is very cagey about just how he manages to detect particular difference -- what 'tell' he used. As such it is very difficult to extrapolate such results to the common real-world case of 'Joe Audiophile says hi rez lifted the veil from his system, even his wife could hear it!'.

 

And isn't *that* what we really want to know?

 

It's not hard to devise conditions under which Redbook and hi rez can be audibly different (e.g., boost the volume of a fadeout, where hi rez had been truncated to redbook). But are those conditions the ones that 'audiophiles' are hearing? Do they 'explain' the all-too-common claims audiophiles make, in forums like this, in the pages of Stereophile and TAS?

 

The pushback is against the too broad*application* of a positive ABX result, coupled with Amir's proselytizing for hi rez as some sort of savior for audio (he runs a boutique hi-end audio and room design dealership). If you *have* really followed all those threads , you have seen that the critiques do not simply boil down to 'he's cheating' . Read 2bdecided's posts for example. Read *mine* (krabapple).

[esldude]

I don;t think I misunderstood, so much as tried to keep the focus on what is right or wrong with HA. I don't really care what other forums do (I'm familiar enough with several of them to know already what they do)

 

 

 

A negative control is same-vs-same, which is built-in to audio DBT. Positive control -- some level difference that *should* just be audible -- is advisable for complete rigor but needs work to determine it for each listener.

 

The thing is, though, in most cases what is happening (for DBTS on HA , where there has been an historical focus on lossy codecs) is not testing some universal proposition, but instead this:

 

1) listener listens to two files, sighted, and thinks he hears a difference.

2) fires up ABX software to compare them

3) ABX does or does not confirm the difference

 

This is what most of us do, if we ABX. We aren't testing whether it's possible for someone else, somewhere, to hear a difference (an academic study). We are testing our immediate experience-- was what we just heard (sighted), probably real or not? That doesn't need training, or positive controls -- we are simply repeating our listening , only this time, making it blind. The result tells us whether we, at our current level of training, are likely to have heard a real difference.

 

Which also means (at least theoretically) that with training, perhaps we *could* score significantly on an ABX. But if so, that still wouldn't mean that we 'really' heard the difference the first time.

 

These are all niggling points in a way; but they play into the need for proper interpretation of results.

 

(And for actual lossy coded DBTs, if audible differences is *expected* -- e.g., low bitrates -- then the appropriate DBT is not ABX, but something like ABC/hr, where *preference* is being tested)

 

 

 

 

 

But always keep in mind that typically, the audiophile *does* claim to hear a difference, sighted. At which point much of the objection to 'crudeness' goes by the wayside. If all you have done is 'blind' that listener -- obscure the identity of the source --and suddenly the difference 'goes away' then that says something.

 

IOW, we aren't talking about a lab study, where one is investigating the possibility of whether *anyone* can hear difference between A and B. We are testing a particular subject's claim.

 

 

 

 

An 'ABX type test' *can* be stressful but it's not a requirement that it be so. (typoically it's 'stressful' when the differences are....subtle or nonexistant). Not clear why 2AFC would be less so; I'm not even clear on how it could be adapted to audio difference DBT. Can you point me to to the discussion at HA?

 

 

 

 

Could be that 'the atmosphere' has to do with the fair number of gunslingers who show up and *do* claim that ABX, DBT, Shannon-Nyquist, whatever, are all wrong.

 

Can you point me to the exchanges you describe above?

 

sullis02,

 

Several of your posts give me the impression you are speed-reading or scanning other people's replies. With heavy slant on what you are looking for. You have more than once replied in a way that didn't really fit.

 

In your most recent reply to me. You first read incorrectly, that some of my comments were about HA forums. I then point out that no they were about forums other than HA. You then reply you don't care about other forums you are responding to what happens on HA. Dude, slow down, read what someone says in some reasonable context, and then reply. The number of posts to communicate something that way will be about half as much.

 

You then respond, just like the majority of people did about my comments on ABX DBT at HA. Maybe you all are right and I am wrong. You first say negative control is built into DBT by its nature. Sorry, wrong on that one. Just ask JJ. You then explain how in casual use you don't need positive controls. JJ disagrees with that also. There is an additional reason beyond just making DBT more reliable even in casual use. Among subjectivists is the idea DBT always returns negative results. It of course is in no way true. One way to allay that concern is if those guys take part in blind testing is for them to experience it. To include a positive control. For them to hear something they hardly perceive as different yet reliably detect it. To show them through their experience that blind tests do detect small differences.

 

At least a handful of times I have provided some simple files with a minimally detectable difference. To then have people accurately detect it, but tell me how surprisingly hard it was. To tell me cables, or DACs or some other effect was much larger. If they then test those other things so much more easily perceived only blindly, and find they suddenly can't, you can get through to people that way. Talking otherwise or testing the list of other things first which they fail to detect almost universally fails to sway anyone. They just think the blind testing thing is really no good.

 

As for ABX being stressful or not I agree with you. It need not be stressful. Two things help, one is not always possible. If you aren't taking it as a test of your audiophile manood it will be far less stressful. The other is some experience, some familiarity with it. Having taken a few, gotten comfortable (maybe gotten some interesting positive results), simply be at ease with the procedure helps with stress. It doesn't change the tedium and eventual boredom, but it helps with any stress.

 

So, your attitude is much like I see on HA. We know what we are doing, we are right, no need to accommodate subjectivist complaints about it yadda, yadda, yadda.....

 

All well and good unless you might hope to convince some that don't believe you or agree with you. By making some concessions or addressing complaints I am not saying anything is wrong with the ABX method. And as we are not doing genuine research, a few caveats to allay concerns even if the concerns are groundless seem like a good idea to me. That is unless the desire to be exactly right, correct and those other guys are wrong is awfully strong.

 

As for the atmosphere at HA being due to gunslingers showing up saying all this is wrong, I am certain that is lots of it. Been guilty of that type of reaction to these groundless claims myself far too often. Seeing how things go in other forums than HA, despite you saying you don't care (and still here you are on CA), certainly leaves one prone to it. But HA is a well established community at this point. If there is anywhere you could drop that attitude it should be there.

[Paul R]

Actually, that's just, like, your opinion, man.

 

 

 

Your beef seems to be with scientific standards of proof. Also some serious misunderstanding, since DBT (of which ABX is one species) exists in order to circumvent expectation (and other) cognitive biases. Given that, I'm not really sure there's any productive value in arguing further.

 

(*sigh*) As you say, that is your opinion, and you are welcome to it, as nearsighted, warped, and emotional as it is. It is also just wrong, IMNSHO.

 

You example the problems so endemic on the gasbag forum. Any challenge at all to what you believe is the pinnacle of reasoned thought is met with enthusiastic hostility. Pretty much like undergrads who think they just figured out the last ultimate mystery of the universe.

 

I strongly suggest that your understanding of what the ABX tests on the gasbag forum reveal or do not reveal is incomplete. You need to think about it more.

 

-Paul

 

[Edit] Go back and read Dennis' post again. Just because Dennis and I disagree (often) on some conclusions or the meaning of some evidence, doesn't mean we disagree often or even seriously on methodology. And he just posted a really good example there. -PR

[Don Hills]

... Well, for one, by using the "gasbag forum" instead of it's proper name, I inject a small amount of humor and more importantly, avoid giving the place free advertisement by putting the name into search engines. ...

 

"Gasbag" is a derogatory term where I come from. If you're using it to express your opinion of the HA forums, carry on. If you merely wish to avoid the search engines, I submit that "HA" is as effective and avoids the negative connotation.

[sullis02]

(*sigh*) As you say, that is your opinion, and you are welcome to it, as nearsighted, warped, and emotional as it is. It is also just wrong, IMNSHO.

 

My aren't you the charmer? No love for the Big Lebowski?

 

At least you didn't try to 'prove' I have no sense of humor (though that would have been entertaining).

 

You example the problems so endemic on the gasbag forum. Any challenge at all to what you believe is the pinnacle of reasoned thought is met with enthusiastic hostility. Pretty much like undergrads who think they just figured out the last ultimate mystery of the universe.

 

I guess you think CA differs from HA in that regard. Your posts suggest otherwise.

 

 

I strongly suggest that your understanding of what the ABX tests on the gasbag forum reveal or do not reveal is incomplete. You need to think about it more.

 

As I already strongly suggested that you don't understand ABX testing, I guess we're even.

[sullis02]

sullis02,

 

Several of your posts give me the impression you are speed-reading or scanning other people's replies. With heavy slant on what you are looking for. You have more than once replied in a way that didn't really fit.

 

 

In your most recent reply to me. You first read incorrectly, that some of my comments were about HA forums. I then point out that no they were about forums other than HA. You then reply you don't care about other forums you are responding to what happens on HA. Dude, slow down, read what someone says in some reasonable context, and then reply. The number of posts to communicate something that way will be about half as much.

 

 

It's tiresome for us to keep telling each other we are misreading. Rather than an other round of that, let's just agree to disagree on it.

 

 

JJ simply doesn't concern himself with 'casual use'. He is interested in lab-quality results. Nothing wrong with that. I certainly don't *object* to negative and positive controls. I'm just saying that if the subject *already* claims to hear A vs B, the results of a simple act of blinding should be convincing enough for them (if not for others), no?

 

 

(And btw, if you accept JJ's authority, what do you make of claims made here at CA that you must know he would find dubious at best, if not outright laughable?)

 

 

Subjectivist 'complaints' often run a repetitive gamut, points which have been addressed over and over (ABX is too short/too long/wrong equipment/wrong test signals/subjects were insensitive etc.) It's rare that something new under the sun appears. Putting that all to one side, I simply ask: you say you already hear a difference when you compare A to B. What happens when you compare them blind?

 

 

Anyway, though you ignored my request for it, I found an HA thread where you suggested 2AFC.

Other Listening Test Methodologies? - Hydrogenaudio Forums

 

 

First, it appears you had already raised the same point a decade earlier and received several thoughtful replies then, e.g. from ff123.

Blind tests and HydrogenAudio - Hydrogenaudio Forums

 

 

In the newer case, you received a couple of responses that raised good points; you took objection to one or two of them; response was given; you never replied. And your blunt assertion that 2AFC is 'best' for preference testing needs some more detailed backup; see for example possible problems when an attribute unders test affects several others https://books.google.com/books?id=w4hwAgAAQBAJ&pg=PA66&lpg=PA66&dq=2afc+food&source=bl&ots=aC5YK3wCxB&sig=nXdze8QfkweETPXh4SyhOmRGbL4&hl=en&sa=X&ei=5TejVO6rO4eWyQTQw4HIBg&ved=0CDUQ6AEwBg#v=onepage&q=2afc%20food&f=false

 

(Also suggest searching the phrase 'forced choice' on HA, many discussions are returned)

[esldude]

It's tiresome for us to keep telling each other we are misreading. Rather than an other round of that, let's just agree to disagree on it.

 

 

JJ simply doesn't concern himself with 'casual use'. He is interested in lab-quality results. Nothing wrong with that. I certainly don't *object* to negative and positive controls. I'm just saying that if the subject *already* claims to hear A vs B, the results of a simple act of blinding should be convincing enough for them (if not for others), no?

 

 

(And btw, if you accept JJ's authority, what do you make of claims made here at CA that you must know he would find dubious at best, if not outright laughable?)

 

 

Subjectivist 'complaints' often run a repetitive gamut, points which have been addressed over and over (ABX is too short/too long/wrong equipment/wrong test signals/subjects were insensitive etc.) It's rare that something new under the sun appears. Putting that all to one side, I simply ask: you say you already hear a difference when you compare A to B. What happens when you compare them blind?

 

 

Anyway, though you ignored my request for it, I found an HA thread where you suggested 2AFC.

Other Listening Test Methodologies? - Hydrogenaudio Forums

 

 

First, it appears you had already raised the same point a decade earlier and received several thoughtful replies then, e.g. from ff123.

Blind tests and HydrogenAudio - Hydrogenaudio Forums

 

 

In the newer case, you received a couple of responses that raised good points; you took objection to one or two of them; response was given; you never replied. And your blunt assertion that 2AFC is 'best' for preference testing needs some more detailed backup; see for example possible problems when an attribute unders test affects several others https://books.google.com/books?id=w4hwAgAAQBAJ&pg=PA66&lpg=PA66&dq=2afc+food&source=bl&ots=aC5YK3wCxB&sig=nXdze8QfkweETPXh4SyhOmRGbL4&hl=en&sa=X&ei=5TejVO6rO4eWyQTQw4HIBg&ved=0CDUQ6AEwBg#v=onepage&q=2afc%20food&f=false

 

(Also suggest searching the phrase 'forced choice' on HA, many discussions are returned)

 

Yes, I think JJ is an authority. I prefer to not rely on authority, but he is a good one in the field. I think the claims here are laughable just about as much as JJ does. That shouldn't be considered a dismissal of respect for the people who put credence in those claims. Once again you seem to imply I must find credence in such things on CA just by my participation here.

 

About the only point I disagree with you about is trying to address complaints from 'the other side' when possible. I have already said even the crudest blind test is a better attempt to understand than sighted listening. Maybe the quest to get through to subjectivists is tilting at windmills. Trying to find enough common ground to communicate useful information isn't all black and white, but I don't consider it always a waste of time. On the other hand, catch me in a different mood, and seeing repeated false complaints for the umpteenth time that have already been addressed I might be exactly in agreement with you.

[wgscott]

[ATTACH=CONFIG]15997[/ATTACH]

 

Pay up.

[Paul R]

"Gasbag" is a derogatory term where I come from. If you're using it to express your opinion of the HA forums, carry on. If you merely wish to avoid the search engines, I submit that "HA" is as effective and avoids the negative connotation.

 

I submit that if someone names their forum with "Hydrogen" as a keyword, they best have a good sense of humor. Jokes about explosive releases of gas and lightweight arguments, as well as containers to hold the gas are inevitable.

[Paul R]

My aren't you the charmer? No love for the Big Lebowski?

 

At least you didn't try to 'prove' I have no sense of humor (though that would have been entertaining).

 

 

 

 

I guess you think CA differs from HA in that regard. Your posts suggest otherwise.

 

 

 

 

As I already strongly suggested that you don't understand ABX testing, I guess we're even.

 

Hardly- so far as I can see, you have barely begun. You cling to some ideas, but do not truely understand them or their implications. Nor can you competently defend those ideas.

 

As I said, Dennis and I have quite different opinions about the subject of ABX and DBT testing in the audio field.

 

He perhaps, feel they are the last word and is determined to convert people to his point of view. I feel they are incomplete without sighted testing and evaluation. That dispute will probably continue for the forseeable future. Both of us have enough experience to realize at least some truths in the area.

 

However, even when he is at his infuriating best, one must respect him. Disagree certainly, but still respect him. You would do well to look at him and understand why. One could do even better perhaps to study many of the participants here, almost all of whom hold degrees and experience enough to impress anyone. Bill

Scott for example.

[esldude]

Pay up.

 

Yes, yes I must attribute that reference to you. It was the picture I had in mind when I posted the phrase as it seemed most appropriate. I won't claim great minds think alike. Only that even a small mind can copy great ones effectively sometimes.

[esldude]

Hardly- so far as I can see, you have barely begun. You cling to some ideas, but do not truely understand them or their implications. Nor can you competently defend those ideas.

 

As I said, Dennis and I have quite different opinions about the subject of ABX and DBT testing in the audio field.

 

He perhaps, feel they are the last word and is determined to convert people to his point of view. I feel they are incomplete without sighted testing and evaluation. That dispute will probably continue for the forseeable future. Both of us have enough experience to realize at least some truths in the area.

 

However, even when he is at his infuriating best, one must respect him. Disagree certainly, but still respect him. You would do well to look at him and understand why. One could do even better perhaps to study many of the participants here, almost all of whom hold degrees and experience enough to impress anyone. Bill

Scott for example.

 

Thanks Paul for the kind comments. This respect goes both ways.

[wgscott]

Yes, yes I must attribute that reference to you. It was the picture I had in mind when I posted the phrase as it seemed most appropriate. I won't claim great minds think alike. Only that even a small mind can copy great ones effectively sometimes.

 

I had a gf who used to say "soft minds run together."

[audiventory]

What is an 'exact tool'? One that yields 0 or 100% probability answers?

 

How exact a tool is the standard operation procedure of audio hobbyists -- sighted listening?

 

I'm affraid 100% exact tool don't exist in the world

 

Correct experiment is tough not only for hobbyists, but for pro audio.

 

Double blind test demand concidering of all details:

1. What you compare?

2. What test environment and conditions?

3. How goal 2 impact to goal 1?

 

What need to execute the details?

1. Enough deep know theory of subject of comparision

2. Know practical particularity of used equipment and software

3. Be pedantic to methodics, equipment, yourself.

 

Audio hobbist can learn the subject and with time can become pro.

 

Always need have open mind (for hobbist and for pro), and re-check yourself everytime.

[mmerrill99]

Yes, but you are leaving out the other part: what does that say about the untrained listener's claim that they DID hear a difference? Do you find it likely to be true?

No more true or untrue than badly organised blind tests - actually, correct that, if one is judicious about whose sighted test results one accepts then I find these results to be far more reliable than such unqualified blind tests

 

This is the real world: audiophiles claiming , in all sorts of situations, that they heard a clear difference between A and B. Claiming, in effect , to *already* possess enough training to hear that difference. Do we need to 'train' *them* further to test that proposition? Do we need elaborate controls? If the 'night and day' differences they claim , are not supported in an ABX, do you conclude that teh ABX must bee flawed?

If there is enough sighted reports, some from those whose opinion I trust & I have verified the sighted results myself, then yes, I distrust an ABX delivering contrary results & figure it is flawed. I especially distrust any blind test that hasn't included any positive/negative controls & wonder why they are not done seeing as such blind tests are held up as of more veracity than sighted tests. So, can you answer me, why these controls are not included - they are relatively easy to include in such tests. Would it not further cement the validity of such tests? Is there a fear that most tests run would be seen as not capable of differentiating differences at the level claimed?

 

(And btw, the recent Meridian test of digital filter audibility, touted as 'proof' , did not include all the controls you want.)

I find it interesting that one should cite others tests as a reason for not including such controls. Is there some problem you have with using these controls yourself? From your earlier post you seemed not to be aware of what controls were being spoken of - which is not really unusual for supporters of such blind testing. It always comes across that the focus of such tests is based on the notion that "people are fooling themselves & we will prove it" - great concern is given to the "blind" part of the test & little consideration given to the actual validity/accuracy of the test itself.

 

 

Ah, I guess you're talking about the amazing Amir and his positive ABX's of redbook vs hi rez. (In fact, positive ABXs pass through HA fairly often, with little pushback -- again, you must recognize that historically the focus there was lossy codecs).

 

The issues there are not just his rather offputting personality, and his endless diversionary arguing, but whether 'Amir heard a difference' means 'what audiophiles report when they claim Redbook and hi rez sound different, is what Amir heard". Amir claims a high level of ear-training, and is very cagey about just how he manages to detect particular difference -- what 'tell' he used. As such it is very difficult to extrapolate such results to the common real-world case of 'Joe Audiophile says hi rez lifted the veil from his system, even his wife could hear it!'.

I have seen very few positive ABX results reported on HA & some push back is always evident when it occurs. Can you give your examples of the often positive ABX results reported?

As regards Amir, I find the fact that his honesty was questioned (& continues to be questioned) to be an outrage & in fact reduces ABX test to nothing more than a circus - the accusation of dishonesty/gaming is always possible for any future positive results.

Amir has given the "tells" & timestamps for his listening so I don't know what you are talking about - seems more irrational argument designed to reject results you don't like. Anyway, what does it matter whether he tells what he hears - does it negate his results or are you being disingenuous & slyly suggesting he is being dishonest & finding differences some other way?

 

And isn't *that* what we really want to know?

No, it's not the point - you fail to understand what testing is about! If you want to test what the common man can audibly differentiate then you need a different type of test. What this test (designed by ArnyK) was designed to do was test if there was an audible difference between high-res & RBCD (if anybody could differentiate between high-res & RB recordings). And Amir (& others) passed it without much difficulty. Remember, this is a test that stood for 15 years or so with no positive results & therefore I expect that it was smugly accepted that case proven, QED So when some positive results are reported it is probably natural for a knee-jerk reaction against it but this push back has been going on long enough for people to have left their initial emotional state & reverted to a more considered opinion. Still we see the pushback so I guess this is the considered opinion.

 

It's not hard to devise conditions under which Redbook and hi rez can be audibly different (e.g., boost the volume of a fadeout, where hi rez had been truncated to redbook). But are those conditions the ones that 'audiophiles' are hearing? Do they 'explain' the all-too-common claims audiophiles make, in forums like this, in the pages of Stereophile and TAS?

Sorry, don't know what you are talking about - is ArnyK's test flawed so obviously?

 

The pushback is against the too broad*application* of a positive ABX result, coupled with Amir's proselytizing for hi rez as some sort of savior for audio (he runs a boutique hi-end audio and room design dealership). If you *have* really followed all those threads , you have seen that the critiques do not simply boil down to 'he's cheating' . Read 2bdecided's posts for example. Read *mine* (krabapple).

Ah, you are krabapple, that explains a lot. I find your attitude on HA abominable & shameful without a shred of anything to excuse it. What Amir is saying is simple - he wants the file delivered in the samplerate that it was recorded in - no down sampling to RBCD (& the possibility of messing this up) - if it was recorded in 16/44 then fine