A different take on ABT testing

[sdolezalek]

I spent a number of hours last week trying to decide whether I was joining the "on-the-fly upsample to DSD" fan club. One of the things I discovered was that I had a very hard time telling the difference in rapidly switching back and forth; but if I was willing to just listen to one format (all upconverted vs all not) over a period of at least 30 minutes, I found that I could really begin to hear differences that were not obvious when rapidly switching back and forth.

 

This wasn't a true blinded test, but the results made me wonder whether our ears/brain require some time to fine-tune what they are hearing? Most obvious to me was that with the upconverted to DSD material I kept wanting to listen to more as I started hearing more fine detail whereas on the non-upconverted those details never seemed to show up as I listened longer.

 

Anyone else come to this "it takes longer than a couple of minutes switching back and forth" conclusion?

[Speedskater]

You might use that little triangle at the bottom left of your post and ask the moderators to edit your title typo to "ABX".

 

Experenced and skilled ABX testers, first listen to the test samples for an extended period of time to get familial with them and find their weak points. Only then do they do the real ABX test. A good tester will use very short samples (a few seconds) and quickly switch back and forth.

[kumakuma]

The subject was recently beaten to death in this thread:

 

http://www.computeraudiophile.com/f8-general-forum/should-objective-oriented-computer-audiophile-members-stop-asking-dbt-test-results-computer-audiophile-20848/

[beanbag]

You can take as long as you want before switching. Depends on whether you want to base your comparisons on memory or impressions.

[mayhem13]

I spent a number of hours last week trying to decide whether I was joining the "on-the-fly upsample to DSD" fan club. One of the things I discovered was that I had a very hard time telling the difference in rapidly switching back and forth; but if I was willing to just listen to one format (all upconverted vs all not) over a period of at least 30 minutes, I found that I could really begin to hear differences that were not obvious when rapidly switching back and forth.

 

This wasn't a true blinded test, but the results made me wonder whether our ears/brain require some time to fine-tune what they are hearing? Most obvious to me was that with the upconverted to DSD material I kept wanting to listen to more as I started hearing more fine detail whereas on the non-upconverted those details never seemed to show up as I listened longer.

 

Anyone else come to this "it takes longer than a couple of minutes switching back and forth" conclusion?

 

I've found you have to be very mindful of fatigue and the order of material and which appears to be the preference. Just like staring causes eye strain, critical listening is tiring. I've found myself nearly asleep at the wheel behind the board many a time.

[sdolezalek]

Oops typed too fast, thanks for the correction.

 

Your response re "experienced and skilled ABX testers" certainly doesn't describe me, but I also didn't realize that these tests involved "experts" as opposed to using whomever was available.

[sdolezalek]

The subject was recently beaten to death in this thread:

 

http://www.computeraudiophile.com/f8-general-forum/should-objective-oriented-computer-audiophile-members-stop-asking-dbt-test-results-computer-audiophile-20848/

 

Yes, I read that thread from end to end, but it focused broadly on the validity/usefulness of ABX testing. My question was more specific as to whether what our ears heard varied as a function of time.

[kumakuma]

Yes, I read that thread from end to end, but it focused broadly on the validity/usefulness of ABX testing. My question was more specific as to whether what our ears heard varied as a function of time.

 

I think Teresa mentioned several times in this thread that she had reached the same conclusion as you.

[sdolezalek]

I think Teresa mentioned several times in this thread that she had reached the same conclusion as you.

 

Now that I found it. Here is what Teresa said: Quoting Prof. Johnson: "Play a bad recording and the ears will shut down. Now go to a good recording and you have to wait for the feedback to start recovering and process. That means you can’t A-B something by flipping a switch. That isn’t going to happen. You got to wait. And it means that if I play a really crummy recording… then you have to wait a long time for the ears to settle down and be back to a perceptual mode again.  At that point you can go into “sonic nirvana” which is the visual acoustic."

 

"Prof." Keith O. Johnson chief engineer and technical director of Reference Recordings

 

I guess I totally agree...

[Audio_ELF]

It really depends what you are trying to do...

 

Your description of listening and making a choice is the correct way to go about finding your preference between two playback systems.

 

What your listening / testing methodology does not provide is any kind of proof that two very close signals are different (IMO).

 

Eloise

[RealAudio]

 

This wasn't a true blinded test, but the results made me wonder whether our ears/brain require some time to fine-tune what they are hearing? Most obvious to me was that with the upconverted to DSD material I kept wanting to listen to more as I started hearing more fine detail whereas on the non-upconverted those details never seemed to show up as I listened longer.

 

Anyone else come to this "it takes longer than a couple of minutes switching back and forth" conclusion?

 

Thank you for demonstrating why double blind testing has been the standard since the late 1700s. See what happens if you don't know the signal source. "wanting to listen to more" could be your preconceived notions that DSD is better making you search for justification.

[4est]

Welcome to the dark side! I can only catch larger differences if I rapidly switch, and then I only have about 5-6 tries before I get "confused".

Anyone else come to this "it takes longer than a couple of minutes switching back and forth" conclusion?

[Paul R]

Whoo boy- I hope you have on your lead underwater.

 

Yes, what you found matches up pretty closely with what accredited researchers have found for the past 60 years or so, when working with perceptual testing. You might try limiting the back and forth switches to two, maybe three times, but note your first impression, as long term listening will later probably validate it.

 

Now expect to be jumped on by people who say you have to train yourself to be a good ABX tester. It's possible of course, but rather defeats the whole purpose. In the Navy, rapid switching will easily confuse young AW ratings enough they can't tell the acoustic signature of a merchant from a more interesting target. At least not for an hour or two after such "tests." And those acoustics are far more different than DSD and PCM.

 

I know a couple people who have trained themselves so well they might as well be deaf, getting no enjoyment from the music at all. (*sigh*). ABX is pseudo-scientific twaddle the way it is used by some folks, right up there with Ron Hubbard and stuff.