Blue or red pill?

[adamdea]

2 hours ago, mansr said:

I don't like the idea of involving preference. That said, if someone consistently prefers A over B, that should count for something. Then again, if they can do this, why is it necessary to state a preference in the first place?

Most people are really interested in what they prefer, and if they don't actually come out with a consistent preference blind then that provides the answer. The ABX becomes a bit academic at that point. Another advantage is that people can listen to A and B as long as they like while forming the preference.

I accept though that if the question is "can you tell the difference between...", then that doesn't strictly answer it. That said, a random preference does tell a story. 

[semente]

2 hours ago, mansr said:

I don't like the idea of involving preference. That said, if someone consistently prefers A over B, that should count for something. Then again, if they can do this, why is it necessary to state a preference in the first place?

 

After you're done with your listening do ask Mani to point out which aspects sound different to him and see if that'll help you identify the different versions.

[semente]

12 hours ago, Ralph Glasgal said:

But color digital photography is not going to replace oil painting either if it is a matter of taste and they are deemed equivalent except for a few minor defects.

 

 

 

Green Queen

Olio su tela smaltata

120 x 140 cm

 

Marco Grassi | Painter

http://www.marcograssipainter.com/project/green-queen/

[pkane2001]

13 hours ago, Ralph Glasgal said:

I thought this was a group interested in computer/DSP audiophile techniques.  

 

I certainly am. DSP has the potential to correct a lot of ills that are introduced in the signal recording/processing chain. Some things cannot be reversed, such as mixing multiple microphone tracks into two, but many others can be corrected. From frequency response to timing issues to HRFT. I've worked a lot with DSP on digital images, and am a true believer in the power of a properly constructed and properly applied convolution filter.

 

[mansr]

1 hour ago, adamdea said:

Most people are really interested in what they prefer, and if they don't actually come out with a consistent preference blind then that provides the answer. The ABX becomes a bit academic at that point. Another advantage is that people can listen to A and B as long as they like while forming the preference.

I accept though that if the question is "can you tell the difference between...", then that doesn't strictly answer it. That said, a random preference does tell a story. 

An ABX test doesn't have to involve rapid switching.

[adamdea]

4 minutes ago, mansr said:

An ABX test doesn't have to involve rapid switching.

No -true. Although I can't imagine hearing x , then listening to a for a week and then listening to b for a week before deciding whether a or b was x.

 

[mansr]

3 minutes ago, adamdea said:

No -true. Although I can't imagine hearing x , then listening to a for a week and then listening to b for a week before deciding whether a or b was x.

In a typical ABX procedure, you are given samples A and B. Then for a sequence of X samples, you get to listen to A, B, and X as much as you like in any order before deciding whether X is equal to A or B. What does your preferred protocol look like?

[adamdea]

13 minutes ago, mansr said:

In a typical ABX procedure, you are given samples A and B. Then for a sequence of X samples, you get to listen to A, B, and X as much as you like in any order before deciding whether X is equal to A or B. What does your preferred protocol look like?

I don't think it matters - I was just making the point that the standard line (little details only come across over a long time , when the greater sense of air and ease....) seems perfectly consistent with an AB preference test, but sits strangely with an ABx.  On reflection I suppose the greater sense of air and space can after 1 month turn out to be in x having a month before been ruled as missing in B but ruled as present in A two months before. Personally I can only decide whether A or B is x by rapid switching, but my audio memory lacks any super powers.

[manisandher]

52 minutes ago, mansr said:

An ABX test doesn't have to involve rapid switching.

 

It certainly won't in our case. What I have in mind is the following:

 

1. set software to playback configuration A

2. play file in config A for 30 secs or so

3. switch to bit-identical playback configuration B (will take 5-10 seconds at least)

4. play file in config B for 30 secs or so

5. randomly switch between playback configs A and B, playing back file for 30 secs or so each time (ensuring that there's no way I can tell which playback config is being used)

6. edit: repeat 5. for 10 or so times (I don't think I'll need to remind myself of A and B again)

 

On each random playback, I'll simply jot down which of A or B I think I've just listened to.

 

I think I'll actually need each 5-10 second gap to 'reset my mind'... perhaps even longer.

 

Mani.

[mansr]

5 minutes ago, adamdea said:

I don't think it matters - I was just making the point that the standard line (little details only come across over a long time , when the greater sense of air and ease....) seems perfectly consistent with an AB preference test, but sits strangely with an ABx.

It's pretty much the same thing. In an ABX test, each round consists of listening to samples A, B, and X, with a randomly chosen X, and deciding what X is. In your scheme, each round entails listening to pairs, either A-B or B-A, and deciding which it was. If anything, the ABX version should be easier.

[gmgraves]

6 hours ago, mansr said:

If the effect can be heard or not depending on whether one wants to, can it really be said to exist? It certainly must be rather weak. Left/right positioning, on the other hand, is impossible not to hear no matter how hard you try.

That's a good question. If the effect is, indeed, wishful thinking, then the answer is obviously, no, but human preconceptions and biases are pretty strong, so perhaps the question should be: If the effect is real, how come some listener's hear it instantly while those who insist that it doesn't exist before listening to a sample supposedly illustrating the effect say they can't hear it? Wouldn't that constitute an example of confirmation bias?

[sdolezalek]

One of the things I think almost all of these tests get wrong is that they conflate difference and preference.  If we simply focused on finding situations where listeners could reliably distinguish between A and B (without expressing any preference), then we would have a starting point from which could can then ask (does my equipment show a measurable difference between A and B or is there a consistent preference among those who can separate A from B as to which they prefer.  We can also begin to test where does a difference that is clearly measurable in equipment begin to be clearly audible to most listeners.  

 

Quite often I find that when I can tell A from B and clearly prefer one at the moment that I have preferred something that moved away from accuracy.  Only when I then listen to the preferred, but inaccurate version over time does it become tiresome.  A second A/B test a month later will often cause me to then prefer B even though a month earlier I preferred A.  

[Sonicularity]

13 minutes ago, gmgraves said:

That's a good question. If the effect is, indeed, wishful thinking, then the answer is obviously, no, but human preconceptions and biases are pretty strong, so perhaps the question should be: If the effect is real, how come some listener's hear it instantly while those who insist that it doesn't exist before listening to a sample supposedly illustrating the effect say they can't hear it? Wouldn't that constitute an example of confirmation bias?

 

If the effect is real, then a valid test to determine if it is audible would be an ABX listening test.  That is what an ABX is for, determining if a difference can be identified.  It is NOT a tool that can be used to prove that no difference exists.   The ABX test would remove confirmation bias if administered correctly which is not always a simple endeavor.

[adamdea]

3 minutes ago, sdolezalek said:

One of the things I think almost all of these tests get wrong is that they conflate difference and preference. 

No: all these tests don't (or not all of them do). Which of them do and in what way? Are you sure it's the tests which are doing the conflation?

[SoundAndMotion]

8 hours ago, manisandher said:

* My feeling is that an A/B/X is very much a 'left-brain' test, and that the audible differences I and others hear are more easily discerned by taking a 'right-brain' perspective. How do you do the latter? Well, that's not easy to say. I

 

I collect and publish psychophysical data regularly. I'm not sure I can help with the left/right brain thing, but I have some tips:

- Try to ignore past trials and focus on the current trial. I assume an objective randomization will be used (e.g. computer), since people are notoriously bad at trying to create a random sequence on their own. That means the "correct" answer could be A (or B) 3 or 4 or more times in a row. Don't assume "I answered B the last 3 times; the next one is more likely A". Treat each trial as separate from previous ones.

- When an effect is near your threshold for detection, there is a small range of the effect where you don't feel confident, and believe you have to guess. But in this small range (above threshold, but below cognitive recognition), you will "guess" statistically significantly above 50% correct. If you are not confident, go ahead and guess, using a hunch or a feeling, and then stick with that. If you try to "second-guess" your answer, you may change to a wrong answer. (Look up psychometric function if interested to understand this).

- Take breaks. When your alertness and ability to pay attention to listening gets fuzzy, stop after the current trial. I also recommend (and force in my own work) periodic breaks whether you feel the need or not, just so you are as "fresh" as possible.

 

2 hours ago, mansr said:

It's pretty much the same thing. In an ABX test, each round consists of listening to samples A, B, and X, with a randomly chosen X, and deciding what X is. In your scheme, each round entails listening to pairs, either A-B or B-A, and deciding which it was. If anything, the ABX version should be easier.

 

The ABX you describe will certainly be easier to interpret. How will you randomize? Will you do a block design (equal # of A and B; sequence randomized) or will you have a random choice for each trial (# of A's may not equal # of B's)? It is also very important to decide the number of trials before you start. I imagine you'll suggest more than the 10 Mani has suggested, but I don't really know.

 

30 minutes ago, sdolezalek said:

If we simply focused on finding situations where listeners could reliably distinguish between A and B (without expressing any preference), then we would have a starting point from which could can then ask...

... several questions. One problem with a preference test is as you point out, one can (possibly) easily and consistently hear the difference, but depending on track, mood, fatigue, etc. change your preference, which would confound interpretation.

[gmgraves]

9 minutes ago, Sonicularity said:

 

If the effect is real, then a valid test to determine if it is audible would be an ABX listening test.  That is what an ABX is for, determining if a difference can be identified.  It is NOT a tool that can be used to prove that no difference exists.   The ABX test would remove confirmation bias if administered correctly which is not always a simple endeavor.

I'd go along with that if we were talking about a simple, manually switched DBT, but not an ABX. I've never been convinced that the ABX comparator doesn't color the result.

OTOH, how would one go about such a test? The effect in question is the ability of a true stereophonic recording to render depth and height cues while a multitrack, multi-miked recording cannot. I don't see how any DBT can confirm or dispel this assertion. First of all, you'd have to have two samples of the same performance of the same work; one recorded stereophonically while the other was recorded using a forest of microphones captured to a multi-track recorder (16 or perhaps 24 tracks) and then mixed-down to a two-track final version. Then those setting-up the test would need to go through the true stereo recording and find those places where the characteristics being tested for showed themselves and then they would need to sync-up the same place in the multi-mike/track version so that the listeners knew what it was in the playback that they were listening for. Highly unlikely, if you ask me.  

[sandyk]

Quote

• 
•  
• 10,181 posts

• Report post
•  

Posted 10 hours ago

11 hours ago, gmgraves said:

 Like I said before, I strongly suspect that you don't hear any depth or height for the very reason that you have decided beforehand that you're not going to hear it.

 

That appears to be a general problem with many from the Objective side, and why he is highly unlikely to hear any differences that Mani hears when they meet up.

This is why I have ZERO expectations that Archimago will hear any differences with the CD-R that is being sent to him, and why I said to feel free to pass the CD around among Audiophile friends.

[SoundAndMotion]

4 minutes ago, gmgraves said:

I'd go along with that if we were talking about a simple, manually switched DBT, but not an ABX. I've never been convinced that the ABX comparator doesn't color the result.

 

Um, I'm not sure we are using the same terms. An ABX can be done double blind (DBT) or not. It can be done with the hardware "ABX Comparator" or with another method. ABX is a protocol, not a piece of hardware.

[Ralf11]

16 hours ago, STC said:

 

???. Now FAS24 is going to get upset. He is using some Philips HT box setup. 

 

 

yeh but he tuned the hell out of it

[gmgraves]

6 hours ago, SoundAndMotion said:

 

Um, I'm not sure we are using the same terms. An ABX can be done double blind (DBT) or not. It can be done with the hardware "ABX Comparator" or with another method. ABX is a protocol, not a piece of hardware.

All ABX tests are generally done double-blind, but not all double blind tests are ABX. Simple double blind tests are done with some (hopefully) non-interested party simply switching randomly between two test units, where the listeners never know which DUT they are listening to and don't know when (or, indeed, if) the DUTs are being switched. ABX comparators are usually more complex than just someone turning a selector switch on some pre-amp or amp between two high-level inputs (for example). 

[gmgraves]

49 minutes ago, Ralf11 said:

 

 

yeh but he tuned the hell out of it

Right!

[fas42]

57 minutes ago, Ralf11 said:

 

 

yeh but he tuned the hell out of it

 

For anyone who's interested, some of the meandering is recorded here ... http://www.whatsbestforum.com/showthread.php?2966-A-Search-for-Truth-and-Tonality.

[gmgraves]

7 hours ago, sdolezalek said:

One of the things I think almost all of these tests get wrong is that they conflate difference and preference.  If we simply focused on finding situations where listeners could reliably distinguish between A and B (without expressing any preference), then we would have a starting point from which could can then ask (does my equipment show a measurable difference between A and B or is there a consistent preference among those who can separate A from B as to which they prefer.  We can also begin to test where does a difference that is clearly measurable in equipment begin to be clearly audible to most listeners.  

 

Quite often I find that when I can tell A from B and clearly prefer one at the moment that I have preferred something that moved away from accuracy.  Only when I then listen to the preferred, but inaccurate version over time does it become tiresome.  A second A/B test a month later will often cause me to then prefer B even though a month earlier I preferred A.  

Ostensibly, the only thing a DBT can do is show differences between two test units in an instantaneous and sharp relief. It is not its purpose to show which is better or preferred, although I suppose it could be used for that, although the results would be very personal and not statistically computable.  

[STC]

48 minutes ago, Ralf11 said:

 

 

yeh but he tuned the hell out of it

 

There was someone like him who used a JVC player, a mysterious clock, stacked speakers and 8 subwoofers. During his lifetime he probably visited more setups than anyone else and known to almost all audiophiles in my country. The first time I met him was when he tagged along another audiophile who came over for a visit. After he made several visits, he invited me over to his place but I procrastinated for along time considering his ridiculous setup. Finally, out of common courtesy I decided to visit him and invited my other audiophile friends to join me. None of them wanted to join me. Among them one guy who previously visited him criticized his bed room cum listening room (  10ft x 8ft) and his cheap system.

 

Having no choice, I reluctantly went there alone. 

 

That's when I learned not to look down at someone's setup based on the price or approach. I heard a good setup, much better than some tens of times more expensive system.

 

@fas42 could be doing something like that and until then I will give him the benefit of doubt.  

 

 

 

 

 

[fas42]

A favourite way I used at one stage to pick whether there was a true audible variation between, say, two file versions of some track was to load both into Audacity, align them accurately, then select a short but 'interesting' area of the waveform, and set off play in repeat mode for that tidbit. First run one version, over and over again - it becomes an almost hypnotic rhythm of pure sound, no longer music - at some point, flick over to the other version, without missing a beat. Any difference stands out immediately; the original, hypnotic pattern has been broken, and a new pattern is now clearly evident.