Blue or red pill?

[Audiophile Neuroscience]

On 2/10/2018 at 8:38 AM, manisandher said:

Edit April 18th 2018

 

Here are my final thoughts on this thread:

 

A) We know for sure that :

1. the two playback means, A and B, were bit-identical
2. A and B were audibly different (to a 99% probability), once they passed through a DAC

 

B) With no evidence to the contrary, it's reasonable to assume that:

1. the DAC received bit-identical data in both cases
2. the A/B/X was conducted in a trustworthy manner

 

C) We can speculate that:

1. the audible differences were caused by different 'jitter signatures' in the DAC during the D-to-A process
2. the effect of these jitter signatures on the sound is difficult to capture with a modern, well-respected ADC

 

My key take-away from this experience is that the 'red pill' is way more bitter than I could ever have imagined, for those who are entrenched in their beliefs. No amount of evidence contrary to their beliefs seems to help. If you score a 99% probability in a well-executed A/B/X, then "the A/B/X must have been at fault". Etc, etc. I think it's a shame...

 

 

 

Mani I applaud you for having the balls and strength of character to have volunteered for this 'exercise'.

 

Some people will always believe what they want to believe contrary to evidence and that applies to both sides of the audio fence. You on the other hand and it would appear Mans and some others in this thread have approached this with an open mind and with scientific rigour using available evidence.

 

IMHO you have managed to cast reasonable doubt on some well entrenched beliefs and have done so with integrity and an equanimity not always seen on internet boards.

 

17 hours ago, STC said:

I don’t think anyone here can positively identify which one of the parameter really improves the SQ. 

 

I would say the first step is finding a positive correlate for the *difference* in SQ. Causation is another matter.

 

9 minutes ago, austinpop said:

 

 

Thanks for your summary in the first post, and for your patience and forbearance during the exercise.

 

Your conclusion is unfortunate, but certainly not surprising.

 

1+

[STC]

5 hours ago, pkane2001 said:

Since the only difference between A and B was a different SFS setting, we'd know which SFS setting produces the best SQ. In theory.

 

That’s exactly my point. If the sound quality is good enough that you cannot tell the difference without a reference than the importance or difference is insignificant. 

 

It is like distinguishing shades of green. How many can tell which is Jade green and Persian green just by looking at the color with side by side comparison. At least here the difference is more pronounced than the A and B of the analogue capture. 

[STC]

13 hours ago, Andyman said:

I suspect at least one (double) Dutch man can...

 

After much effort. 

[Audiophile Neuroscience]

5 minutes ago, STC said:

 

If the sound quality is good enough that you cannot tell the difference without a reference than the importance or difference is insignificant. 

 

 

The question of whether the difference matters (or is "insignificant") is personal and subjective. IMO the point of the thread is/was IS there a difference and if so why? Does it matter? Depends on your perspective.

[STC]

9 minutes ago, Audiophile Neuroscience said:

 

The question of whether the difference matters (or is "insignificant") is personal and subjective. IMO the point of the thread is/was IS there a difference and if so why? Does it matter? Depends on your perspective.

 

Preference is subjective. You can like vinyl and other s may like digital. The SQ is quite obvious there. But if you were to listen to one without the reference to the other and unable to tell whether that’s  vinyl or digital then the diff doesn’t matter. Anyway, I am going OT so I will stop here. 

[Audiophile Neuroscience]

21 minutes ago, STC said:

 

Preference is subjective.

 

Exactly

Quote

 But if you were to listen to one without the reference to the other and unable to tell whether that’s  vinyl or digital then the diff doesn’t matter.

 

"The diff doesn't matter" is a preference on your part. Others prefer that the difference matters. Just sayin'

[Ralf11]

We still have not established that a difference that is hearable by most audiophiles exists.  Possibly, only 1 or 2 very careful listeners can hear this difference.

[STC]

9 minutes ago, Audiophile Neuroscience said:

 

Exactly

 

"The diif doesn't matter" is a preference on your part. Others prefer that the difference matters. Just sayin'

 

 There was no difference reliably spotted in the first 20 trials. Methodology doesn’t matter. If you know or hear the difference than it should be identifiable under all normal circumstances. 

 

 

[esldude]

2 minutes ago, STC said:

 

 There was no difference reliably spotted in the first 20 trials. Methodology doesn’t matter. If you know or hear the difference than it should be identifiable under all normal circumstances. 

 

 

This is going way to far.  Careful listening, good listening conditions, quality of the gear do matter.  You couldn't hear lots of differences in the middle of a noisey crowd.  So no a difference isn't always audible under all conditions. 

 

Yes a large enough difference would have been heard with both procedures.  But this is a smaller difference.  The topic of this was never how large the difference as much as whether there is such a difference even if miniscule.  

[STC]

1 minute ago, esldude said:

This is going way to far.  Careful listening, good listening conditions, quality of the gear do matter.  You couldn't hear lots of differences in the middle of a noisey crowd.  So no a difference isn't always audible under all conditions. 

 

Under perfect condition and method chosen by the subject himself. 

 

I still think I heard the difference between the A and B analogue output. Managed to configure Foobar to the main system and will give it a try. 

[Ralf11]

also " normal circumstances" is not very specific

 

The one thing that bothers me some in this test is that some trials were apparently thrown out.  Otherwise it seems there a small difference that can be heard by at least some humans.

 

The mechanism is unclear but software settings seem a likely culprit...

[esldude]

1 minute ago, STC said:

 

Under perfect condition and method chosen by the subject himself. 

 

I still think I heard the difference between the A and B analogue output. Managed to configure Foobar to the main system and will give it a try. 

I still don't see this as a criticism. 

 

Now MQA tests (actually they were tests of sampling and filters)  are a difference too small.  Extraordinary gear under very extraordinary listening conditions using highly trained, and especially trained for the difference listeners, using much steeper than normal filtering were able to eventually choose 60% correct (barely enough under more than 100 choices to be at p=.05) rather than 50% of guessing.....that is a difference too small to worry with. 

 

9 of 10 if regularly possible is quite a different level of difference even if small. 

[fas42]

1 hour ago, STC said:

 

That’s exactly my point. If the sound quality is good enough that you cannot tell the difference without a reference than the importance or difference is insignificant. 

 

It is like distinguishing shades of green. How many can tell which is Jade green and Persian green just by looking at the color with side by side comparison. At least here the difference is more pronounced than the A and B of the analogue capture. 

 

This is not how to approach the exercise. If there is a difference, and one displays the qualities of distortion less then you're going in the right direction - the goal is convincing sound, and this can only "pop out" if enough of all those "little things" are addressed.

 

I learnt what happens 3 decades ago - each, seemingly tiny step contributes to the whole - it may take 10, 20, 30 steps; but it is the steady unraveling of the "bugs" that makes it occur ... I use exactly the same methods to advance the SQ today as I did 30 years - because it works ...

[STC]

15 minutes ago, esldude said:

I still don't see this as a criticism. 

 

Now MQA tests (actually they were tests of sampling and filters)  are a difference too small.  Extraordinary gear under very extraordinary listening conditions using highly trained, and especially trained for the difference listeners, using much steeper than normal filtering were able to eventually choose 60% correct (barely enough under more than 100 choices to be at p=.05) rather than 50% of guessing.....that is a difference too small to worry with. 

 

9 of 10 if regularly possible is quite a different level of difference even if small. 

 

I am not criticizing. In fact, I wish there are more people like Mani and Peterst over here. 

 

3 minutes ago, fas42 said:

 

This is not how to approach the exercise. If there is a difference, and one displays the qualities of distortion less then you're going in the right direction - the goal is convincing sound, and this can only "pop out" if enough of all those "little things" are addressed.

 

I learnt what happens 3 decades ago - each, seemingly tiny step contributes to the whole - it may take 10, 20, 30 steps; but it is the steady unraveling of the "bugs" that makes it occur ... I use exactly the same methods to advance the SQ today as I did 30 years - because it works ...

 

Unless you have the same setup that you had 30 years ago, then you do not not for sure what really changed the SQ. You can only speculate .  Most of the so called difference can be attributed to other things. I may change a cable and perceive better sound but I may have also changed my sitting or speakers location or cleaned the contact along that. 

 

That reminds me of of someone who specialized in tweaking a Quad amplifier. After three years of hard work he decided to do the modification to another similar Quad. Guess what? The unmodified Quad sounded better. You need a reference.

[Audiophile Neuroscience]

57 minutes ago, STC said:

 

Methodology doesn’t matter.

 

 

 

Oh dear !

[Audiophile Neuroscience]

1 hour ago, Ralf11 said:

The one thing that bothers me some in this test is that some trials were apparently thrown out. 

 

If interpreting correctly, the trials that were "thrown out" were the ABXXXXXXX trials. As said previously you cannot combine results from different trials using different test procedures. Each must be evaluated individually on their merits. All tests are not made equal.

 

I have been a long term critic of ABX testing as being not adequately scientifically validated in the specific environment of hearing differences in complex music. Mani has provided evidence that an ABX test done in the *correct manner* ( ie demonstrated correct methodology)  *may* qualify as a sufficiently sensitive, valid, reliable scientific test tool. IMO this is awesome and could potentially finally kill off wasted hours of futile debate.

 

Science aligns with evidence, that's the way it is. Whether I or others prefer the evidence or not is immaterial. The evidence thus far is that Mani can hear a difference in  bit identical playback to a significance level of p=0.01. Unless anyone has evidence to the contrary and not just 'noise' ?

 

Edit: The question is Why? There has to be an explanation based in science not magic.

[fas42]

1 hour ago, STC said:

Unless you have the same setup that you had 30 years ago, then you do not not for sure what really changed the SQ. You can only speculate .  Most of the so called difference can be attributed to other things. I may change a cable and perceive better sound but I may have also changed my sitting or speakers location or cleaned the contact along that. 

 

That reminds me of of someone who specialized in tweaking a Quad amplifier. After three years of hard work he decided to do the modification to another similar Quad. Guess what? The unmodified Quad sounded better. You need a reference.

 

That's the point of repeating an experiment, say by different researchers. If doing things using a certain procedure always achieves a particular result then you have a high confidence that "you're on to something".

 

What changed the SQ was locating issues, weaknesses in the equipment, and the combination of the parts - just like  a mechanic repairing a certain model of a vehicle; various units come in with specific problems, all very different, but after repair they all perform "just like the reference" ... that's what's going on, here.

 

The reference I use I have described often: completely disappearing speakers, all recordings come to life, ability to go to any sane volume level with complete comfort, etc, etc. Most systems are so far from this goal that one doesn't know where to start ... .

 

A friend of my local audio friend is caught up in the strange world most audiophiles inhabit - he has money to burn, and sometimes his system shows great promise; but the next time it's pretty well a mess again. He most certainly has no reference - just using the metric that the more expensive the gear he buys, the better it has to be, QED.

[fas42]

13 minutes ago, Audiophile Neuroscience said:

Science aligns with evidence, that's the way it is. Whether I or others prefer the evidence or not is immaterial. The evidence thus far is that Mani can hear a difference in  bit identical playback to a significance level of p=0.01. Unless anyone has evidence to the contrary and not just 'noise' ?

 

And I can hear the difference in the analogue captures of that event, as can Mani - I'm not interested in playing ABX games; if the difference is clearly audible, end of story. What now is of interest is locating the data in the tracks which causing this.

[Audiophile Neuroscience]

27 minutes ago, fas42 said:

 

That's the point of repeating an experiment, say by different researchers. If doing things using a certain procedure always achieves a particular result then you have a high confidence that "you're on to something".

 

I agree that further trial numbers (>10) and/or reproducible results strengthens confidence....beyond existing 99%

 

 

20 minutes ago, fas42 said:

 

And I can hear the difference in the analogue captures of that event, as can Mani - I'm not interested in playing ABX games; if the difference is clearly audible, end of story.

 

The Difference is Mani has played the ABX "game" and won and so people have evidence supporting his claims. I do agree with you though that if convincing others is not your goal then no point in demonstrating it.

 

Quote

What now is of interest is locating the data in the tracks which causing this.

 

1+

[STC]

3 hours ago, STC said:

 

Under perfect condition and method chosen by the subject himself. 

 

I still think I heard the difference between the A and B analogue output. Managed to configure Foobar to the main system and will give it a try. 

 

 

I told so. I could hear a difference. I got 9/10 incorrect. I have been consistently picking the wrong one. One attempt only. Does it count?

 

foo_abx 2.0.4 report
foobar2000 v1.3.17
2018-04-25 10:53:59

File A: 3. analogue capture _ A.wav
SHA1: 9c7dbfe1ac5f00c6f3d3acd73c3d6a3fd1eaa59d
File B: 4. analogue capture _ B.wav
SHA1: 6c1ec457e27079d3e864c0396851e91185fcc982

Output:
DS : CABLE Input (VB-Audio Virtual Cable)
Crossfading: NO

10:53:59 : Test started.
10:54:37 : 00/01
10:56:09 : 00/02
10:57:02 : 00/03
10:57:43 : 01/04
10:58:22 : 01/05
10:59:22 : 01/06
10:59:35 : 01/07
10:59:58 : 01/08
11:00:29 : 01/09
11:00:46 : 01/10
11:00:46 : Test finished.

 ----------
Total: 1/10
Probability that you were guessing: 99.9%

 -- signature --
46e11e874a48c2d8abc029a909bb6467ba45c5f2

[fas42]

Considering that Foobar is highly flawed in its own right, that's pretty good ...

[STC]

7 minutes ago, fas42 said:

Considering that Foobar is highly flawed in its own right, that's pretty good ...

 

Frank, I like the way you think.  

[Audiophile Neuroscience]

2 hours ago, STC said:

 

 

I told so. I could hear a difference. I got 9/10 incorrect. I have been consistently picking the wrong one. One attempt only. Does it count?

....

Probability that you were guessing: 99.9%

 

 

Credit where credit is due, it takes a certain talent to be wrong that consistently !

[PeterSt]

5 hours ago, esldude said:

Yes a large enough difference would have been heard with both procedures.  But this is a smaller difference.

 

There's always the possibility of the "made to suit" track(s) for comparison. I'd (we'd) rather continue with the files as of present because I know how long it can take to get acquainted with the "situation". If I had been digging in files for 4 hours I rather don't have a new set.

[PeterSt]

2 hours ago, STC said:

I told so. I could hear a difference. I got 9/10 incorrect. I have been consistently picking the wrong one. One attempt only. Does it count?

 

We may wonder about the probability of *this*. This is as good as Mani's. So now what.

 

The only thing I can come up with is that something in your mind certainly got it right but next some aversion or something made you decide the other way around each time. I think this most certainly can exist (the "aversion" thing).