Blue or red pill?

[manisandher]

8 minutes ago, Sonicularity said:

If I understand correctly, this was how the test was conducted:

1. Play system configuration A for about 30 seconds
2. 5-10 seconds are used to reconfigure for system configuration B
3. Play system configuration B for about 30 seconds
4. then what?  A pause for 5-10 seconds and restart system configuration B or an actual 5-10 second reconfiguration back to A to play until a determination of A or B is playing?
5. Repeat steps 1-4 10 times

 

Almost. More specifically it was like this:

1. A then B then X
2. A then B then X
3. A then B then X
4. A then B then X
5. A then B then X
6. A then B then X
7. A then B then X
8. A then B then X
9. A then B then X
10. A then B then X

In each case, 'X' was either 'A' or 'B', chosen randomly (by an app on Mans's phone).

 

Each 'A', 'B' and 'X' were played for ~15 secs.

 

There was a ~20-30 sec gap between 'A', 'B' and 'X'. So each 'A then B then X' set took 90-120 secs to complete.

 

HTH.

 

Mani.

[Sonicularity]

Considering the gap between the plays, it would seem that any audible cues used to identify them would have had to be rather obvious to you.  This is why I find it confusing that the first test wasn't passed with flying colors as well.  Maybe it would have with more practice?  It is one thing to switch instantly between two audio test samples to identify a difference, but the procedure in this test was not ideal for identifying differences between very close-sounding audio test samples.

 

Essentially you seem to clearly hear a difference, but it is not obvious if you were able to tell which one you preferred, only that there was a difference when played back to back.  I wonder if you even needed to hear A each time?  Just listen to B and then play X and see if it was the same or not, or play only A then X.  Not sure what the point of playing both A and B first, then X if you were not switching back and forth instantly.  

 

Very interesting and a fun read.  Thanks a bunch.

[manisandher]

16 minutes ago, Sonicularity said:

Considering the gap between the plays, it would seem that any audible cues used to identify them would have had to be rather obvious to you.  This is why I find it confusing that the first test wasn't passed with flying colors as well.

 

The best explanation I can come up with is the following...

 

A, B, X, X, X... X

- I felt I could clearly hear a difference between A and B when they were first played in turn

- all I then had to go by was whether the current X was different from the previous X

- within a couple of Xs, I had lost any sense of what A and B had actually sounded like

- the further we went, the more the Xs just seemed to sound the same

- an efficient, but not particularly effective, way of doing things

(- but I'm 100% confident that I could apply this protocol successfully if, say, A were a piano and B a guitar )

 

A, B, X... A, B, X

- before each X, I was reassured that I really could hear a difference between A and B

- I had an absolute reference with which to work

- there was an opportunity to have a short rest and 'reset' before the next A, B, X

- the previous A, B, X had absolutely no bearing on the current A, B, X

- an effective, but not particularly efficient, way of doing things

 

(Why does the first protocol remind me somewhat of SDM and the second of PCM? Never mind...)

 

16 minutes ago, Sonicularity said:

Very interesting and a fun read.  Thanks a bunch.

 

You're welcome 

 

Mani.

[esldude]

1 hour ago, manisandher said:

 

The best explanation I can come up with is the following...

 

A, B, X, X, X... X

- I felt I could clearly hear a difference between A and B when they were first played in turn

- all I then had to go by was whether the current X was different from the previous X

- within a couple of Xs, I had lost any sense of what A and B had actually sounded like

- the further we went, the more the Xs just seemed to sound the same

- an efficient, but not particularly effective, way of doing things

(- but I'm 100% confident that I could apply this protocol successfully if, say, A were a piano and B a guitar )

 

A, B, X... A, B, X

- before each X, I was reassured that I really could hear a difference between A and B

- I had an absolute reference with which to work

- there was an opportunity to have a short rest and 'reset' before the next A, B, X

- the previous A, B, X had absolutely no bearing on the current A, B, X

- an effective, but not particularly efficient, way of doing things

 

(Why does the first protocol remind me somewhat of SDM and the second of PCM? Never mind...)

 

 

You're welcome 

 

Mani.

Yes a string of choices without reference is not the best way.  Maybe presenting A and then asking if X were same or different would work.  But the best way is to present A, then B and then X.  I am surprised it worked with those long pauses between however. 

 

Another option is a triangle test.  Present three and of the three two are the same and one is different. Choose the different one.  This is statistically a little better with a test using fewer samples.  At random one expects each choice one third of the time.  So 7 of ten would pass the p=.05 threshold.  Yet it takes the same amount of time as ABX.  9 of 10 on a triangle test is p=.001 just fyi. 

 

So if you do this again you might experiment with a triangle test.  Or since you seem to be doing fine with ABX and now have experience probably best to stick with it.   Do ten take a break and do ten more.  

 

Of course I'm hoping when mansr gets done he will have found a reason you heard the difference.  

[sandyk]

7 minutes ago, esldude said:

Of course I'm hoping when mansr gets done he will have found a reason you heard the difference.  

 

 Don't hold your breath while waiting !

 For Mani's sake let's hope that he doesn't need to perform another series or ten of ABX sessions .

[manisandher]

On 4/1/2018 at 5:10 AM, PeterSt said:

This is how I have thought to many many times that we must be missing something like Windows outputting something different to a "digital recording device" than to a DAC (technically the OS will be able to differentiate).

 

You're talking about USB, right?

 

However, for the test, we used an spdif PCI card - no USB anywhere. How could the OS possibly detect what the spdif output of the PCI card is connected to?

 

Mani.

[PeterSt]

45 minutes ago, manisandher said:

However, for the test, we used an spdif PCI card - no USB anywhere. How could the OS possibly detect what the spdif output of the PCI card is connected to?

 

To be honest, I don't know. And especially then things get dangerous. 

All I know is that since Vista (and W7 different and W7 SP1 different again) all sorts of stuff regarding "End Points" showed up. So the OS knows that you plugged in a subwoofer in your MoBo's output port for it, etc. My idea would be that it knows that SPDIF is connected (and coax differently from Toslink). From there I don't know because I never looked at it. So all I can do as of now is agree with you because I don't know (there's nothing left to think of). But thinking "conspiracy", in W7 the loop-back via software emerged so it can detect that too (but it is a setting first). Some software depends on this (JRiver at least did with WASAPI). Thus a kind of virtual wire. Notice that it is about the detections and not so much for now about the technical result of it. One stream could be treated differently than the other.

 

And if you really want to think "conspiracy" then you should remember your own question on WBF to AmirM who is virtually the cause of the "since Vista" audio chain about the odd (phasing) sounding W8 which took us a year to get right and how in the world it could be that it happened while being bit perfect and ...

... And while it would be his very own subject he did not even respond to your question in the dedicated thread.

Of course I already had been fighting with him for a small year (although I wasn't really involved in your question over at WBF) - with that telling that I don't/did't trust the bloke anyway and which did not improve much over time (the contrary). But still. He *is* keen about these matters (why does sound change) - he *did* have inside information about the OS all over for audio, and then does not even respond ? NDA stuff ?

But never mind, it is a way which won't lead anywhere. Still I'll keep it in mind though (yes, I can be worse than you, some times).

 

Peter

 

[STC]

@manisandher and @mansr, guys how about trying a different DAC? 

[pkane2001]

4 hours ago, STC said:

@manisandher and @mansr, guys how about trying a different DAC? 

 

There are three factors that can affect the output of a DAC based on a source

 

1. Bits change -- supposedly not in this test

2. Timing change (jitter) -- possibly happened, remains to be seen if jitter is detectable in the analog captures

3. Electrical or EMI Noise coming from the PC - again possibly can be seen in the analog captures

 

@STC is proposing another possibility due differences in DAC and the recorder. That the DAC is reacting differently to incoming digital data than the recorder used in the test. For example, if the receiver circuit at the DAC is very sensitive to noise, it may not receive every bit exactly as the recorder circuit would. So, while the recorded bits are matching perfectly, perhaps the DAC didn't get the same exact bits. Another example, if the DAC PHY forms a ground loop with the PC while the recorder doesn't, the DAC may not receive the same unmolested bits as the recorder. I think this is a possibility, as well, although harder to check from existing captures unless the DAC has a digital output that allows recording.

 

 

 

 

[STC]

26 minutes ago, pkane2001 said:

o, while the recorded bits are matching perfectly, perhaps the DAC didn't get the same exact bits.

 

I think they both are getting exactly the same bits. However, the recorder ensures the bits are identical to the original files. In DAC case, I think the priority is to convert the digital to analogue for audio and some compromise could have take taken place. If I am not mistaken the DAC used in the Altmann DAC was PCM1604 which is rather dated and the error correction if any may not be as good as modern chips. I am just speculating here.  

 

 

[manisandher]

19 minutes ago, pkane2001 said:

1. Bits change -- supposedly in this test

 

I'm assuming you meant "supposedly not in this test"?

 

But even so, why the "supposedly"? Mans has said that all the digital captures were bit-identical. Everyone else (including PeterSt, who initially felt that there was a possibility they might not be, until Mans shared his findings) seems to have accepted this.

 

Mani.

[pkane2001]

9 minutes ago, STC said:

 

I think they both are getting exactly the same bits. However, the recorder ensures the bits are identical to the original files. In DAC case, I think the priority is to convert the digital to analogue for audio and some compromise could have take taken place. If I am not mistaken the DAC used in the Altmann DAC was PCM1604 which is rather dated and the error correction if any may not be as good as modern chips. I am just speculating here.  

 

Ok, then I propose yet another failure mode Remember that bits are carried by analog waveforms. If the receiver circuit in the DAC is not as good as the receiver in the recorder, it may not recover all the same bits as the recorder might if the circuit is flakey or more sensitive to noise, or forms a ground loop, or has a different threshold of detection, or...?

 

[pkane2001]

9 minutes ago, manisandher said:

 

I'm assuming you meant "supposedly not in this test"?

 

But even so, why the "supposedly"? Mans has said that all the digital captures were bit-identical. Everyone else (including PeterSt, who initially felt that there was a possibility they might not be, until Mans shared his findings) seems to have accepted this.

 

Mani.

Yes, sorry. Corrected.

 

'Supposedly' because I'd still want to see Mans report his findings and how he checked for bit-correctness.

[STC]

I think for a proper ABX tests, the equipment must meet the accepted standard. Maybe. A Mytek, Teac or even any good pro audio engineered to accepted specs. 

[manisandher]

2 minutes ago, STC said:

However, the recorder ensures the bits are identical to the original files.

 

The spdif protocol is uni-directional, from what I understand. There is no way that the recorder could ensure that the bits were identical to the original file. Happy to be corrected though.

 

2 minutes ago, STC said:

If I am not mistaken the DAC used in the Altmann DAC was PCM1604 which is rather dated and the error correction if any may not be as good as modern chips. I am just speculating here.  

 

Correct on the DAC, but wrong on the chip. It uses a single 1543 chip.

 

Mani.

[manisandher]

31 minutes ago, pkane2001 said:

3. Electrical or EMI Noise coming from the PC - again possibly can be seen in the analog captures

 

Mans took some digital and analogue captures of 'digital silence' playing through the DAC. Presumably electrical and/or EMI noise would be detectable in the analogue captures (though I can't see anything untoward in my cursory analysis.)

 

And remember, what we're looking for in the analogue captures isn't anything 'absolute'. We're looking for differences between the analogue captures of 'A' and 'B'. It wouldn't be enough to show that there was electrical and/or EMI noise getting to the DAC - it would be be necessary to show that this noise was consistently different between the analogue captures of 'A' and 'B'.

 

Mani

[pkane2001]

10 minutes ago, manisandher said:

 

Mans took some digital and analogue captures of 'digital silence' playing through the DAC. Presumably electrical and/or EMI noise would be detectable in the analogue captures (though I can't see anything untoward in my cursory analysis.)

 

And remember, what we're looking for in the analogue captures isn't anything 'absolute'. We're looking for differences between the analogue captures of 'A' and 'B'. It wouldn't be enough to show that there was electrical and/or EMI noise getting to the DAC - it would be be necessary to show that this noise was consistently different between the analogue captures of 'A' and 'B'.

 

Mani

 

Mani, I agree. But what's been proposed by others, including Peter as I recall, is that different algorithms produce different load on the CPU, memory, bus, etc., resulting in different noise patterns and current draw. The conclusion that's drawn from this is that different player programs, different versions of the PC operating system, or even just where the source file is being accessed from, can all result in different patterns of PC noise that can somehow infect the DA process and result in audible differences. Following this logic, it's possible that different SFS settings produced wildly different CPU activity in the PC that resulted in very different noise patterns. Put an AM radio next to a PC, and you can actually hear the differences

 

I'm skeptical about this process in a general case, but I have seen measurements of some really poorly designed DACs where PC activity was clearly reflected in the analog output. Proper design should take care of this, and according to measurements, it appears to do so even in some of the very inexpensive DACs.

 

[manisandher]

3 minutes ago, pkane2001 said:

Mani, I agree. But what's been proposed by others, including Peter as I recall, is that different algorithms produce different load on the CPU, memory, bus, etc., resulting in different noise patterns and current draw. The conclusion that's drawn from this is that different player programs, different versions of the PC operating system, or even just where the source file is being accessed from, can all result in different patterns of PC noise that can somehow infect the DA process and result in audible differences. Following this logic, it's possible that different SFS settings produced wildly different CPU activity in the PC that resulted in very different noise patterns. Put an AM radio next to a PC, and you can actually hear the differences

 

I'm skeptical about this process in a general case, but I have seen measurements of some really poorly designed DACs where PC activity was clearly reflected in the analog output. Proper design should take care of this, and according to measurements, it appears to do so even in some of the cheaper DACs.

 

 

Yep Paul, I agree with all of this.

 

I reckon Mans's analysis will show that the Altmann was as resolving as a 16-bit DAC can possibly be. But let's see...

 

Mani.

 

 

 

[manisandher]

10 minutes ago, STC said:

I think for a proper ABX tests, the equipment must meet the accepted standard. Maybe. A Mytek, Teac or even any good pro audio engineered to accepted specs. 

 

I agree that the ADC must be engineered to accepted specs. The Tascam that we used certainly is. The DAC though is a totally different matter.

 

Being only 16-bit, I doubt the Altmann DAC is going to impress anyone with its measurements. But I have to say that it's the second best-sounding DAC I've ever had in my system. (I've had loads of well regarded DACs from Mytek, Schiit, Chord, Pacific Microsonics... and loads more besides.) But this is purely subjective of course.

 

Mani.

[Summit]

To hear the difference between two audio gears or between different software and so on is the easy evaluation part IMO – to say which one that sounds best or are most accurate is much harder and takes time and profound knowledge of the pros and cons of the whole audio system.

 

Also fast A/B and A/B/X test are likely to exaggerate some parameters over others that you would need longer time to release, correctly hear and appreciate.      

[STC]

30 minutes ago, manisandher said:

But I have to say that it's the second best-sounding DAC I've ever had in my system. (I

 

I know of DAC that is expensive and measures poorly but well liked by audiophiles. For this purpose of experiment we need a DAC that is as accurate as possible. 

 

The output gain of a DAC accounts  for much difference besides deliberately induced noise. 

 

The DAC itself can work differently when it comes to errors. Not necessarily bits alone but also other noises. 

 

IMO, just because the recorder  managed to ensure bit identical files after error correction, it doesn’t mean the signals are identical. You have to establish that. 

 

 

[manisandher]

5 minutes ago, STC said:

IMO, just because the recorder  managed to ensure bit identical files after error correction, it doesn’t mean the signals are identical. You have to establish that. 

 

Yes. That's why we've got a bunch of analogue captures of the DAC's output.

 

(And by the way, I totally dispute that any error correction was occurring during the digital captures.)

 

Mani.

[manisandher]

4 minutes ago, STC said:

For this purpose of experiment we need a DAC that is as accurate as possible.

 

I think it's totally irrelevant.

 

Mani.

[sandyk]

6 minutes ago, manisandher said:

 

I think it's totally irrelevant.

 

Mani.

 

 The only thing relevant about that, is that a higher quality Coax SPDIF input DAC would almost certainly make the audible differences even  greater.

[STC]

5 minutes ago, manisandher said:

 

Yes. That's why we've got a bunch of analogue captures of the DAC's output.

 

(And by the way, I totally dispute that any error correction was occurring during the digital captures.)

 

Mani.

 

Capturing the analogue output proves nothing. If you heard a differences than what was coming out the analogue output must be different. 

 

Measure ( if that’s ever possible) what’s reaching before the conversion takes possible. 

 

You could be right about error correction as my knowledge about the workings of DAC is next to zero. Yet, it is hard to accept that digital comes without error correction.