Investigation Into Effects Of PC load On DAC Analogue Output

[manisandher]

8 hours ago, March Audio said:

With all respect to Mani, unfortuntely Its a hopelessly flawed method.

 

In my defence, I've already undertaken many of the tests you're now trying. They've all come back blank. So, I thought I'd see if capturing the actual sound waves with a mic might prove insightful. This might actually be the case, but not in the noisey enviroment in which I conducted the most recent tests. I've just been too busy to repeat them in my much quieter listening room, but hope to do so at some point.

 

What isn't in question AFAIC is that bit identical playback can sound different. For example, bit identical settings in XXHighEnd change the sound (even with a modern DAC like the RME, that both of us have).

 

I had experienced this for years before inviting Mans up to conduct an essentially double blind ABX test - Mans was controlling the test in a room separated by two doors and a corridor from the listening room where I was, and using a random generator on his phone to select A or B. I scored 9/10 (1% probability of guessing). I really should have scored 10/10, as the differenc between A and B was clearly audible, but it was 15 minutes into the test, I was tired and I got #9 wrong. We captured the digital input to the DAC in real-time througout the ABX. A and B remained bit-identical throughout.

 

Just consider the following sequence of events here:

 

- I was convinced I could hear differences between bit-identical playback

- I was confident I could demonstrate this in a (essentially double) blind ABX

- I invited Mans up to my place (paid for his train ticket, no less)

- I scored 9/10

 

Coincidence? Chance? Luck? I don't think so.

 

Mani.

[manisandher]

1 minute ago, March Audio said:

 

With respect the devil is in the detail of the ABX test and a sample of one doesnt really tell us much.

 

 

It's your prerogative to take it however you want.

 

The most telling part of the whole day was after the ABX when I sat Mans down in the listening room and told him what to listen out for between A and B. I heard the differences easily, he said he couldn't.

 

Couldn't, or couldn't bring himself to???

 

Mani.

[manisandher]

8 minutes ago, March Audio said:

Even a quiet listening room wont work Im afraid.

 

And yet, I can hear the differences even in my very far from quiet office. My ears are obviously totally ignoring whatever ambient noise there is.

 

Mani.

[manisandher]

17 minutes ago, March Audio said:

With respect you are entering the territory of "I hear it therefore it is", which is not what an objective thread is about.

 

I give up.

[manisandher]

6 hours ago, PeterSt said:

I tend to refer to the red or blue pill thread from ManiSandher where I also promised that Mansr would not be able to measure a thing of what Mani definitely would be able to hear (which he did - the 9 out of 10 story).

 

Well, I haven't given up trying to measure the differences that I clearly hear.

 

I think @pkane2001's DeltaWave software is the most promising approach. But at the moment, there's just too much variation between the analogue captures... without changing a thing, let alone varying any bit-identical parameters. Matching the captures against the original anywhere near as well as is necessary is just really, really difficult to achieve. (Matching captures against captures gives much better results.)

 

I still hold the belief that if we can hear a difference, then it should be measurable... somehow. The key is, I believe, in using dynamic content (i.e. real music) and not constant tones (including IMD test tones).

 

Mani.

[manisandher]

On 9/1/2021 at 12:26 AM, fas42 said:

I for one would very much appreciate if you could do several consecutive captures of some piece playing, without changing anything in between ... and post them.

 

Happy to do that at some point, but not sure when I'll be able to get around to it.

 

Mani.

[manisandher]

3 hours ago, pkane2001 said:

It's a fun historical fact that your red/blue pill thread is what started me on DeltaWave development ;)

 

Haha... so something good did come out of the whole endeavour! That's really nice to know. Thanks.

 

Mani.

[manisandher]

On 9/4/2021 at 3:52 PM, plissken said:

I just wanted to clear this up about Red pill vs Blue pill with a comment from someone that is no longer able to participate here:

 

"When I got there, Mani informed me that we wouldn't be listening to that after all. Instead, he insisted on playing the same local track with two different values of some undocumented setting in an utterly bizarre playback application (XX-something-or-other)."

 

He's a liar.

 

Sent to @mansr on March 25, 2018 (day before his visit to my place):

 

Hi Mans,

 

This is the procedure I'd like to use tomorrow:

 

1. take a quick listen together

- I'd like to demonstrate a few things to you and get your initial thoughts

 

2. conduct the A/B/X

- I've chosen the track and the bit-identical changes we'll use in the playback software

- you'll be sitting in my office, controlling playback from there, and I'll be sitting in the listening room

- we'll have the Tascam set to auto-record, sitting in the basement next to the audio PC and DAC, capturing the digital output of the audio PC in real time

3. ensure that the digital captures are identical

- I have Audacity and MusicScope here

- if you have other software you'd rather use feel free to bring your laptop along with you

 

4. capture analogue outputs (test track plus tones)

a. directly from the DAC, using the Tascam

b. from the speakers, using my Earthworks microphone and portable Korg recorder

 

5. analyse analogue outputs

- we could either attempt to do this here or you could take the files away with you

 

Obviously, there's no need to go on to step 3 unless you're convinced there really are audible differences (either because you hear them too, or because I manage to demonstrate that this is the case in the A/B/X).

 

It'd be great if we can get through all of this tomorrow, but really only need to get through 1 and 2 as a must.

 

See you tomorrow!

 

Cheers,

Mani.

 

Reply received from @mansr on March 25, 2018:

 

Sounds like a plan.

 

Edit: I had laid out a number of ABX options in the thread a few days before the visit, including my preferred option of using the software player, and why. I then sent the PM to confirm that he was OK with it. Not my fault if he was still confused.

 

(If @The Computer Audiophile has access to PMs on this site, he's welcome to check that what I've pasted is letter-for-letter correct.)

 

Mans was totally flummoxed by my 9/10 in the ABX, and tried to find any excuse not to accept it. Including my being able to hear the keyboard strokes through two closed doors and a corridor, and being able to decipher the software settings from them.

 

What a total joke he turned out to be.

 

Mani.

[manisandher]

Found it:

 

 

Mani.

[manisandher]

13 minutes ago, pkane2001 said:

Mani, I don't know why this is being rehashed yet again, but as a forum participant in that thread, before Mans' visit, I was completely unaware of the plan to use some obscure setting in a software that the author himself couldn't clearly explain.

 

Paul, on March 2 2018, @mansr wrote:

 

The hypothesis is that identical files played from different storage media sound different.

 

On the same day, I replied:

 

That's a specific case of the more general hypothesis: a file played* back bit-identically can sound different.

 

Specific cases then include:

- different storage media

- different digital cables (spdif, USB, etc)

- different software player configurations (buffers, etc)

 

I'm considering which of these would be best for the ABX, and am leaning towards different software player configs.

 

So, I was leaning towards using bit-identical software settings a full 3 weeks before the test. If you were following, you should have been aware of this.

 

Exactly what the software player was doing was totally irrelevant... as long as we could prove that the inputs to the DAC were bit-identical... which we did.

 

22 minutes ago, pkane2001 said:

... this test did not seem to be well planned or well executed with last minute changes...

 

What last minute changes?

 

23 minutes ago, pkane2001 said:

... and malfunctioning equipment.

 

The only malfunctioning equipment was the Tascam recorder, which did some funky stuff for a few ms when set to auto-start. But this had zero bearing on the ABX, and proved irrelevant anyway (we simply ignored the first few ms, and checked that the remaining 15000 ms or so of each ABX were bit-identical).

 

Are there any other issues you can cite with the ABX? Keyboard strokes through two closed doors and corridor perhaps? 😜

 

Mani.

[manisandher]

20 minutes ago, pkane2001 said:

The planning should've had a discussion of the player setting.

 

Why? As long as everything remained bit-identical, which it did, it should be irrelevant.

 

20 minutes ago, pkane2001 said:

And, again, I had the impression that USB DAC was going to be used...

 

Indeed, I had originally intended to use my USB DAC. But at some point Mans insisted that we needed to capture the digital input to the DAC in real time during the ABX (something I came around to agreeing with). I couldn't figure out a way to do this with the USB DAC I had, so we agreed that we'd use an spdif DAC instead.

 

In retrospect, I wish we hadn't used the DAC we did. But it was the only one I had to hand with an spdif input. If I were to repeat the test today, I'd use my RME ADI-2, and capture its spdif output to ensure everything remained bit-identical.

 

Does this totally negate the test? I don't think so. We proved (to a 99% probability) that bit-identical playback can sound different... which was the aim, after all.

 

20 minutes ago, pkane2001 said:

I don't recall seeing anything to prove that the issue came from Tascam recorder.

 

I later proved that it was the Tascam's auto-start function.

 

20 minutes ago, pkane2001 said:

150ms is plenty of time to create an audible tell...

 

Where did you get 150ms from?

 

20 minutes ago, pkane2001 said:

I also recall very large errors in the ADC captures from the test. You had to rerun these multiple times after the fact and with different ADCs.

 

This had no bearing on the ABX whatsoever.

 

Sorry Paul, I think you're throwing around a lot of red herrings. But I do agree that we should have chosen a DAC less susceptible to the effects of jitter.

 

[manisandher]

29 minutes ago, The Computer Audiophile said:

Seems like it could all be duplicated by anyone. That’s kind of how science works. 

 

Except that you'd need someone open to the possibility. Indeed, @mansr couldn't hear any difference between A and B even with my pointing the differences out to him... And this with a supposedly highly susceptible DAC!

 

No, his conclusion (after my refuting all his objections) was that the 9/10 must have been a fluke on my part. And people wonder why those of us who routinely hear such things aren't keen to engage in such tests?

 

Mani.

[manisandher]

3 minutes ago, pkane2001 said:

The fact that you took two tests and failed one is another procedural problem. In real testing, you can’t pick and chose test results, regardless of what you think caused you to fail. Each test is significant and should count.

 

I agree that you can't pick and choose which test results you include. But what you're implying here is totally wrong Paul.

 

I actually took three tests in total. The first two were: A, B, X, X, X, X, X, X, X, X, X, X. In both of these, after the first X, I had no reference to go by. I came back into the office (where Mans was controlling the test) in the full knowledge that I had been simply guessing. But these tests actually proved very valuable. They proved that there were no tells during playback.

 

The third test was: ABX, ABX, ABX, ABX, ABX, ABX, ABX, ABX, ABX, ABX. I scored 9/10.

 

Conclude from this what you will.

 

Mani.

 

[manisandher]

8 hours ago, pkane2001 said:

The digital recorder error was for a number of milliseconds at the beginning.

 

Actually, it was 8000 samples @ 176.4k, so around 45ms. But the discrepency was minute - from Mans's analysis:

 

On 3/27/2018 at 12:42 AM, mansr said:

Here you can see the difference:

 

Indeed, Mans was happy to accept that the signals all remained bit-identical throughout.

 

And in any event, this had no effect on the ABX. The bits entering the DAC were not affected in any way, shape or form. Only the first ~8000 samples of some digital captures (not all) were mangled by the auto-start function of the recorder.

 

Mani.

[manisandher]

8 hours ago, pkane2001 said:

Bit identical can sound different, as noise and timing errors can interfere. But maybe that’s all you were trying to prove to Mans. If so, the test design was OK, but the result is really not that surprising, and to me, not that interesting.

 

Yes, that was all we were trying to prove!

 

What's interesting for me is that it seems you can influence the noise and timing errors with bit-identical changes in a software player.

 

On 2/9/2018 at 9:38 PM, manisandher said:

A 10kHz sine tone was captured at the analogue outputs of the DAC, in configuration '1' (SFS=0.1) and configuration '2' (SFS=200). The configurations '1' and '2' were measured to be bit-identical. Mans found a small difference in the FFTs of these captures, due to a small difference in jitter between them:

 

 

 

But really, these captures would need to be repeated to ensure that the difference seen is not down to random fluctuations.

 

Mani.

[manisandher]

1 hour ago, PeterSt said:

It is the first time that I hear about this.

 

Hey Peter, it's all documented in the thread.

 

1 hour ago, PeterSt said:

If true, it sheds some new lights on matters like these, namely that apparently people exist that even after pointing out, still won't hear.

 

I'm certain it was because he didn't want to hear. A bias working in reverse perhaps?

 

Mani.

[manisandher]

BTW, where's @March Audio gone?

[manisandher]

16 minutes ago, pkane2001 said:

I disagree. A 45msec error can’t be safely ignored. To me, this is an error in measurement that needs to be eliminated if we are to trust the test results. Where’s the recording demonstrating that this error was not in the original digital stream?

 

IIRC, the auto-start did something weird like swapping the channels (for the first ~45ms)... of just a few of the digital captures. None of the analogue captures displayed this behaviour, and so the 'glitch' in the digital capturing equipment had no bearing whatsoever on what I heard during the test. I can look into it further when I have some time, if you'd like.

 

But quite honestly, I'd be more inclined to use my time in repeating some tests using my RME.

[manisandher]

29 minutes ago, pkane2001 said:

 

If you do, use RME ADC at 384k rate with slow filter. This compensates for the effects of the digital filter up to 100kHz or so. At least that's what RME recommends.

 

OK, will do.

 

Meanwhile, feel free to take a quick look at the digital captures yourself (you'll no doubt be able to do things a lot faster than I can). Here they are for the ABX (30 samples in total), along with the original file used for playback:

 

https://drive.google.com/drive/folders/1z-oXn3Dvouop0pKdjTXliYseGqUnVaOt?usp=sharing

[manisandher]

All the files ending in 0.1 are A. Those ending in 200 are B. (0.1 and 200 are essentially buffer settings in XXHighEnd.)

 

So each ABX starts with 0.1, then 200, and then X (0.1 or 200).

[manisandher]

3 minutes ago, pkane2001 said:

We should probably switch this to another thread, as this is going completely OT here.

 

The original thread is probably as good a place as any?