red or blue pill - Part II?

[mansr]

1 hour ago, pkane2001 said:

1. Test something that's more of a common concern. A setting in Phasure XXX software is too obscure and not of much interest to anyone who doesn't use this software (I'd say the majority here).

My main concern with this is that PeterSt is, to put it mildly, extremely coy about what this setting does. We have no idea whatsoever what is going on. Furthermore, I was initially told the comparison would be between playback of the same file from local vs network storage and/or Tidal. Only on arrival at Mani's house did I find out that he had changed his mind.

 

1 hour ago, pkane2001 said:

2. Run more tests. You had a few low-scoring attempts before getting 9/10 (I'm aware of the reasons you gave, but an explanation of why a test failed is usually not acceptable in lieu of a positive result). Two consecutive 9/10 tests or 8/10 would be much more convincing. If you do run multiple tests, the results of all of those tests should be included in the calculation, good or bad.

Very much this. Although both the music selection and the test protocol were altered, the overall impression is much weaker than some would make it out to be.

 

1 hour ago, pkane2001 said:

3. Use better-vetted equipment to record digital and analog feeds -- I was confused by the provided files when trying to analyze them, there were some issues with one ADC and different issues with another, etc. It wasn't clear to me which recordings were accurate representation of what was tested (if any).

The equipment was dreadful. I offered to bring my own audio interface and run it in pass-through mode between the DAC and amp. I was told this wouldn't work for vague reasons. Instead, we were limited to recording some samples afterwards using an ADC that turned out to have some issues.

 

Then there's the general environment of it all. The playback computer and DAC were sitting in a cluttered basement with god knows what electrical interference. The DAC itself was a naked PCB nailed to a wooden plank. This was connected to the monoblock amps (on the ground floor) with 10 metres or so of coax (RG-59 or similar).

 

For any future tests I'd insist on containing all the equipment in a somewhat controlled setting (not necessarily a shielded room). Then I'd record the output of the DAC and/or the amps during the listening test rather than separately.

 

1 hour ago, pkane2001 said:

4. Document and review the test procedure with us before the test, not after. 

+1

 

1 hour ago, pkane2001 said:

5. Some photos/video recording of the session might be helpful in reviewing. The blinding procedure should be carefully considered and any possible tells eliminated (not accusing you of cheating, but there are often tells that have nothing to do with the actual audio, like a small click, or a slightly longer delay before playing, etc.)

Mani had the opportunity to cheat, but I really don't think he did. Still, I'd remove that temptation in any rerun.

 

As for tells, the software had a huge delay (more than 10 seconds) with one of the settings. Since I was unprepared for this, the best I could do was try to mask it by waiting a randomish time before pressing play on each iteration. The timestamps of the recordings didn't show any pattern obvious at a glance, but these things can be rather insidious. I'd prefer to have the entire thing automated.

 

All that said, at this point, this is little more than a thought experiment. The "believers" will never be swayed by any test, and I have a hard time imagining any of them going through the rigours it would take to produce compelling evidence in their favour.

[mansr]

4 minutes ago, Archimago said:

Wowzers. Pictures are definitely mandatory for the reader to understand what the heck is going on over there!

Being there wasn't sufficient. Pictures couldn't convey the half of it.

[mansr]

26 minutes ago, semente said:

Will capturing the output affect the audibility of the differences?

Maybe, but as long as the difference is audible during the capture, it doesn't matter.