According to Barry Diament, the quality of the original recording in terms of mike setup, etc. can fully be had from PCM done properly at 4x rates (24/176.4 or 24/192), i.e. he has testified that it is audibly indistinguishable from the live feed in his studio. Also according to him, IIRC the same cannot be said about DSD, and, based on my own, personal experience, I have to perfectly agree with him.

 

That being said, one of the most plausible explanations as to why this is so IMO is the long established fact 1-bit quantizers, when properly dithered, are in constant overload. The resulting problems are distortion, noise modulation, and idle tones.

 

While it is true that choosing a higher order Sigma-Delta modulator shifts more noise out of the audible band, it is also true that there exists an optimum order when designing such a modulator, due to practical constraints. However, oversampling not only shifts even more noise out of the audible band, but also shifts it farther above the audible band, where it can be more effectively filtered.

 

On top of that, decimation increases accuracy by means of averaging, i.e. decimation does not in any way imply that redundant sampled values are merely being discarded. So in modern audio ADCs outputting PCM, there is usually a more complex combination of the digital filter and several other optimizations. The idea that properly done DSD is more accurate or more close to the original analog input signal than properly done PCM is a myth.

 

Thanks for a post which I found very informative.