BLIND TEST INVITE: Do digital audio players sound different?

[beerandmusic]

On 1/28/2019 at 6:36 PM, Archimago said:

Greetings guys and gals. For those who might be unaware, I opened up another one of these "Internet Blind Tests" this past weekend to have folks test out some samples and report back on what they hear. As you might recall, the last time we did this was back in 2017 with MQA.

The question this time is actually much more "basic". It's one where one can find all kinds of opinions but I have yet to see any clear data using modern devices which the consumer has easy access to. Do CD/digital players converting 16/44.1 sound the same/different? If so, how different?

Come and try the "Blind Test": INTERNET BLIND TEST: Do digital audio players sound different? (Playing 16/44.1 music.)

4 "blinded" devices to listen to from different classes, 4 sample excerpts to try. Which device(s) sound best? Is there a big difference? Lemme know! It might affirm your opinion, or maybe you'll be surprised by your results...

Test closes on April 30, 2019. Plenty of time to listen and let me know. As usual, once the test closes, I'll let you guys know which devices were used in the recordings and how people voted. Plus I'll show you the measured results as well. Feel free to spread this around.

Have fun listening!
Arch

 

I have ALWAYS had a very difficult time believing that the digital front end would make much difference, that if there was any difference that it would be very difficult to distinguish even in a blind test....that said, i do believe that cd's will sound different based mostly on their dac, but if you used a digital output into the same dac, that again, it would be very difficult to distinguish differences, let alone which one sounded better for all types of music. 

 

My guess is that if you do get enough people doing the test, that there will be no concensus as to one digital front end being superior to any other.

 

as always...put your money in your speakers and your amp!

 

[pkane2001]

4 hours ago, Miska said:

 

It cannot capture even many of the large differences of D/A converter and analog reconstruction filter performance, because it's Nyquist is even lower than Nyquist of digital filter of practically any oversampling DAC. And lower than -3 dB point of most analog reconstruction filters...

 

 

Jussi, is that really important? If there is a 1MHz signal at -100dB, how is this going to affect any normal audio system? And even if it does through IM or through another mechanism, wouldn’t the ADC that Arch is using capture anything that is reflected into the audible range with plenty of margin? While it may be interesting to see the MHz range effects of a DAC as a curiosity, why is this significant for this particular test?

[jabbr]

1 hour ago, pkane2001 said:

 

Jussi, is that really important? If there is a 1MHz signal at -100dB, how is this going to affect any normal audio system? And even if it does through IM or through another mechanism, wouldn’t the ADC that Arch is using capture anything that is reflected into the audible range with plenty of margin? While it may be interesting to see the MHz range effects of a DAC as a curiosity, why is this significant for this particular test?

 

You tell me, I mean mathematically 24/96 doesn’t capture everything (and we can go into this in much more detail there’s really no question about this) ...

 

So tell me, how is this validated? How have you determined what is enough? No tell me, how have you determined what “enough margin” is? ... this is actually a very common mistake and I can tell you that in my area of expertise : digital imaging, this mistake is made all the time : look up “sub pixel” imaging/resolution etc to get a hint at some of the math ...

[pkane2001]

21 minutes ago, jabbr said:

 

You tell me, I mean mathematically 24/96 doesn’t capture everything (and we can go into this in much more detail there’s really no question about this) ...

 

So tell me, how is this validated? How have you determined what is enough? No tell me, how have you determined what “enough margin” is? ... this is actually a very common mistake and I can tell you that in my area of expertise : digital imaging, this mistake is made all the time : look up “sub pixel” imaging/resolution etc to get a hint at some of the math ...

 

Well, you tell me. If the point of the test it to determine the audibility of some process, is it enough to measure just the audible range, or must I measure everything up to the GHz range and above? If there is some distortion well above audible frequencies and it doesn't get reflected in the audible range, why should I care?

 

(btw, I don't need to look up sub pixel resolution, I've been coding image processing algorithms for about 30 some years, everything from resampling to filters in frequency domain, to feature extraction, PSF analysis, reconstruction and deconvolution )

 

[PeterSt]

5 minutes ago, pkane2001 said:

If there is some distortion well above audible frequencies and it doesn't get reflected in the audible range, why should I care?

 

Because it may or even will after all ? Take HF IMD as an example ...

So we should care about the bandwidth of the whole system, and that anticipating HF "source" behavior in the first place. For example, have your reconstruction not the best and reproduce that with a high bandwidth system and you're cooked.

 

And so I think that this test may not even work out the same for everyone. It depends on where it is played.

[PeterSt]

3 minutes ago, PeterSt said:

Take HF IMD as an example ...

 

I meant HF signals that imply in-band IMD.

[pkane2001]

27 minutes ago, PeterSt said:

 

Because it may or even will after all ? Take HF IMD as an example ...

So we should care about the bandwidth of the whole system, and that anticipating HF "source" behavior in the first place. For example, have your reconstruction not the best and reproduce that with a high bandwidth system and you're cooked.

 

And so I think that this test may not even work out the same for everyone. It depends on where it is played.

 

That's fair,  Peter, but I don't think this test is about what these players might sound like in some other system. The test files are produced by @Archimagoand his system and captured by his ADC. I assume he's not planning on generalizing the results of this one test to apply to all other systems out there, at least I wouldn't. It's just a single data point, just like me testing your USB cable in my own system 

 

[Ralf11]

how high must we go to get the phase right?

[esldude]

It would obviously be highly important.  -100 db at 1 mhz, and then fed into a speaker or headphone........its going to get reproduced how exactly?  Sure seems as if 96 khz is going to be enough on the bandwidth end anyway.  

[fas42]

Quite easy to see why the versions might sound different - I had a look at the waveforms of the particular track I listened to, and just aligned them, and filtered out everything below 5kHz. In "interesting" parts of the track the waveforms very obviously visually differed - was the variation due to the playback, or capture process, or the combination?

[AudioDoctor]

Subscribing

[ferenc]

The real question is:

 

is there any relevance of a test likes this in a real listening environment? If one wants to build an audio system listening music?

 

The "distortions" of the real acoustical environment, loudspeakers will mask the difference anyway, if there is any. 

What can we learn depending if we hear the difference or do not hear anything? 

[Miska]

3 hours ago, esldude said:

It would obviously be highly important.  -100 db at 1 mhz, and then fed into a speaker or headphone........its going to get reproduced how exactly?  Sure seems as if 96 khz is going to be enough on the bandwidth end anyway.  

 

Question is how much intermodulation products you get from your amplifiers and other gear for example from the the images around 352.8 kHz? This is music content with inverse and forward spectrum at about -65 dB level, so for example 1 kHz tone in the base band has 2 kHz difference around 352.8 kHz.

 

For example these kind of things are what make DACs different. And the sample uses 96 kHz sampling rate, so it has 48 kHz bandwidth.

 

But sure, still regardless of limitations of the test, no problems hearing differences.

[Miska]

7 hours ago, pkane2001 said:

Jussi, is that really important? If there is a 1MHz signal at -100dB, how is this going to affect any normal audio system? And even if it does through IM or through another mechanism, wouldn’t the ADC that Arch is using capture anything that is reflected into the audible range with plenty of margin? While it may be interesting to see the MHz range effects of a DAC as a curiosity, why is this significant for this particular test?

 

Look more at the 352.8 kHz which is output rate of the digital filter. That is only down a bit over -65 dB.

 

ADC would capture it only if the IM products are from the DAC itself. Since the recording is likely not from loudspeaker terminals, it doesn't capture IM products of rest of the chain. And even then it would only apply to the particular setup.

 

If you want to test DACs, you need to capture all that comes out and not just selectively one piece of it based on some assumptions.

 

If you'd use only proper digital filters and a DAC with good analog filters, you wouldn't need to worry about these details, because there wouldn't be anything outside your audio band.

 

You also create another aspect here is that what ever method is used to play back these samples will add it's own fingerprint on top.

 

[Miska]

3 hours ago, Ralf11 said:

how high must we go to get the phase right?

 

I think this test captures relevant phase errors, because largely their practical impact reaches only up to about 30 kHz.

 

I'm not sure which ADC anti-alias filter setting @Archimago used in this test... They also have some impact on the result, depending on the setting.

 

[jabbr]

16 hours ago, pkane2001 said:

Well, you tell me. If the point of the test it to determine the audibility of some process, is it enough to measure just the audible range, or must I measure everything up to the GHz range and above? If there is some distortion well above audible frequencies and it doesn't get reflected in the audible range, why should I care?

 

(btw, I don't need to look up sub pixel resolution, I've been coding image processing algorithms for about 30 some years, everything from resampling to filters in frequency domain, to feature extraction, PSF analysis, reconstruction and deconvolution )

 

My answer is : no it isn't "enough" to measure the audible range... if that were the case then why not just measure 16/44? ... answer that for yourself, and start from there ... in any case a validation process might indeed sweep into the Mhz or Ghz range to see if that might have an effect (e.g. IMD), rather than simply assuming.

 

In any case I'm not saying that the testing procedure is incapable of demonstrating differences, rather I was asking if the testing procedure had been validated ... it doesn't seem to be, and this is intended to capture large differences, and of course large differences in the audible range would be expected to have the biggest SQ difference, and might be totally fine for the purposes here ...

[esldude]

So ghz has to be taken into account.  Or megahertz.  

 

So even at 352 or 384 khz already 65 db down in one example.  And we are worried about IMD showing up back in the audio band?  What is the IMD level here?  If it were 100% it is down 65 db and not likely heard due to masking by music.  Almost surely the IMD will be at a lower level than the originating signals by quite a large amount.  

 

Does anyone have any examples of IMD showing up in the audible band at audible levels with such things?

 

Oh, and it is perfectly okay on the other hand to use those slow roll off audiophile reconstruction filters that have real levels of aliasing/imaging because they sound better. 

[jabbr]

1 minute ago, esldude said:

So ghz has to be taken into account.  Or megahertz.  

 

 

Well I once had a jfet which worked fine in a circuit (and sounded great) and later I built another and lo and behold it came from a different plant, and was oscillating way up in the upper Mhz region, too high for my oscilloscope at the time to measure, and it sounded like sh*t, and when I remeasured it with a Ghz scope I saw the problem ...

 

Does this always happen: no! can it happen: yes!

[esldude]

1 minute ago, jabbr said:

 

Well I once had a jfet which worked fine in a circuit (and sounded great) and later I built another and lo and behold it came from a different plant, and was oscillating way up in the upper Mhz region, too high for my oscilloscope at the time to measure, and it sounded like sh*t, and when I remeasured it with a Ghz scope I saw the problem ...

 

Does this always happen: no! can it happen: yes!

And did it effect the 20 khz band results while it was oscillating?  In other words was it causing distortion or IMD products to show up in your conventional audio band tests?

 

I had a DAC that had a 1.1 mhz high level idle tone once.  No problem except I had a Spectral amp.  Happily able to reproduce that tone.  Didn't effect sound I could tell, but used about 20 wpc all the time which would make the amp warm up more than it should listening at normal levels.  Presumably had it used more power it could have effected the audible band for lack of spare power.  And presumably testing such would have revealed some problem just in the audio band even had I not known what.

[pkane2001]

8 minutes ago, jabbr said:

 

My answer is : no it isn't "enough" to measure the audible range... if that were the case then why not just measure 16/44? ..

 

I would have absolutely no problem with that. Maybe 24/48, just to leave a margin for ADC filter outside the audible range. 24/96 if you want to be absolutely, positively, 110% safe.

 

21 minutes ago, jabbr said:

a validation process might indeed sweep into the Mhz or Ghz range to see if that might have an effect (e.g. IMD), rather than simply assuming

 

We are not talking about measurements here, we are checking if a component produces audible distortions. Sweeping to GHz range will tell me nothing about audibility. Say there is a -100dB noise at 1MHz in the sweep. What can I conclude about its audibility? But, if I see this tone producing a 1KHz IMD, then I can certainly make claims about it being audible (or not).

 

15 minutes ago, jabbr said:

large differences in the audible range would be expected to have the biggest SQ difference, and might be totally fine for the purposes here

 

That's all I've been trying to say. 

[jabbr]

1 minute ago, esldude said:

And did it effect the 20 khz band results while it was oscillating?  In other words was it causing distortion or IMD products to show up in your conventional audio band tests?

 

Um, a gate stopper fixed it ... we could speculate. My point is that, essentially in every field, if you don't look for stuff, you won't find stuff.*

 

* to paraphrase Kuhn

[esldude]

1 minute ago, jabbr said:

 

Um, a gate stopper fixed it ... we could speculate. My point is that, essentially in every field, if you don't look for stuff, you won't find stuff.*

 

* to paraphrase Kuhn

IMO, Kuhn wouldn't be happy with your use of of his idea in this context. 

 

You can't look at everything, so unless you see problems crop up where it matters, it is wasteful to look all over the place.  

[Ralf11]

52 minutes ago, jabbr said:

 

My answer is : no it isn't "enough" to measure the audible range... if that were the case then why not just measure 16/44? ... answer that for yourself, and start from there ... in any case a validation process might indeed sweep into the Mhz or Ghz range to see if that might have an effect (e.g. IMD), rather than simply assuming.

 

In any case I'm not saying that the testing procedure is incapable of demonstrating differences, rather I was asking if the testing procedure had been validated ... it doesn't seem to be, and this is intended to capture large differences, and of course large differences in the audible range would be expected to have the biggest SQ difference, and might be totally fine for the purposes here ...

 

post-hoc validation ??

 

or at east, post hack argumentation

 

of course, we do not know which players were used, so it is possible that the range of types is so wide that some differences will be heard no matter what

[fas42]

1 hour ago, esldude said:

IMO, Kuhn wouldn't be happy with your use of of his idea in this context. 

 

You can't look at everything, so unless you see problems crop up where it matters, it is wasteful to look all over the place.  

 

When you learn the art of 'one ear listening', then you can hear problems galore. Where they are cropping up is the biggy - it's a combination of experience, intuition, engineering, and common sense that gets the answers - an adequate fix can almost be trivial to implement.

[Ralf11]

43 minutes ago, fas42 said:

 

When you learn the art of 'one idea spamming', then no one can hear anything except problems galore from your posts. Where they are cropping up is the biggy - it's a combination of low level AI and fast servers that gets the spam all over every audio site on the internet - an adequate fix can almost be trivial to implement - you ban those IPs.

 

fify