BLIND TEST INVITE: Do digital audio players sound different?

[Archimago]

Greetings guys and gals. For those who might be unaware, I opened up another one of these "Internet Blind Tests" this past weekend to have folks test out some samples and report back on what they hear. As you might recall, the last time we did this was back in 2017 with MQA.

The question this time is actually much more "basic". It's one where one can find all kinds of opinions but I have yet to see any clear data using modern devices which the consumer has easy access to. Do CD/digital players converting 16/44.1 sound the same/different? If so, how different?

Come and try the "Blind Test": INTERNET BLIND TEST: Do digital audio players sound different? (Playing 16/44.1 music.)

4 "blinded" devices to listen to from different classes, 4 sample excerpts to try. Which device(s) sound best? Is there a big difference? Lemme know! It might affirm your opinion, or maybe you'll be surprised by your results...

Test closes on April 30, 2019. Plenty of time to listen and let me know. As usual, once the test closes, I'll let you guys know which devices were used in the recordings and how people voted. Plus I'll show you the measured results as well. Feel free to spread this around.

Have fun listening!
Arch

[STC]

Arch, thank you for your continuous effort to help those who want to be helped. Meanwhile, why not also include one with the volume being level matched. IME, that is usually the deciding factor provided other measurements do not produce any audible effect.

[pkane2001]

1 hour ago, STC said:

Arch, thank you for your continuous effort to help those who want to be helped. Meanwhile, why not also include one with the volume being level matched. IME, that is usually the deciding factor provided other measurements do not produce any audible effect.

 

The few files I checked all match to around 0.01dB or better.

 

@Archimago: were the digital players set to upsample 44.1K content to 96K? It appears that different filters were applied to different captures, all above 22.05KHz, with different noise shaping.

 

[STC]

2 minutes ago, pkane2001 said:

The few files I checked all match to around 0.01dB or better.

 

My mistake again. Meant to say without the volume level matched. 

[Ralf11]

Thx for doing this!  Will give it a try as time permits

[Axial]

Excellent, right on right on right on! 

[Miska]

7 hours ago, pkane2001 said:

 

@Archimago: were the digital players set to upsample 44.1K content to 96K? It appears that different filters were applied to different captures, all above 22.05KHz, with different noise shaping.

 

I understood it is what ever the DAC in the player does...

[pkane2001]

25 minutes ago, Miska said:

 

I understood it is what ever the DAC in the player does...

 

I assume some are configurable. But it wasn’t clear to me if the ones in this test were configured to be as similar as possible or just using random or default settings.

[luisma]

So we are evaluating specific player DAC's here, the player is the only variable, everything else is the same

[jabbr]

Looks reasonable at first glance and since you are including Cécile McLorin Salvant as a test, the process might be enjoyable.

 

Question: how have you validated this as a test procedure? Its not immediately obvious to me that this works as a test procedure -- it might but I'm not sure -- might be valid in certain cases but not others etc etc

[audiobomber]

35 minutes ago, luisma said:

So we are evaluating specific player DAC's here, the player is the only variable, everything else is the same

I believe these are digital music sources, not just DAC's. For example, could be a Chromecast Audio streamer and its internal DAC vs. CCA with external DAC vs. a Raspberry Pi with whatever DAC vs. a CD player. 

[Miska]

53 minutes ago, jabbr said:

Looks reasonable at first glance and since you are including Cécile McLorin Salvant as a test, the process might be enjoyable.

 

Question: how have you validated this as a test procedure? Its not immediately obvious to me that this works as a test procedure -- it might but I'm not sure -- might be valid in certain cases but not others etc etc

 

It just barely catches the first image band between 22.05 - 44.1 kHz and a little bit of next between 44.1 kHz and 48 kHz. One of the devices is clean though. The ADC anti-alias filter fixes rest by improving the reconstruction by removing further image frequencies. Strongest image for most current DACs is around 352.8 kHz.

 

Of course playback system then defines how much overlay of these come back again at different frequencies from the DAC, since it is now running at different rate family than the original source.

 

This doesn't really replicate real device playback performance, but at least it captures some apparent differences. So it gives kind of hint or shadow of how the device actually did.

 

[fas42]

Finally - after blundering doing the download several times, - had a quick listen ... for a particular track, very obvious differences between some of the devices, without going to any trouble optimising the playback. Now, what's causing that? Differences as mentioned above affecting my simple playback chain - or has the device playback variation been captured - will be interesting to do some exploring ...

[STC]

1 hour ago, fas42 said:

Finally - after blundering doing the download several times, - had a quick listen ... for a particular track, very obvious differences between some of the devices, without going to any trouble optimising the playback. Now, what's causing that? Differences as mentioned above affecting my simple playback chain - or has the device playback variation been captured - will be interesting to do some exploring ...

 

That was quick. Can you do that with MP3 too? 

[fas42]

1 minute ago, STC said:

 

That was quick. Can you do that with MP3 too? 

 

ST, your samples were obviously different on a casual listen -  which is not the same thing as listening closely to determine what the pluses and minuses are ...

[Archimago]

On 1/29/2019 at 3:46 AM, STC said:

Arch, thank you for your continuous effort to help those who want to be helped. Meanwhile, why not also include one with the volume being level matched. IME, that is usually the deciding factor provided other measurements do not produce any audible effect.

 

Thanks for the note STC. For this as you can see, the volume is closely matched :-).

 

 

[Archimago]

On 1/29/2019 at 5:46 AM, pkane2001 said:

 

The few files I checked all match to around 0.01dB or better.

 

@Archimago: were the digital players set to upsample 44.1K content to 96K? It appears that different filters were applied to different captures, all above 22.05KHz, with different noise shaping.

 

 

No special upsampling done. The >22kHz content are from the devices themselves and of course the small amount of modulator noise from the ADC.

 

I used the default settings (or at least a common setting) for the devices.

[Archimago]

On 1/29/2019 at 11:36 AM, Ralf11 said:

Thx for doing this!  Will give it a try as time permits

Ya got until end of April 🙂

 

[Archimago]

On 1/29/2019 at 2:23 PM, jabbr said:

Looks reasonable at first glance and since you are including Cécile McLorin Salvant as a test, the process might be enjoyable.

 

Question: how have you validated this as a test procedure? Its not immediately obvious to me that this works as a test procedure -- it might but I'm not sure -- might be valid in certain cases but not others etc etc

 

?What validation are we looking for in this experiment?

 

These are all 16-bit devices being recorded using the same equipment capable of greater resolution at higher-than-16/44. I know the cables I use and the ADC is capable of this higher resolution based on other tests. There are of course things I cannot control for but I trust that within reason, if there are very significant audible differences, the home listener with good equipment can at some level detect the significant.

 

All I can say is that when I reveal the devices used, I think audiophiles will recognize that the price points and types of devices being used are quite different! 

[Archimago]

On 1/29/2019 at 3:16 PM, Miska said:

 

It just barely catches the first image band between 22.05 - 44.1 kHz and a little bit of next between 44.1 kHz and 48 kHz. One of the devices is clean though. The ADC anti-alias filter fixes rest by improving the reconstruction by removing further image frequencies. Strongest image for most current DACs is around 352.8 kHz.

 

Of course playback system then defines how much overlay of these come back again at different frequencies from the DAC, since it is now running at different rate family than the original source.

 

This doesn't really replicate real device playback performance, but at least it captures some apparent differences. So it gives kind of hint or shadow of how the device actually did.

 

 

I'll have to disagree with you @Miska about this though: "This doesn't really replicate real device playback performance". IMO, high quality 24/96 capture is all we need especially with these devices...

[Archimago]

On 1/29/2019 at 9:08 PM, fas42 said:

Finally - after blundering doing the download several times, - had a quick listen ... for a particular track, very obvious differences between some of the devices, without going to any trouble optimising the playback. Now, what's causing that? Differences as mentioned above affecting my simple playback chain - or has the device playback variation been captured - will be interesting to do some exploring ...

 

Great @fas42. Looking forward to your submission... Hopefully it's already in :-).

 

 

[Ralf11]

I'd love to see some Ethernut cable tests, and one where various processes in the computer's OS were turned on & off.

[jabbr]

12 minutes ago, Archimago said:

?What validation are we looking for in this experiment?

 

The same validation that anyone would want with any experiment particularly when using a new measurement technique ... but specifically: are the SQ differences in the recordings, the same as they would be with the device in my home? 

 

The certainly might be, particularly if the changes are dramatic, but (not knowing the details) I can think of a bunch of reasons that 24/96 wouldn’t capture small differences in devices. My back of the hand rule of thumb is that I want my measuring device to have 10x the resolution of the difference I am trying to measure...

 

That said, the ADC you’ve selected looks very reasonable and a good choice.

[Miska]

31 minutes ago, Archimago said:

I'll have to disagree with you @Miska about this though: "This doesn't really replicate real device playback performance". IMO, high quality 24/96 capture is all we need especially with these devices...

 

It doesn't capture everything the devices put out, that's why it doesn't replicate the real performance...

 

For example for the TEAC UD-501 to reproduce all it puts out, you need to capture at least 1.5 MHz wide spectrum.

 

This is output of UD-501 when input is 44.1k sample rate sweep from 0 - 22.05 kHz and UD-501's digital filter is set to "sharp".

[Miska]

7 minutes ago, jabbr said:

The certainly might be, particularly if the changes are dramatic, but (not knowing the details) I can think of a bunch of reasons that 24/96 wouldn’t capture small differences in devices. My back of the hand rule of thumb is that I want my measuring device to have 10x the resolution of the difference I am trying to measure...

 

It cannot capture even many of the large differences of D/A converter and analog reconstruction filter performance, because it's Nyquist is even lower than Nyquist of digital filter of practically any oversampling DAC. And lower than -3 dB point of most analog reconstruction filters...