BLIND TEST INVITE: Do digital audio players sound different?

[Miska]

7 hours ago, pkane2001 said:

 

@Archimago: were the digital players set to upsample 44.1K content to 96K? It appears that different filters were applied to different captures, all above 22.05KHz, with different noise shaping.

 

I understood it is what ever the DAC in the player does...

[Miska]

53 minutes ago, jabbr said:

Looks reasonable at first glance and since you are including Cécile McLorin Salvant as a test, the process might be enjoyable.

 

Question: how have you validated this as a test procedure? Its not immediately obvious to me that this works as a test procedure -- it might but I'm not sure -- might be valid in certain cases but not others etc etc

 

It just barely catches the first image band between 22.05 - 44.1 kHz and a little bit of next between 44.1 kHz and 48 kHz. One of the devices is clean though. The ADC anti-alias filter fixes rest by improving the reconstruction by removing further image frequencies. Strongest image for most current DACs is around 352.8 kHz.

 

Of course playback system then defines how much overlay of these come back again at different frequencies from the DAC, since it is now running at different rate family than the original source.

 

This doesn't really replicate real device playback performance, but at least it captures some apparent differences. So it gives kind of hint or shadow of how the device actually did.

 

[Miska]

31 minutes ago, Archimago said:

I'll have to disagree with you @Miska about this though: "This doesn't really replicate real device playback performance". IMO, high quality 24/96 capture is all we need especially with these devices...

 

It doesn't capture everything the devices put out, that's why it doesn't replicate the real performance...

 

For example for the TEAC UD-501 to reproduce all it puts out, you need to capture at least 1.5 MHz wide spectrum.

 

This is output of UD-501 when input is 44.1k sample rate sweep from 0 - 22.05 kHz and UD-501's digital filter is set to "sharp".

[Miska]

7 minutes ago, jabbr said:

The certainly might be, particularly if the changes are dramatic, but (not knowing the details) I can think of a bunch of reasons that 24/96 wouldn’t capture small differences in devices. My back of the hand rule of thumb is that I want my measuring device to have 10x the resolution of the difference I am trying to measure...

 

It cannot capture even many of the large differences of D/A converter and analog reconstruction filter performance, because it's Nyquist is even lower than Nyquist of digital filter of practically any oversampling DAC. And lower than -3 dB point of most analog reconstruction filters...

 

[Miska]

3 hours ago, esldude said:

It would obviously be highly important.  -100 db at 1 mhz, and then fed into a speaker or headphone........its going to get reproduced how exactly?  Sure seems as if 96 khz is going to be enough on the bandwidth end anyway.  

 

Question is how much intermodulation products you get from your amplifiers and other gear for example from the the images around 352.8 kHz? This is music content with inverse and forward spectrum at about -65 dB level, so for example 1 kHz tone in the base band has 2 kHz difference around 352.8 kHz.

 

For example these kind of things are what make DACs different. And the sample uses 96 kHz sampling rate, so it has 48 kHz bandwidth.

 

But sure, still regardless of limitations of the test, no problems hearing differences.

[Miska]

7 hours ago, pkane2001 said:

Jussi, is that really important? If there is a 1MHz signal at -100dB, how is this going to affect any normal audio system? And even if it does through IM or through another mechanism, wouldn’t the ADC that Arch is using capture anything that is reflected into the audible range with plenty of margin? While it may be interesting to see the MHz range effects of a DAC as a curiosity, why is this significant for this particular test?

 

Look more at the 352.8 kHz which is output rate of the digital filter. That is only down a bit over -65 dB.

 

ADC would capture it only if the IM products are from the DAC itself. Since the recording is likely not from loudspeaker terminals, it doesn't capture IM products of rest of the chain. And even then it would only apply to the particular setup.

 

If you want to test DACs, you need to capture all that comes out and not just selectively one piece of it based on some assumptions.

 

If you'd use only proper digital filters and a DAC with good analog filters, you wouldn't need to worry about these details, because there wouldn't be anything outside your audio band.

 

You also create another aspect here is that what ever method is used to play back these samples will add it's own fingerprint on top.

 

[Miska]

3 hours ago, Ralf11 said:

how high must we go to get the phase right?

 

I think this test captures relevant phase errors, because largely their practical impact reaches only up to about 30 kHz.

 

I'm not sure which ADC anti-alias filter setting @Archimago used in this test... They also have some impact on the result, depending on the setting.

 

[Miska]

18 hours ago, esldude said:

Does anyone have any examples of IMD showing up in the audible band at audible levels with such things?

 

Yes, take almost any NOS R2R ladder DAC and listen how it sounds!

 

18 hours ago, esldude said:

Oh, and it is perfectly okay on the other hand to use those slow roll off audiophile reconstruction filters that have real levels of aliasing/imaging because they sound better. 

 

That is precisely why I'm against them and use digital filters that have >192 dB stop-band attenuation and Nyquist at 6 or 12 MHz! So first image would be there instead of 352.8 kHz... Quite a bit easier for low-order analog filters in the DAC output.

 

Quote

So ghz has to be taken into account.  Or megahertz.  

 

So even at 352 or 384 khz already 65 db down in one example.  And we are worried about IMD showing up back in the audio band?  What is the IMD level here?  If it were 100% it is down 65 db and not likely heard due to masking by music.  Almost surely the IMD will be at a lower level than the originating signals by quite a large amount.

 

Now you are throwing in quite a bit of assumptions. I don't like "almost surely" and such. I just go and look at everything. If you are fine to go with mediocre "good enough", it is all fine for me. But I personally  want things perfect and then improved all over to be even more perfect. Things cannot be "too perfect", ever.

[Miska]

18 hours ago, pkane2001 said:

I would have absolutely no problem with that. Maybe 24/48, just to leave a margin for ADC filter outside the audible range. 24/96 if you want to be absolutely, positively, 110% safe.

 

Then you both test your ADC anti-alias filter heavily (they are not actually designed to clean up DAC outputs, but instead real analog sources). And if your ADC anti-alias filter manages well, it massively cleans up output of typical DAC. But of course if that' what you specifically want...

 

In any case, just take that into account, it is a fact that you are altering the DAC output signal by doing that.

[Miska]

44 minutes ago, esldude said:

Here are some tests I did of aliasing with a some recording interface ADCS I had on hand. None are NOS R2R, and there is a good reason I don't own such DACs. 

 

https://www.audiosciencereview.com/forum/index.php?threads/comparing-aliasing-in-three-adcs.3272/

 

Now all of these were with max signals.  So an ADC will cut off almost completely any ultrasonic garbage a DAC is putting out.  Of course the recordings under discussion were done to 96 khz.  Is there anything put out by such players above what 96 khz rates sample that will heavily react to create signals down into the audible band?  Seems unlikely.  

 

So for this set of ADC you do get aliasing. You could have eased up your testing by checking the ADC chip data sheets about the anti-alias filter specifications. The AK5574 used in ADI-2 Pro has digital anti-alias filters with 85 dB stop-band attenuation. If the selected filter used was "short-delay" it is minimum-phase so it creates some additional phase shift. If it was "slow-roll off" then it will have some extra aliasing at the top.

Datasheet is here:

https://www.akm.com/akm/en/file/datasheet/AK5574EN.pdf

Page 12 has the filter specs and page 13 has response and ripple plots.

 

Yes, most DACs create such signals and yes they have impact too.

 

In the samples in question here, only one sample seems mostly clean from DAC effects. But the recording captured only small part of these and ADC cleaned up the rest. Still no problem hearing differences. Personally, I would have made the recording at least in DSD256 which is also supported by the ADI-2 Pro.

 

[Miska]

5 hours ago, Archimago said:

This was not a problem with the music used.

 

All the music repeats at those higher frequencies as well with forward and inverse spectrum. So it doesn't matter whether it is music or test tone.

 

To properly capture the behavior, I use sweep tone and "peak hold" spectrum. This works for music as well, leave the spectrum at peak hold and play entire song.

 

My point was mostly that the ADC's anti-alias filter removes those images when you run it at 96k. If you've made the recording at DSD256, a little bit more would have been preserved.

 

[Miska]

3 hours ago, esldude said:

And what's the audible significance  of these images above 40 khz? 

 

I answered that already earlier. Amount of intermodulation products (and in this case aliasing) you get from those fully correlated inverse and forward frequency spectrum components is system dependent.

 

Essentially it just demonstrates incomplete and non-precise reconstruction of the original waveform. If you don't capture this for the tests, you don't capture whole picture.

 

Good quality D/A conversion process doesn't produce any images at all.

 

[Miska]

1 hour ago, esldude said:

I agree, and don't promote the idea of poor DA or AD conversion.  But what I've seen is aliasing and imaging in the -90 dbFS and lower range.  This with a full max signal.  Reduce the level of the aliased signals or the signals that image and you reduce what leaks thru.  So just how much can make for any difference that corrupts a test like Archimago's?  Looks to me like pretty much a non-problem on playback and equally so for Archimago recording these devices at 96 khz while they playback at lower rates. 

 

Images are quite a bit higher typically, depending on the reconstruction analog filter. If it is more aggressive with lower cut-off point, then it spoils phase response at top of audio range.

 

RME ADI-2's image levels at 44.1k input for the record:

 

Run the DAC with upsampling to 705.6/768 and it gets much lower, run it with properly upsampled DSD256 in DSD Direct mode and you have no images at all and any noise left overs are lower level than images at 705.6/768k - and totally uncorrelated!

 

1 hour ago, esldude said:

This with a full max signal.

 

Given current loudness wars, RedBook content is typically full max signal at all times with clipping.

 

1 hour ago, esldude said:

Reduce the level of the aliased signals or the signals that image and you reduce what leaks thru.

 

So now you need to select test music so that it works well for a DAC?

 

1 hour ago, esldude said:

So just how much can make for any difference that corrupts a test like Archimago's?

 

You don't know, so you cannot disregard it as an important factor. Let's not make assumptions here.

 

1 hour ago, esldude said:

Looks to me like pretty much a non-problem on playback and equally so for Archimago recording these devices at 96 khz while they playback at lower rates.

 

The recordings reveal some of the badness of the devices, but not all. That's the point.

 

Sounds like you want to conveniently ignore aspects that don't fit in your PCM ideology...

 

[Miska]

As with any recording, you first need to analyze bandwidth needed to accurately reproduce the source you want to record. So you need to check what is the highest frequency it puts out. And then you select sampling rate that is sufficient for the required bandwidth. This is the fundamental requirement for accurate reproduction.

 

I'm not at all saying this is wouldn't already show differences, it does. But it doesn't tell the whole picture, only part of it.

 

[Miska]

59 minutes ago, esldude said:

I don't agree with you here.  If the hearing is the bottleneck of bandwidth, and it is, then you don't need all the frequencies that are above a noise floor out to some megahertz range.  You end up with differences that make no difference.  Analyzing the bandwidth needed in such a case would indicate 96 khz sampling is enough with quite a margin for safety. 

 

Again, intermodulation and other effects they cause... To make such choice you would first need to prove that it certainly doesn't cause any detectable differences in anybody's system.

 

You cannot claim to accurately represent a DAC if you don't capture all of it's output. In addition now you have effect of ADC and another DAC baked in.

 

For me, margin of safety would be to use rate 2x higher than highest detected output frequency from a DAC.

 

Again, why not make the recording for example using DSD256 where you don't have for example effects of the digital anti-alias filtering baked into the result?

 

[Miska]

1 hour ago, esldude said:

So what are the effects of an image at 300 khz?  The amp likely won't respond much usually, the speakers won't for sure.   Our ears don't care much about phase at higher frequencies. 

 

When you play 1 kHz tone, you have it present also at 351.8 kHz and 353.8 kHz and thus their difference frequency of 2 kHz as well.

 

1 hour ago, esldude said:

Well not the way it works.  The 90 db or so I've seen from 192 khz, is with a single tone max frequency.  Get multiple tones or noise and max total signal is still much lower at a given frequency and much lower in what images across it.  Way below the noise floor of any gear for the most part. 

 

Well, see above my real world measurement of ADI-2 how much you have image levels. Can you show some measurements for image levels of 44.1k content played back? You digitization test you referred to earlier is just test of particular ADC anti-alias filtering without even reference to what what original image level from the DAC output. Meaningless stuff.

 

Still far from perfect, from the often touted perfectness of PCM systems.

 

1 hour ago, esldude said:

No I simply don't wish to chase more ghosts that are smaller than the angels dancing on the head of a pin at great complication, expense and no benefit to human ears.

 

Where's the complication and cost? No extra complication, no extra cost. And certainly easy to hear differences.

 

If you cannot hear a difference in your system, that doesn't mean someone else out of billions of people cannot in their own system.

 

You are claiming a system to be perfect based on adjusting requirements and boundaries to match the results. This way of course any system can be always perfect, you just need to make your requirements match it's performance!

[Miska]

I had some fun and recorded 0 - 22.05 kHz 0 dBFS sweep from Meridian Explorer2 DAC. So you can see some fun with the MQA's upsampling filters it has too. Recording was done using ADI-2 Pro AE in both in DSD256 and 768/32 PCM, usable audio bandwidth in both is about the same. File size difference is quite notable though, DSF file is 186 MB (7z compressed 114 MB) and WAV 399 MB (7z compressed 242 MB), when WAV is 705.6/24 the size is 274 MB. So some about of space saving for DSD's benefit, these WAVs cannot be compressed into FLAC because FLAC maxes out at 384/24.

 

Wide band spectrum:

 

Zoom into 250 kHz banwidth:

 

And into 100 kHz bandwidth:

 

 

And for comparison recording done using 96 kHz sampling rate:

Does this tell the whole story?

 

[Miska]

23 minutes ago, esldude said:

IMD would show up even if the ADC is running at 44.1 khz.  You don't need wideband measures for that.  So whatever the ultrasonic sources of IMD into the audible band you'd already be seeing them.  They aren't much apparently. 

 

Only if it originates from the DAC itself, and not for example from power amplifier.

 

23 minutes ago, esldude said:

The reason I don't record such things with DSD256 is I don't have an ADC for that.

 

That's not a technical reason justifying why it wouldn't be helpful. This test was recorded with ADI-2 Pro which can record DSD256.

 

23 minutes ago, esldude said:

I'm not worried if something shows up at -120 db, because you'll not hear it.

 

You cannot make statements about what I will or won't hear. You can only make statements about your own hearing. You would be surprised what people are capable of hearing.

 

23 minutes ago, esldude said:

I've done the reverse test for imaging as well.  Similar levels were seen.

 

Can you explain your testing methodology and DACs in question? At 192k ADC rate you cannot test imaging except for first step of on-chip digital filter. And what I'm talking about here is output rate of the on-chip digital filter at 352.8k (for 44.1k input). You need at least 768k sampling rate to see even lower half of that image.

 

In this another DAC example I posted, that image is down about -45 dB.

[Miska]

Just now, esldude said:

This spectrogram goes to gray at -144 db.  You see an idle tone which is in the Forte DAC at about 31 khz, -95 db in level which isn't great.  You see double that frequency at -125 db.  Otherwise no imaging above -144 db for 2hz-20khz.   On the 2hz-40 khz sweep, you get a little imaging just above 20 khz at about -132 db.  It gets a little higher in level as the frequency increases, but never exceeds -115 db.  This is not a perfect result, but there is not a whole lot up there to 96 khz at least. 

 

 

Because you don't reach output rate of the digital filter at all!

 

You are looking at very narrow frequency band and believe that that's all? That's what I'm trying to say, you are not seeing the whole picture.

[Miska]

3 minutes ago, esldude said:

Okay so how does the imaging of the RME show up where we can hear it?  And I will say if someone is saying they can hear such things at -120 db they'll have to prove it. I'll say I don't believe it for all 7 billion people on the planet.

 

Some people have proven it to me in practice, that's enough for me.

 

Rather than going around asking people to prove their hearing, I rather go to the way that I don't need to think about the subject.

 

I don't understand why you are against correcting measurable and correctable problems?

 

6 minutes ago, esldude said:

Now okay the RME has some imaging at -46 db.

 

RME doesn't, Meridian Explorer 2 does. Which is mostly function of the DAC chip in question (which, by the way has 60 dB stop-band attenuation in the built-in digital filters).

 

Same DAC chip can perform much better already by running it at 352.8/384k with good external upsampling. But you probably want to ignore that too?

 

[Miska]

57 minutes ago, esldude said:

Here are some for the Forte, which I think you also have.  

 

OK, let's take a look at Forte. Test tones as 44.1k sampling rate 32-bit PCM.

 

First playing 1 kHz tone, wide band spectrum:

Whoops, not very clean...

 

Then a 0 - 22.05 kHz sweep:

Now we can see the noise shaping bump, two image pairs and huge HF noise bumps.

 

Let's zoom in to low frequencies:

Here we can see first image pair around 176.4k and second around 352.8k. And also remains of the modulator noise hump. This looks like Cirrus Logic DAC chip, except that for example my own implementations based on their CS4398 DAC chip are totally clean above the noise hump, and so is for example Marantz HD-DAC1 that is based on the same chip. Maybe this is some other chip model from Cirrus. Doesn't explain the HF noise humps though.

 

It also has some aliasing and funky behavior with full levels signals near Nyquist. The switching PSU makes a faint tone just above 30 kHz.

 

 

 

[Miska]

P.S. Possibly the 2.8 MHz bump is the actual converter fundamental, looks a bit different than what the old CS4328 was spitting out and that I know is running at 2.8 MHz (44.1k x64). There's also a peak at half of that 1.4 MHz. 4.2 MHz is 1.5 times 2.8.

 

[Miska]

18 minutes ago, esldude said:

If upsampling fixes the issues then good, but it would be nicer not to need that fix. 

 

As long as you deal with PCM, at rates like 44.1k, you will need digital filters as part of the reconstruction. And you need to run those somewhere. DAC chips are very constrained in terms of DSP power, so they cut a lot of corners and leave it short. And that shows up.

 

[Miska]

1 hour ago, esldude said:

This is unusual behavior.  It only does this at max level and between 21,750 hz and 22,050 hz with the Forte I have.  An ADC shouldn't let such signals thru, but the Forte shouldn't act that way.  Tried another DAC and it does none of this.  However, if you drop the signal to -1 dbFS that goes away. 

 

Modern ADC's will let things through up to 22.05 kHz and a little bit over as aliasing. It is property of the on-chip digital filter, if you look at the level response you can see that there's a frequency reponse boost around that area. If you upsample 44.1k content to 192k that goes away. Although related to this the problem is not so much what the ADC originally has let through, but mastering in digital domain pushing levels to clipping and then that triggering such behaviors in clipped transients.

 

1 hour ago, esldude said:

I suppose you've noticed the 31 khz tone around the PSU, also causes faint aliasing in the ADC not around Nyquist, but around the 31 khz tone.  Making me wonder if that is from the PSU.

 

Yes, I mentioned that. It is from the internal switching PSU (likely some DC-DC converter), since it is independent whether the device is wall-wart powered or not. And the frequency matches such. Quite typical, I'm less worried about that than the mess it leaves at audio band.

 

1 hour ago, esldude said:

Here is a 48 khz and then 96 khz sweep recorded at 192 khz. 

 

 

Yeah, it is somewhat messy. And running it at 192k doesn't fix that either.

 

But afterall it is quite cheap device. I originally purchased it for portable room acoustic measurements because it has 48V phantom output. But they've already cut software support for that product and it works only to limited extent as UAC2 class compliant device... I replaced Forte with Prism Lyra, but I'm not entirely happy with that either...

 

[Miska]

It can disturb the modulation scheme in analog class-D. But nowadays it is better to use direct-digital amplifiers where the modulator is software generating 1-bit PWM (aka DSD) stream driving directly the power switching stages. Essentially the amplifier becomes "power DAC" and you don't anymore have analog stage between DAC and power amp. (for example we did that with Estelon in the Estelon Lynx speaker)

 

It was explained in more detail here:

https://www.youtube.com/watch?v=6gdbdgKWJ2s

 

I think most amplifiers (speaker and headphones) used in mobile phones and tablets these days are similar, for efficiency reasons. Also microphones used there are digital MEMS type that output 1-bit stream.