Tom's Hardware Blind DAC Test

[alexwgoody]

Audiophile PC Sound - The Real Cost of Hi-Fi - Tom

 

Thoughts?

[Savage]

Headphones make for difficult testing. He could have minimized this a bit by using a separate amp matched to the HD800s, so the testing became only dac vs dac and the amplifier portion could be left for another article. Soundstage/separation/presentation are also going to be more difficult to discern with headphones and this seems to be where there is a good differences in dacs.

 

Interesting read for his POV if nothing else.

[alexwgoody]

Headphones make for difficult testing. He could have minimized this a bit by using a separate amp matched to the HD800s, so the testing became only dac vs dac and the amplifier portion could be left for another article. Soundstage/separation/presentation are also going to be more difficult to discern with headphones and this seems to be where there is a good differences in dacs.

 

While the testing was meant for Dac vs Dac, doesn't the fact that they used the internal amps here on these products of widely varying price give some legitimacy to the claim that headphone amps also all sound similar or the same, like Dacs?

 

Obviously this test is not perfect, though it is still a reasonable and a worthwhile methodology in my opinion; but given your comment on how you believe it is more difficult to discern the purported advantages to high end dacs on headphones, would it be fair to say that you concede or consider the notion reasonable that, at least in the case of headphones, there is minimal to no difference between high end dacs and/or headphone amps?

[mayhem13]

I think headphones would be a very clinical, accurate device to use to compare DACs. Soundstage and imaging are products of the speakers and room as a system. If a DACs filters have poor impulse response and create ringing, that would be audible in a quality headphone just as much as a free space speaker system.

[Snowmonkey]

Bit hard to make sense of those results. They picked a predominance of electronic/pop tracks. The one exception, Rachmaninov's Symphonic Dances was heavily downsampled and they seemed to be testing the results of that process, rather than comparing the DACs (the results were all "N/A").

Why didn't they just use all PCMs at the same sample rate? Why didn't they compare some "natural" sounds, like a jazz trio or a piano or guitar or violin solo?

[mayhem13]

Bit hard to make sense of those results. They picked a predominance of electronic/pop tracks. The one exception, Rachmaninov's Symphonic Dances was heavily downsampled and they seemed to be testing the results of that process, rather than comparing the DACs (the results were all "N/A").

Why didn't they just use all PCMs at the same sample rate? Why didn't they compare some "natural" sounds, like a jazz trio or a piano or guitar or violin solo?

 

solo performance work might have been more revealing of the differences you suggest?

 

when you say 'natural' sound......you mean acoustic instruments/vocal?

[Snowmonkey]

solo performance work might have been more revealing of the differences you suggest?

 

when you say 'natural' sound......you mean acoustic instruments/vocal?

Yes, I would have thought the rendering of say, cymbals or piano, would be a better test of a DAC's mettle than a wall of synthesised sound.

[gmgraves]

I think headphones would be a very clinical, accurate device to use to compare DACs. Soundstage and imaging are products of the speakers and room as a system. If a DACs filters have poor impulse response and create ringing, that would be audible in a quality headphone just as much as a free space speaker system.

 

Yes, they can be, no doubt about it. A pair of Audeze, or Stax Earspeakers, or HiFiMan HE-500, or Sennheiser HD-800 (and I'm sure there are others) make excellent analytical tools. OTOH, when testing DAC, CD players, or any other sound source, the only thing in the playback chain that should be changed from test to test is the component of interest (in this case, the digital-to-analog conversion part of a DAC. To do this properly, the tester should have come out of the analog audio outputs and fed a common headphone amplifier. That way, you can be more reasonably sure that you are hearing the difference between DACs, not DACs and headphone driver stages.

[mayhem13]

Yes, they can be, no doubt about it. A pair of Audeze, or Stax Earspeakers, or HiFiMan HE-500, or Sennheiser HD-800 (and I'm sure there are others) make excellent analytical tools. OTOH, when testing DAC, CD players, or any other sound source, the only thing in the playback chain that should be changed from test to test is the component of interest (in this case, the digital-to-analog conversion part of a DAC. To do this properly, the tester should have come out of the analog audio outputs and fed a common headphone amplifier. That way, you can be more reasonably sure that you are hearing the difference between DACs, not DACs and headphone driver stages.

 

So you're pretty confident that all sharing a common amplifier, the results would be vastly different?, moderately?, or pretty much the same indecisiveness expressed in their format?

[gmgraves]

So you're pretty confident that all sharing a common amplifier, the results would be vastly different?, moderately?, or pretty much the same indecisiveness expressed in their format?

 

Good question. No, I'm not at all confident about the result. But I do know this: using one headphone amp and one good pair of phones while listening to various DACs via their analog audio output jacks will be much more like the way the DACs will ultimately be used in a complete system and the DAC itself will be the only variable in the test. The way it was done by "Tom's Hardware", they were testing both the DAC and the headphone amplifier, thus giving the listeners two variables to ponder. I, personally, have never had much luck auditioning equipment that way.

[speavler]

I didn't get past this:

 

As long as the content arrives to the DAC in a bit-perfect state, the source really doesn't matter. A quality CD-player or a PC playing a bit-perfect stream over USB should sound the same.

[mayhem13]

I didn't get past this:

 

It's a controversial topic in regards to clock timing variations and jitter. There seems to be a singular reoccurring theme is that almost every anomoly within a decoded digital format that's audible is lumped under the term of jitter......and that's accurate at all.

 

That being said, my personal opinion is that if the data stream is bit perfect and the DAC having suitable jitter correction implementation.....IF the rest of the signal chain is identical, the DACs will be indistinguishable in a blind listening test as conducted by Tom's Hardware.

[speavler]

It's a controversial topic in regards to clock timing variations and jitter. There seems to be a singular reoccurring theme is that almost every anomoly within a decoded digital format that's audible is lumped under the term of jitter......and that's accurate at all.

 

That being said, my personal opinion is that if the data stream is bit perfect and the DAC having suitable jitter correction implementation.....IF the rest of the signal chain is identical, the DACs will be indistinguishable in a blind listening test as conducted by Tom's Hardware.

 

I can imagine this is a topic that's been discussed quite a bit here I'm not an expert, but through my own experimentation, I went from thinking of computer playback as just a convenient way to listen to my music to thinking of it as a superior way to listen to my music. Until very recently, I could barely, if at all, distinguish between playback software - specifically itunes vs jriver on a pc. Then I tried the Fidelizer optimization program, and it changed everything. And now with Audirvana on a mac mini, I've put my sacd player up for sale. I'm not sure where I fall on the spectrum between 'everything sounds the same' and 'everything matters', but I am certainly of the opinion that the source matters.

[Harpy]

I always liked Tom's Hardware and always valued their advice. Thought I heard a nice difference between an old Burr-Brown and the newer Sabre, but I never blind tested either. That was on speakers though too.

[Boris75]

Even more than the DACs, I'm surprised that they hear no difference between the headphone amp of a Realtek board and a Benchmark. Also, rather than trying to guess what is what, I think it would have been more interesting if they had ranked the perceived sound quality of each DAC for each track.

[mayhem13]

Even more than the DACs, I'm surprised that they hear no difference between the headphone amp of a Realtek board and a Benchmark. Also, rather than trying to guess what is what, I think it would have been more interesting if they had ranked the perceived sound quality of each DAC for each track.

 

Ranking would have thrown the purpose of the test to the dogs Boris as the results would be indisputable subjective opinions instead of an identifiable difference. Was the test flawed in some areas.......sure.....not enough test subjects for one and as pointed out earlier, music a bit less complex and more acoustic in nature would have given the participants a better chance. But I think they made their point....

 

It doesn't have to cost uber $$$ to get great sound from a PC rig.

 

For me, being somewhat bias in favor of the results, the test supports my opinions on diminished returns and small signal variance within the digital domain. I have an ODAC ( no amp,just DAC) and have been extremely impressed with it. It's matched my Benchmark on every turn SQ wise. Price wise.....obviously it's a kick to the groin followed by a down and out stomping.

 

I would also add that in regards to the on board Realtek chip, the guys got REALLY lucky here to have a PC environment so free of destructive interference and noise. I've spent A LOT of time with cans and webinars at work and I can always here the machine playing its own tune in the background.

[Blake]

mayhem: In case you are interested, a friend over on headfi did some measurements on the ODAC (he owns and likes the ODAC), which can be found here:

 

https://files.secureserver.net/0sfKG65s75aKat

 

He is a recording professional. His commentary to me (note, the "Reflink" mentioned below is the Bel Canto Reflink usb to spdif converter):

 

"Here is a link to a zipfile containing an RMAA report in html format. The program puts all the graphs into a subdirectory in png format. Unzip all this into directory of choice and look at the html file in a browser. The summary and spectrum graphs should all appear on the page so you can scroll down and view whatever interests you.

 

Note how the V800 by itself outperforms the ODac in every way. So while I think the ODac is a very fine USB Dac, (I own one too, for good reason) it is NOT "pefection, end of story" as one head-fier stated. It hits a very high price performance ratio. There are better to be had, for serious money. What floored me is the improvement in dynamic range from 103.4 to 114.5 when the RefLink fed the V800 via AES3. As you can imagine, I measured all those configurations several times, but more than several for the RefLink+V800 combination. That is an objective measurement that probably gives explanation to why we perceive greater clarity. Also note the substantial improvement in the IMD% + Noise.

 

To make sure that RMAA isn't fooled by any Windows Audio Crap, I have the RMAA program generate the test wavefile. This wavefile is sent out to the USB via a pro audio Digital Audio Workstation using bit and sample perfect drivers, bypassing all the Windows Audio Crap (k-mixer and other consumer oriented bs). The result from the Lavry AD11 is sent back to the DAW via the same driver methods also avoiding WAC. The recording of the test file via the DAC-ADC loopback is dropped into a wavefile and the RMAA program analyzes the resultant wavefile. This way we don't have to worry about RMAA and its audio interface. It only has to provide analysis, non real time, which it does very well. We also get to assure quality of the wavefile to be analyzed, both by visual inspection and listening to the wavefile."

 

 

[pooger]

NOTE: I did not look at the article! BUT—HD800s are really difficult to drive—very amp dependent. Every review I've seen for them seems to stress this. This alone means that the review is based more on how well or not any of the built in amps could drive the phones. (And since he couldn't tell a difference, to me it also implies that none of the DACs under test had an amp that mated well with the HD800s, allowing them to perform at the level they are capable of. Either that, or he just can't hear worth beans.)

[Mark Manner]

NOTE: I did not look at the article! ............

Take a look at the article, he talks about 800 driving issues.

Mark

[mayhem13]

mayhem: In case you are interested, a friend over on headfi did some measurements on the ODAC (he owns and likes the ODAC), which can be found here:

 

https://files.secureserver.net/0sfKG65s75aKat

 

He is a recording professional. His commentary to me (note, the "Reflink" mentioned below is the Bel Canto Reflink usb to spdif converter):

 

"Here is a link to a zipfile containing an RMAA report in html format. The program puts all the graphs into a subdirectory in png format. Unzip all this into directory of choice and look at the html file in a browser. The summary and spectrum graphs should all appear on the page so you can scroll down and view whatever interests you.

 

Note how the V800 by itself outperforms the ODac in every way. So while I think the ODac is a very fine USB Dac, (I own one too, for good reason) it is NOT "pefection, end of story" as one head-fier stated. It hits a very high price performance ratio. There are better to be had, for serious money. What floored me is the improvement in dynamic range from 103.4 to 114.5 when the RefLink fed the V800 via AES3. As you can imagine, I measured all those configurations several times, but more than several for the RefLink+V800 combination. That is an objective measurement that probably gives explanation to why we perceive greater clarity. Also note the substantial improvement in the IMD% + Noise.

 

To make sure that RMAA isn't fooled by any Windows Audio Crap, I have the RMAA program generate the test wavefile. This wavefile is sent out to the USB via a pro audio Digital Audio Workstation using bit and sample perfect drivers, bypassing all the Windows Audio Crap (k-mixer and other consumer oriented bs). The result from the Lavry AD11 is sent back to the DAW via the same driver methods also avoiding WAC. The recording of the test file via the DAC-ADC loopback is dropped into a wavefile and the RMAA program analyzes the resultant wavefile. This way we don't have to worry about RMAA and its audio interface. It only has to provide analysis, non real time, which it does very well. We also get to assure quality of the wavefile to be analyzed, both by visual inspection and listening to the wavefile."

 

 

 

Thanks for the link. The Violetric is certainly a nice piece of gear.........do you know of anyone stateside that owns one?

[sandyk]

Take a look at the article, he talks about 800 driving issues.

Mark

 

Quite a few people have reported that the HD800 sound better when driven from a 120 ohm source impedance and an amplifier with a minimum of + and -20V supply rails.

[esldude]

Ranking would have thrown the purpose of the test to the dogs Boris as the results would be indisputable subjective opinions instead of an identifiable difference. Was the test flawed in some areas.......sure.....not enough test subjects for one and as pointed out earlier, music a bit less complex and more acoustic in nature would have given the participants a better chance. But I think they made their point....

 

It doesn't have to cost uber $$$ to get great sound from a PC rig.

 

For me, being somewhat bias in favor of the results, the test supports my opinions on diminished returns and small signal variance within the digital domain. I have an ODAC ( no amp,just DAC) and have been extremely impressed with it. It's matched my Benchmark on every turn SQ wise. Price wise.....obviously it's a kick to the groin followed by a down and out stomping.

 

I would also add that in regards to the on board Realtek chip, the guys got REALLY lucky here to have a PC environment so free of destructive interference and noise. I've spent A LOT of time with cans and webinars at work and I can always here the machine playing its own tune in the background.

 

The Realtek chip without noise issues is hard to figure actually. All I have seen might be pretty good, but you always can hear the HD or other processes in the low level background. I have a Realtek in my current desktop that is surprisingly good in many ways. 192 capable, and yes it really will do that. The front jacks are obviously noisy. The output jacks on the rear are pretty quiet. You can only notice it if you bother to pay attention though it is there. Though mine is now near 5 years old. Maybe they have gotten better. I could believe the Xonar stuff is worthwhile if only because they somewhat shield it. But I too was taken aback as every on board soundcard I have heard would give itself away with trilling and other little sounds from the PC itself. So would all plug in soundcards other than an RME.

 

The test leaves plenty to be desired and at the same time fits with most people that use a PC for sound in less critical ways than audiophiles. Reminds me of my Dad and his intimate knowledge of fine tuning a model T ford. Most people these days are happy if their old Model T will crank and pull itself around. I have watched him insist on 'tuning it up right' for several people. One of those now useless skills I didn't know he had for a long time. And he can take them that almost don't run, and make them right. Or take those that run, and make them run right too. Right they will run as well. It is somewhat amazing to watch him spend a half hour and use some of his misspent youthful knowledge to tune them to ....well....a T. Some folks never realized they were even capable of what he can make one do. Yet, mostly if it works more or less folks are happy. Makes me worry too for the fate of aging audiophiles as technology makes never before seen levels of cheap tech work better than ever before. Sure we might know the tricks to tune her up a bit more. Will anyone really care and how useful is it? More meaningful to us than to those whom we help.

[Mike Mcsweeney]

Although their belief, that there was little to no difference between the $2 and $2000 DAC in SQ, they did agree that DSD did sound better and more natural than PCM...which is basically where my head is at....buy an inexpensive DSD DAC and you should be happy...that's what I am doing being new to hobby and budget minded....and i made that decision before reading this article, but it solidifies my decision....

 

considering these DSD/PCM DACs

 

ifi $190

denon $500

Korg $380

 

That or buy an AVR that has DSD DAC with an asynchronous usb port that i can plug my pc DIRECTLY into the AVR.

[crisnee]

Although the tests are not without some problems they also make it pretty clear (unless some monumental pile of coincidences rained onto this testing parade, or the testers were complete bozos and deaf to boot) that the differences between well made inexpensive, dare I say cheap audio and very expensive audio, especially when it comes to electronics (not including speakers) are quite small.

 

Chris

[Guidof]

I would not generalize from a sample of two . . .