Why you can't trust measurements

[PeterG]

18 hours ago, firedog said:

The recent extremely positive reviews of the Topping DAC (Absolute Sound, subjective only) and and Pre (Stereophile, subjective plus measurements) in which there was agreement that the two under $1000 pieces were as good SQ wise (and for the pre, measurement wise) as pretty much anything on the market in their segment is telling: Even the mainstream publications are feeling the pressure from the market to review such relatively inexpensive devices that measure as state of the art. Once upon a time, neither of those publications would review such products, as they don't have dealer networks and are only available online.

 

I found the Topping DAC review deeply disappointing, even disturbing, from a publication I have often relied upon.  They did indeed assert the SQ was "reference level".  But then the only comparison offered was to a $600 DAC, and then concluded that you might go either way.  So these are two world-beaters at less than $1000?  It also bothered me that they repeated the no euphonics chant numerous times, almost as if to hedge their assertion--I am not aware of any euphoric reference level DACs.  This was very different than a few years ago when a number of publications asserted the Yggy was better than anything up to $X in price or something like that--it was much less credible.

 

Similarly, the Absolute Sound rave review of the B&W 805 D4 seemed to evade the near universal question around the 800 series--too much treble?

 

I'm not big on measurements in general, but they sure would have helped ground both of these discussions

[firedog]

1 hour ago, PeterG said:

 

I found the Topping DAC review deeply disappointing, even disturbing, from a publication I have often relied upon.  They did indeed assert the SQ was "reference level".  But then the only comparison offered was to a $600 DAC, and then concluded that you might go either way.  So these are two world-beaters at less than $1000?  It also bothered me that they repeated the no euphonics chant numerous times, almost as if to hedge their assertion--I am not aware of any euphoric reference level DACs.  This was very different than a few years ago when a number of publications asserted the Yggy was better than anything up to $X in price or something like that--it was much less credible.

 

Similarly, the Absolute Sound rave review of the B&W 805 D4 seemed to evade the near universal question around the 800 series--too much treble?

 

I'm not big on measurements in general, but they sure would have helped ground both of these discussions

 I think that yes, Steven Stone was basically saying the sound was reference level. The impressive thing to me was that he did blind testing of the two DACs, and he also used his own master recordings of acoustic music, which he obviously knows very well. He said they sounded exactly as they should. I think that's a pretty impressive recommendation. I haven't heard the Topping DAC, but if I was in the market to spend that kind of money and couldn't audition, that to me would be a serious recommendation.

 

The pre and DAC have measurements elsewhere (ASR, Stereophile, and a couple of other places).

[Allan F]

11 hours ago, Archimago said:

Again, what does this mean "sonic differences that cannot be measured"? Since we're talking about a DAC, other than reproducing the intended digital data as analogue electrical waveforms, what does he think is still out there to be "discovered"?

 

11 hours ago, Archimago said:

Jitter has been discussed since the first decade of the release of CDs! Even if it was "ignored", that was just for a few years while there were other more pressing issues to deal with (like achieving a full 16-bit resolution)... "So, Ed, can you give me an example of what newly discovered measurement was found in the last decade?"

 

Re "sonic differences that cannot be measured", I believe that he means that listening to its sound may result in specific changes to a circuit's design that are not suggested directly by the DAC's measurements themselves.

 

Discover has more than one meaning. It can mean gain knowledge of, as in learning the significance of something that was previously ignored or unappreciated, e.g. jitter.

[Superdad]

12 hours ago, Archimago said:

I always find it interesting that audiophile reviewers will drag up heroes like Ed Meitner and ask these questions, get what might or might not be profound answers, but never seem to go the next step and ask. "What do you think is the threshold for good-enough 'hygiene'?"

Wow, talk about missing his point and redirecting!

But perhaps you are a person who weighs your food and measures its salt content to decide how it will taste. Of course the chefs don’t do that…

[pkane2001]

19 minutes ago, Superdad said:

Wow, talk about missing his point and redirecting!

But perhaps you are a person who weighs your food and measures its salt content to decide how it will taste. Of course the chefs don’t do that…

My suggestion would be to ask more questions, not less. Certainly some manufacturers would rather not have their customers ask too many questions. Saying things like believe me we know what we’re doing, we’ll measure it soon, we have many happy customers, so it must be good, etc.

 

Of course, chefs create art, but audio systems don’t. They reproduce art. Asking a question about how good is your reproduction equipment is not just a reasonable question, it is a must-ask for any educated consumer.

[PeterG]

16 hours ago, firedog said:

 I think that yes, Steven Stone was basically saying the sound was reference level. The impressive thing to me was that he did blind testing of the two DACs, and he also used his own master recordings of acoustic music, which he obviously knows very well. He said they sounded exactly as they should. I think that's a pretty impressive recommendation. I haven't heard the Topping DAC, but if I was in the market to spend that kind of money and couldn't audition, that to me would be a serious recommendation.

 

The pre and DAC have measurements elsewhere (ASR, Stereophile, and a couple of other places).

 

I disagree--Stone provided virtually no substantiation for his assertion that the Topping is reference level.  As you note, there were no measurements. (I'm OK with that)  Worse, there were no meaningful comparisons--a $600 Gustard?  Please, be serious.  Who's even heard of Gustard?  Also--"sounded exactly as they should"--what does that even mean?  Compared to what?

 

Compare this with Chris's  much more meaningful substantiation of his assertion that the Yggy is top shelf, from the AS (CA) 2015 review--" It's one of my favorite DACs available today. In fact, I will happily mention the Yggdrasil in the same sentence as some of my other favorites, the Berkeley Audio Design Alpha DAC RS ($16,000) and the EMM Labs DAC2X ($15,500)"

 

When I read that the Topping compares favorably to the Yggy, Berkeley, EMM, Lampizator, Bartok..., then I'll be impressed

[fas42]

21 hours ago, Archimago said:

Again, what does this mean "sonic differences that cannot be measured"? Since we're talking about a DAC, other than reproducing the intended digital data as analogue electrical waveforms, what does he think is still out there to be "discovered"?

 

 

What needs to be "discovered" is that digital replay requires exceptional "hygiene", compared to the typical analogue based setup, for the subjective SQ to switch into, 'magic' mode - every system I've played with over the decades has needed love and attention to detail to get this; the need for this doesn't go away just by throwing enough expensive, street cred gear at the situation. The industry still hasn't properly woken up to the requirement that the "minor things" like noise isolation, and reduction of spurious causes of interference have to be taken really seriously - until this happens it's always going to be a lottery as to whether a particular rig just goes through the motions of playing some recording, as compared with conjuring up an enthralling, immersive listening experience ...

[opus101]

1 hour ago, PeterG said:

 

When I read that the Topping compares favorably to the Yggy, Berkeley, EMM, Lampizator, Bartok..., then I'll be impressed

 

I severely doubt you'll ever read such a thing in an advertiser-funded magazine. Which is of course the 800lb gorilla in the room.

[PeterG]

59 minutes ago, opus101 said:

 

I severely doubt you'll ever read such a thing in an advertiser-funded magazine. Which is of course the 800lb gorilla in the room.

 

It's your lucky day! 

 

Here's Absolute Sound with a similar Yggy review to CA--"I can’t tell you how Moffatt did it, but I can describe how the Yggy sounds, and why its one of the three best DACs I’ve heard regardless of price. (The other two are the $19,500 Berkeley Alpha Reference and the $35,000 dCS Vivaldi. I suspect that the MSB Select is outstanding, after hearing it many times at shows, but I haven’t evaluated it in my own system.)"

[opus101]

'One of the best 3' is suitably vague - I'd interpret that to mean 'its number 3, after those two'. But doesn't say how close it gets to them. If the reviewer mentioned which other DACs he'd heard above Yggy's price point that would set some valuable context.

[firedog]

7 hours ago, PeterG said:

Stone provided virtually no substantiation for his assertion that the Topping is reference level.  As you note, there were no measurements. (I'm OK with that)  Worse, there were no meaningful comparisons--a $600 Gustard?  Please, be serious.  Who's even heard of Gustard?  Also--"sounded exactly as they should"--what does that even mean?  Compared to what?

You are arguing with yourself. Of course he didn't - it's a subjective review, just like 99% of audio reviews. Gustard is a pretty well known brand, with some highly regarded PRO and consumer devices. That YOU haven't heard of them doesn't mean much. It's common to compare similar items in a hi-fi review. That's what Stone did. 

 

Meaningful comparisons? What do think is meaningful? He compared it to another under $1000 DAC under blind conditions - rarely done in the audiophile reviewing world.

 

He played back his own master recordings. "Sounded exactly as they should" - the meaning is totally clear, except if you are making a special effort not to understand the sentence. If you make your own recording, you know exactly how it "should" sound. And he said the DAC did exactly that level of reproduction. Maybe he should have phrased it differently so it would be phrased the way you'd like it to be.

 

In the end, it's a subjective review. If you think it's worthless, don't read it. IMO it was more substantial and more seriously done than virtually all other subjective reviews of audio equipment we commonly see. 

[PeterG]

7 hours ago, firedog said:

You are arguing with yourself. Of course he didn't - it's a subjective review, just like 99% of audio reviews. Gustard is a pretty well known brand, with some highly regarded PRO and consumer devices. That YOU haven't heard of them doesn't mean much. It's common to compare similar items in a hi-fi review. That's what Stone did. 

 

Meaningful comparisons? What do think is meaningful? He compared it to another under $1000 DAC under blind conditions - rarely done in the audiophile reviewing world.

 

He played back his own master recordings. "Sounded exactly as they should" - the meaning is totally clear, except if you are making a special effort not to understand the sentence. If you make your own recording, you know exactly how it "should" sound. And he said the DAC did exactly that level of reproduction. Maybe he should have phrased it differently so it would be phrased the way you'd like it to be.

 

In the end, it's a subjective review. If you think it's worthless, don't read it. IMO it was more substantial and more seriously done than virtually all other subjective reviews of audio equipment we commonly see. 

 

I am not arguing with myself--I am expressing disappointment that a reviewer from a well respected publication asserted that a DAC was "reference" level and then did not substantiate his claim with significant objective or subjective support.   A good subjective review that included such glowing SQ assessment would have compared the Topping to one or two reference level DACs (ie DACs widely reviewed as audiophile leaders), as both Absolute Sound and The Computer Audiophile did when they claimed the Yggy was stellar.

 

 

 

 

 

 

[Windows X]

It seems the points keep expanding and escalating as any other typical two sides with different mindset talking to each other without trying to understand their main points.

 

If you classify facts and opinions and make simple rules about it, it's not that hard to find Q.E.D. that can make us reach to consensus.

 

Facts:

-Audio product has specifications and measurements data

-Measurements can come from different methodologies and controlled subjects

-Construction of product with better specifications and design also cost more (dual DAC, fully balanced, better specs parts, etc.)

-Different products from different companies have different pricing structure and different cost on production/marketing/support/etc.

 

So, qualifying product's quality objectively based on price can't always get fair comparison for price/performance. You don't buy product from price/performance alone. It's also about reputation, beauty, second hand value, support, personal aesthetics, etc.

 

I don't really see the point in what you guys are currently arguing now. If people like something expensive, let them buy it. If you feel disappointed about some product purchased, write a honest review about it.

 

If you want to stay true to your objectivist's goals, don't mix it with your personal interests on subjective aspects. Like I'd rather not name products I bought and regret because I still want to sell it at good price. Or I actually buy it because I like its sound but I also need objective measurements to convince myself that my ears aren't playing trick. Or I can't buy $30k to try and I don't feel it's much better than $1k DAC I own. Those are all your subjective opinions and experience. And these are examples I made up to elaborate the point and I don't mean to anyone here is doing what I've just said.

 

This thread is about how measurements can convince you to buy product that you believe it's better from measurements but you may not actually have enough knowledge to understand that measurements of different products may come from different controlled subjects or from selective samples. Also, measurements don't have a single solid number to qualify product. Measurements are measured data. It can indicate something like lower is better or higher is better "IN ONE ASPECT" only.

 

To make good sound requires all aspects combined to reach the finest tuning point of balance. Sometimes you lose quality of some aspects to maintain something at good quality. Balancing every aspect and settle down with designer's preference isn't something you can pick from measurements and judge because you have little bit of knowledge about electronics. You also need to understand designer's philosophy and how it reaches to that conclusion also. That's why GoldenOne pointed how out changing design philosophy to improve measurements data and get more sales is a bad thing and need to be addressed.

 

Can we just stick to the main point about this thread. GoldenOne tries to point out how companies are using measurements as selling point on budget products even to the point of changing design philosophy or even sacrifice what makes good vibes for better specs or rig measurement's methodology with selected samples to boost their sales.

 

It has nothing to do with objective VS subjective debate. Please do not miss the point and keep exchange civil. I'd like to read and understand more about this matter and hope we can find some solutions to overcome this crisis. It's really big crisis when people start audio hobby from budget segment and they believe they can bet everything on advertised measurements data without even giving a care how people's ears will hear. They chose to rely on measurements because they can no longer trust reviewers or testimonials and now even measurements can no longer be trusted as proven quite clearly in this video.

 

Regards,

Keetakawee

[Samuel T Cogley]

17 hours ago, Windows X said:

It's really big crisis when people start audio hobby from budget segment and they believe they can bet everything on advertised measurements data without even giving a care how people's ears will hear.

 

To me, this reads like, "it would be tragic if newcomers to audiophilia were able to achieve the same level of gratification as those spending orders of magnitude more money".

 

 

[Josh Mound]

On 5/5/2022 at 6:38 PM, JoshM said:

I agree. I think it would be perfectly fine and ethical to say, “I’m not mentioning warranties, or reliability, or conducting teardowns. Just the same suite of measurements. Caveat emptor.” But I don’t think it’s ethical to conduct teardowns and speculate about reliability for some reviews, but not others. 

Apropos of the previous discussion about reliability, I happened upon this exchange while perusing a review over at ASR. This poster’s experience is very similar to (though worse than) mine.

 

 

Here’s Amir’s response:

 

[Josh Mound]

On 5/20/2022 at 6:54 AM, firedog said:

I have no data to back this up, but: I think Amir's measurements of devices under $1000 and under $500 is pretty influential. My impression is that there's a serious segment of people who only want to spend a few hundred on an amp or DAC and since it's often impossible to audition units anyway, they decide to buy units that measure exceptionally well, as their first criterion.

 

It also seems like some manufacturers are responding to this, by making units that have great measurements and by publishing those measurements. Schiit and some of the better known Chinese brands come to mind. This wasn't a "thing" a few years ago.

 

I also bought a Schiit Modi 3E for this reason. Measured well(not by Amir) , had the features I wanted, and was cheap. I'm very happy with it. I wasn't going to go to the trouble of auditioning multiple items for a $125 outlay. I'm sure many others have a similar POV. It btw, didn't embarass itself when I connected it to my expensive system.

 

The recent extremely positive reviews of the Topping DAC (Absolute Sound, subjective only) and and Pre (Stereophile, subjective plus measurements) in which there was agreement that the two under $1000 pieces were as good SQ wise (and for the pre, measurement wise) as pretty much anything on the market in their segment is telling: Even the mainstream publications are feeling the pressure from the market to review such relatively inexpensive devices that measure as state of the art. Once upon a time, neither of those publications would review such products, as they don't have dealer networks and are only available online.

I think this is all correct. What I find puzzling (even concerning) is that many of these folks seem to be constantly “upgrading” their DACs to more expensive models, despite already owning “perfect” measuring DACs. In that sense, I don’t quite get the brand of objectivism at play. Indeed, it looks a lot like the old subjectivist FOMO/FOTM pattern, and certain ASR-favored brands seem to be fueling this by constantly introducing new models.  

 

The early Topping DACs reviewed by ASR were generally $100 to $200 and measured “perfectly,” but perhaps lacked reliability and customer service. Now, however, many of those models are out of production, and there are new Topping DACs in the $500 to $900 range. 
 

But what’s the point, from a measurements-first perspective? Something like the Modi 3 is $125, has “perfect” measurements, and has good reliability and customer support. I can’t see someone who’s a hard-core measurements-only objectivist doing anything other than buying a Modi 3 and being done with DACs forever. 

[Josh Mound]

Also, relevant to @GoldenOne’s video, it thought Bob’s illustration of how amplifier distortion interacts with level was fascinating:

 

https://www.superbestaudiofriends.org/index.php?threads/tutorial-amplifier-distortion-vs-amplitude-a-visual-guide.12279/

 

It’s worth pondering what level is most realistic for one’s use of a product, and it might not be the level at which most measurements are performed. 

[Windows X]

13 hours ago, Samuel T Cogley said:

 

To me, this reads like, "it would be tragic if newcomers to audiophilia were able to achieve the same level of gratification as those spending orders of magnitude more money".

 

 

 

Again with fallacy. Didn't I make myself clear enough that there was attempts to change design philosophy to get better measurements on "one aspect" and explain how good sound was from improvements in "all aspects".

 

And not to mention how newcomers having zero experience will have expectation bias on products with better measurements from one thing leading to better sound quality than another product that lose one measurements but win others that were not measured.

 

If you want to make a valid counter argument, we should talk about how we can make sure we have proper measurements with same standards and how we can avoid using selected samples as references. Quoting one thing you think you can make come back and ignore the main points I raised was not what objectivist should do.

 

Regards,

Keetakawee

[firedog]

3 hours ago, JoshM said:

I think this is all correct. What I find puzzling (even concerning) is that many of these folks seem to be constantly “upgrading” their DACs to more expensive models, despite already owning “perfect” measuring DACs. In that sense, I don’t quite get the brand of objectivism at play. Indeed, it looks a lot like the old subjectivist FOMO/FOTM pattern, and certain ASR-favored brands seem to be fueling this by constantly introducing new models.  

 

The early Topping DACs reviewed by ASR were generally $100 to $200 and measured “perfectly,” but perhaps lacked reliability and customer service. Now, however, many of those models are out of production, and there are new Topping DACs in the $500 to $900 range. 
 

But what’s the point, from a measurements-first perspective? Something like the Modi 3 is $125, has “perfect” measurements, and has good reliability and customer support. I can’t see someone who’s a hard-core measurements-only objectivist doing anything other than buying a Modi 3 and being done with DACs forever. 

At ASR they have difficulty when you ask them why the measurements matter when they are beyond what is considered "audibility". Does my Chinese DAC with Sinad of 113 or 114 sound as good as their "state of the art" DACs with SINAD of 121? And does it sound better than a DAC with SINAD of 109?

 

Some of them will say all DACs with measurements past a certain threshold sound the same. But others will buy the best measuring DAC they can afford, even if it is well above that threshold. It's either: a version of FOMO; a kind of obsession with the numbers; or (and some of them acknowledge this) buying something that they consider to have the best engineering, as shown by the best measurements. 

On 5/21/2022 at 5:49 PM, Allan F said:

Re "sonic differences that cannot be measured", I believe that he means that listening to its sound may result in specific changes to a circuit's design that are not suggested directly by the DAC's measurements themselves.

 

I admit the possibility that this exists. What would be interesting to hear from manufacturers that measure, then listen, and then "adjust by ear" - what they think is happening when they reconfigure a device, the measurements stay the same, yet they think it sounds different?  And I'd be interested to know if they blind test their two versions of the component, or is it possibly self induced confirmation bias?

 

Bruno Putzeys has said (at least as far as amps) that very extensive measurements ( I believe he referred to about 20 different ones he conducts, some of which put the amp in extreme situations) of the right type WILL predict amp performance and sound. Sort of not just what you measure, but how you measure it. He said when he gets those right, he knows the amp will sound good and the way he intended. He also said he can color any amp to sound as he wishes. As an  example, he said he could give the MacIntosh sound to Class D - if he wanted to.

 

My takeaway from this is that perhaps Amir and some others aren't doing all the right measurements, not that the right ones don't exist yet. If I was Bruno, and I'd figured this out, I certainly wouldn't make it public knowledge, it has great proprietary value. 

[Windows X]

4 hours ago, JoshM said:

Apropos of the previous discussion about reliability, I happened upon this exchange while perusing a review over at ASR. This poster’s experience is very similar to (though worse than) mine.

 

 

Here’s Amir’s response:

 

 

He really posted that? Wasn't the point in his website is about recommending budget products with good measurements to his readers? If products have a lot of reports about defects, he should put his priority on readers' interests first. This made me wonder if it's either his pride / ego of being right or he has some companies invested on his website and need to keep that up.

[firedog]

1 minute ago, Windows X said:

 

He really posted that? Wasn't the point in his website is about recommending budget products with good measurements to his readers? If products have a lot of reports about defects, he should put his priority on readers' interests first. This made me wonder if it's either his pride / ego of being right or he has some companies invested on his website and need to keep that up.

Can't say it really bothers me. Don't disagree with Amir. If I order a Chinese DAC from AliExpress, I'm taking into account that I am unlikely to get warranty service for it in a reasonable or cost effective manner. I'd assume that people buying these devices are taking that into account. But if I buy a Topping DAC that's supposed to be "state of the art" and "reference quality" for $700, then I'm figuring that if  it's defective, I can get another and I still won't have spent as much as I would have on a much more expensive 
DAC from a "conventional" supplier. Same for if I don't agree after purchase that it is truly state of the art. I took a relatively small monetary risk that's supposed to give me a non-proportional sonic payoff. If I don't really get that, I haven't lost a significant amount relative to buying something much higher priced with warranty and service. 

 

If you aren't willing to take those risks, you shouldn't be buying such products.

 

And when Stereophile and TAS recommend those same products, I don't think they are standing behind them any more than Amir is. 

[GoldenOne]

On 5/24/2022 at 4:50 AM, JoshM said:

Also, relevant to @GoldenOne’s video, it thought Bob’s illustration of how amplifier distortion interacts with level was fascinating:

 

https://www.superbestaudiofriends.org/index.php?threads/tutorial-amplifier-distortion-vs-amplitude-a-visual-guide.12279/

 

It’s worth pondering what level is most realistic for one’s use of a product, and it might not be the level at which most measurements are performed. 

This is something that I personally feel is very important. For various reasons.

Firstly, I've noticed that some of the devices with stronger 'House Sounds' including DACs such as the Schiit Yggdrasil, show quite drastic changes in distortion level and profile in relation to amplitude.

 

In addition to this, some devices have slightly different distortion at their very uppermost output levels than they do in the rest of the output range.
The Gustard X26 pro for example, which is an excellent DAC, does seem to be somewhat 'optimising' for full scale output to look better on ASR-style tests.

 

You can see that distortion drops a few dB in the top ~6dB.
And in fact even when not limited by THD itself, you can visually see how the harmonics change.

At full scale 0dBfs output it looks like this:

All harmonics at or below about -140dB

But then take it to -3dBfs output and suddenly the 3rd order harmonic jumps up almost 15dB.

 

But ignoring DACs and just looking at amps, as has been mentioned, we don't listen to amps at 4v output. In fact on the vast majority of headphones that level would probably destroy your ears. So it's much more apt to test the device in the range that you're likely to use it.
Personally I offer measurements at 4v (to compare to ASR if people wish), 700mV (as this is a reasonably loud listening level on a few over ear headphones like an Arya) and 50mV (as this is appropriate for IEMs).

If you test something like an HE9 at 4V on 32 Ohm, you'll get about 0.3% THD+N (50dB SINAD). 
But if you test at a more realistic 700mV, you get a reduction to 0.02% THD+N (23dB increase to 73dB SINAD)

It seems silly to me to only test things at 4v as it's not realistic for amps in most cases.
But more to the point, on both DACs and Amps THD+N vs Level behaviour can reveal some pretty interesting stuff

[pkane2001]

On 5/23/2022 at 11:37 PM, JoshM said:

I think this is all correct. What I find puzzling (even concerning) is that many of these folks seem to be constantly “upgrading” their DACs to more expensive models, despite already owning “perfect” measuring DACs. In that sense, I don’t quite get the brand of objectivism at play. Indeed, it looks a lot like the old subjectivist FOMO/FOTM pattern, and certain ASR-favored brands seem to be fueling this by constantly introducing new models.  

 

The early Topping DACs reviewed by ASR were generally $100 to $200 and measured “perfectly,” but perhaps lacked reliability and customer service. Now, however, many of those models are out of production, and there are new Topping DACs in the $500 to $900 range. 
 

But what’s the point, from a measurements-first perspective? Something like the Modi 3 is $125, has “perfect” measurements, and has good reliability and customer support. I can’t see someone who’s a hard-core measurements-only objectivist doing anything other than buying a Modi 3 and being done with DACs forever. 

 

I don't have any Topping DACs, but I do have their PA5 amp. I rarely upgrade, precisely because I've achieved the sound quality I love and enjoy. When I do upgrade, I buy DACs with good measurements, but more importantly, with the features I want. Mostly professional interfaces, no audiophile DACs in 5+ years. Last audiophile DAC was Holo Spring, and that one was to satisfy my curiosity about modern R2R DACs -- something I liked and thought of very highly 25 years ago (I still have a few of the old ones for comparison). And of course, Holo Spring is also a well-measuring device.

 

PA5 is an example of a risk of buying equipment from a small Chinese manufacturer. Mine is working great, but I bought it from the first batch, which appears to be doing well for me and for others. Newer batches of PA5 seem to exhibit a high failure rate, which Topping has not addressed so far. The amp is excellent, but the high failures and no response from manufacturer is disturbing. I'd be cautious about purchasing any new and untested products from them in the future, especially those shipped directly from China. These become more expensive to replace or fix, even under warranty.

[pkane2001]

On 5/23/2022 at 11:50 PM, JoshM said:

Also, relevant to @GoldenOne’s video, it thought Bob’s illustration of how amplifier distortion interacts with level was fascinating:

 

https://www.superbestaudiofriends.org/index.php?threads/tutorial-amplifier-distortion-vs-amplitude-a-visual-guide.12279/

 

It’s worth pondering what level is most realistic for one’s use of a product, and it might not be the level at which most measurements are performed. 

 

That's why a sweep of THD+N over level is useful. Even for DACs, but even more importantly, for amps. A 4v or 2v DAC output is useful for standard comparisons, and when the system includes a preamp. If the DAC feeds directly into the amp, these levels are unlikely to match actual levels in use.

 

[pkane2001]

4 hours ago, GoldenOne said:

This is something that I personally feel is very important. For various reasons.

Firstly, I've noticed that some of the devices with stronger 'House Sounds' including DACs such as the Schiit Yggdrasil, show quite drastic changes in distortion level and profile in relation to amplitude.

 

In addition to this, some devices have slightly different distortion at their very uppermost output levels than they do in the rest of the output range.
The Gustard X26 pro for example, which is an excellent DAC, does seem to be somewhat 'optimising' for full scale output to look better on ASR-style tests.

 

You can see that distortion drops a few dB in the top ~6dB.
And in fact even when not limited by THD itself, you can visually see how the harmonics change.

At full scale 0dBfs output it looks like this:

All harmonics at or below about -140dB

But then take it to -3dBfs output and suddenly the 3rd order harmonic jumps up almost 15dB.

 

But ignoring DACs and just looking at amps, as has been mentioned, we don't listen to amps at 4v output. In fact on the vast majority of headphones that level would probably destroy your ears. So it's much more apt to test the device in the range that you're likely to use it.
Personally I offer measurements at 4v (to compare to ASR if people wish), 700mV (as this is a reasonably loud listening level on a few over ear headphones like an Arya) and 50mV (as this is appropriate for IEMs).

If you test something like an HE9 at 4V on 32 Ohm, you'll get about 0.3% THD+N (50dB SINAD). 
But if you test at a more realistic 700mV, you get a reduction to 0.02% THD+N (23dB increase to 73dB SINAD)

It seems silly to me to only test things at 4v as it's not realistic for amps in most cases.
But more to the point, on both DACs and Amps THD+N vs Level behaviour can reveal some pretty interesting stuff

 

Of course, the third harmonic jumping up 15dB still remains well below any levels of audibility and better than probably 99% of DACs out there, so not a major concern. Interesting nevertheless.

 

4v or 2v output measurement is completely realistic if one is planning to use a preamp or an integrated amp. You're right that it's not a level anyone would normally use when feeding the amp directly. That's one reason why a sweep of THD+N vs. level is important.