Why are objective assessments important...

[Archimago]

I figure it might be good to take a discussion about the importance and utility of "objectivism" to this forum instead of getting embedded in the "Boycott the sub-forum" thread.

 

Personally, I've found running measurements and writing about my findings over the years one of the most rewarding experiences in this hobby. It challenges how/what I hear, how I think, and has shaped my "world view" about the audiophile hobby in a way I had not for years before (I started playing with audio stuff and visiting stores in the late 90's, only writing about it in 2013).

 

Anyhow, here's a response to a comment (not directed at me and not implying anything about @lucretius' position, which I'm sure is very reasonable 😉, just using it as a point to discuss what I believe is true) and I'll talk about what Chris said as well...

 

 

I keep hearing that measurements do not necessarily predict sound quality and I have to take that at face value.  But it would help me immensely if you can provide some examples from currently produced equipment. Thanks.

 

I think it's true that measurements do "not necessarily predict sound quality". But that qualifier "necessary" is a big one. With whose ears and brain are we going to judge that "sound quality" with?

 

If we're simply talking about "enjoyment" of the sound to the point of feeling good about it, heck I can enjoy an AM radio just fine. Does that mean all the "necessary" sound quality I need can be found in a Bose Wave radio because I can feel good about the song and the sound coming out of that? Of course not! We're arguing about much better devices, right? Things with "high fidelity" that achieve a level of transparency and accuracy to the recording, aren't we?

 

Objectivism is literally about taking this concept of the "sound quality" outside and consider whether it measures up to an "ideal". That "ideal" might not be for everyone but at least it provides a level playing field from which we can judge devices using a common yardstick. Furthermore, that ideal exists outside of whether a person's hearing might be failing, or if that person's perception is idiosyncratic, if that person is not an "expert listener", or even if that person lacks insight and may be biased toward a wonderful ad they saw an hour ago or what the salesman just said a few minutes ago before they changed to the expensive cable 🤨.

 

I have of course measured stuff over the years and I can tell you in no uncertain terms that the two come together when you pay attention to what you measure and take time to listen both before and after the process:

 

Synergistic power cables like these sound no different than other power cords. They appear cheaply made and not good value.

 

A "Modded" Oppo like this is a bad deal. Some might like the sound but the measurements are terrible and they've ruined what was pristine high-resolution sound.

 

The highly praised Vitus Audio amplifier in Class A adds nothing to the sound. Despite high price, a 1:1 comparison even to my Emotiva amp, shows that it's noisier objectively and when listening to music in a quiet room.

 

Human perception has its limits and our attention to things also can be limited, missing out on what we actually CAN hear but didn't notice. For example, look at all the positive comments about the recent AudioQuest Dragonfly Cobalt. From my perspective, it totally sucks as a USB DAC at this price point. Good that Mans found similar issues with distortion that I saw. Once one is tipped off to these anomalies, one can start picking out examples and select music that can bring out the anomaly that one might have missed before. This is what "perfectionist audio" IMO is about.  If I am going to pay big(er) bucks, it certainly would be nice to be clear about what performance I'm buying.

 

The opinion of any specific listener is nice, but IMO, not as strong as what objective means might reveal.

[Archimago]

Chris' reply:

Quote

 

Hi Archi, your paragraph above made me think about people's desire for a black and white world, where decisions don't need to be made, one can't be judged by a decision, and one doesn't have to use his/her brain to decide something. I obviously know this isn't what you're getting at, but I can't help but believe some in the objective crowd are this way. 

 

Life is easy when it's 1+1=2. Nobody risks anything and there is no need for discussion. Thus, one possible reason for people to love objective measurements in audio because they are being told that the decision has been made for them and there is nothing more to think about. Again, this is just a stream of thought that just came to me and needs to be fleshed out much more. I'm not directing this at anyone and don't mean to be negative toward any one or group. I think it's human nature to desire simplicity and measurements are one way of taking the brain out of the equation. 

 

Perhaps part of what I'm getting at is the status of measurements in many peoples' eyes. To me they mean something, sometimes. I like them, but always read them with a "how does this effect me" type of lens. There are just so many variable in life when humans are involved, that it's hard for me to look at a measurement and make a decision.  

 

 

I don't think this is true or fair Chris. In fact, I believe objectivism makes the decisions harder because one has to use one's intellectual and "best judgment" resources to decide what to measure and judge for oneself whether the results apply to how "I" understand the world. Ultimately whether that result applies to my home/system and how well I can perceive the difference. This is precisely what you're getting at I think but gauging it differently. Objective attempts at "reality" often intrude into our experience and makes life uncomfortable especially if it pricks at our beliefs and psychological biases! The Industry also gets uncomfortable because then we can also speak about whether the price of something relates to quality/fidelity/accuracy - value as it might be directly related to sound quality rather than appearance, prestige, or luxury.

 

No... "Life is easy when it's 1+1=2" is often not true for the objectivist.

 

It's "harder" to read graphs and understand what they mean than divining what a pure subjectivist might be implying about whether something has "presence", or if a "veil is lifted", or the sound is "faster". Those vague descriptions IMO can be tossed aside with little repercussions whereas if we know that a DAC has strong jitter for example, I think we'd pay a little more attention and consider what this means about audibility and the abilities of the company that makes the product. Compared to the challenges of writing my articles with measurements in mind, the "subjective" world of audiophilia for the years before were comparatively unchallenging. I believe it is the objectivists that "have to use his/her brain to decide something" way more because there is often actual data to wrestle with.

 

Yes, I know this might be inflammatory to some, but hey... This is Objective-Fi 🤪.

 

[tapatrick]

1 hour ago, Archimago said:

It's "harder" to read graphs and understand what they mean than divining what a pure subjectivist might be implying about whether something has "presence", or if a "veil is lifted", or the sound is "faster".

As you seem to be a helpful person I have some basic questions as I have very little understanding of measurements or their meaning. I know what a frequency curve is and the importance of low jitter and noise levels etc. but I'd much rather read a review by someone who knows how to interpret the measurements presented.

 

Apart from the obvious (noise spikes or reduced frequency range), opinion seems to be that these are open to interpretation or only show a partial picture of what constitutes the sound coming from system. I spend time on DIY forums and I put together basic components and modify them following the advice or example of those more qualified - changing wiring, building cables, replacing capacitors and replacing clocks etc as I like to tinker then try them by listening carefully over several weeks of trial in order to hopefully enhance my system. For instance my reference source is a $50 chinese SD card player heavily modified and powered by LifePO4/Ultracaps. So my interest in measurements is in relation to things like this.

 

For myself and other non techies could you please list the types of measurements everyone is referring to and any good resources to understand the relationship between these and sound quality? Thanks in advance

[fas42]

6 hours ago, Archimago said:

Human perception has its limits and our attention to things also can be limited, missing out on what we actually CAN hear but didn't notice. For example, look at all the positive comments about the recent AudioQuest Dragonfly Cobalt. From my perspective, it totally sucks as a USB DAC at this price point. Good that Mans found similar issues with distortion that I saw. Once one is tipped off to these anomalies, one can start picking out examples and select music that can bring out the anomaly that one might have missed before. This is what "perfectionist audio" IMO is about.  If I am going to pay big(er) bucks, it certainly would be nice to be clear about what performance I'm buying.

 

The opinion of any specific listener is nice, but IMO, not as strong as what objective means might reveal.

 

And that is how one can be "objective" in areas where the people usually will use the term, subjective ...what one does is use tracks of music whose content very strongly provokes the system playback to distort - the anomalies are obviously audible; so in that area it's a fail, for the setup.

 

The process of "sorting out" is eliminating each of the failure 'modes', one by one.

[Bill Brown]

As a direct answer to the topic line I would say that we need them to continue to attempt to seek out/try to develop a body of measurements that correlate with perception (positive, negative, and euphonic), identify equipment with significant engineering flaws, and to help us identify components from the huge number that we think are worthwhile to audition.

 

Bill

[Archimago]

On 2/21/2020 at 12:05 PM, tapatrick said:

As you seem to be a helpful person I have some basic questions as I have very little understanding of measurements or their meaning. I know what a frequency curve is and the importance of low jitter and noise levels etc. but I'd much rather read a review by someone who knows how to interpret the measurements presented.

 

Apart from the obvious (noise spikes or reduced frequency range), opinion seems to be that these are open to interpretation or only show a partial picture of what constitutes the sound coming from system. I spend time on DIY forums and I put together basic components and modify them following the advice or example of those more qualified - changing wiring, building cables, replacing capacitors and replacing clocks etc as I like to tinker then try them by listening carefully over several weeks of trial in order to hopefully enhance my system. For instance my reference source is a $50 chinese SD card player heavily modified and powered by LifePO4/Ultracaps. So my interest in measurements is in relation to things like this.

 

For myself and other non techies could you please list the types of measurements everyone is referring to and any good resources to understand the relationship between these and sound quality? Thanks in advance

 

Hi @tapatrick,

Yeah, very good question but a broad one also...

 

I think it starts at first principles which is that we need to appreciate the limits of human hearing; from there we can then talk about specifics like what makes a frequency response "sound different", what noise levels we need be concerned about, then the time domain parameters like phase shifts and related words like "group delay", etc...

 

I suggest making sure you read this first:

http://archimago.blogspot.com/2015/10/musings-meditations-on-limitations-of.html

 

In it, I speak about some of these basics. Then perhaps we can touch on each of these measurement parameters and see if together we can discuss the implications and where we can find evidence of audibility and significance then look at the hi-fi gear we own and see if these characteristics may be inadequate. Some of this I've discussed in my blog and will point there as appropriate...

 

[Archimago]

22 hours ago, fas42 said:

 

And that is how one can be "objective" in areas where the people usually will use the term, subjective ...what one does is use tracks of music whose content very strongly provokes the system playback to distort - the anomalies are obviously audible; so in that area it's a fail, for the setup.

 

The process of "sorting out" is eliminating each of the failure 'modes', one by one.

 

Sure, it's good to one-by-one look at issues and sort them out in our systems. Over time, this should lead to optimization as the system "evolves" to resolve audible issues...

 

[fas42]

8 minutes ago, Archimago said:

 

I think it starts at first principles which is that we need to appreciate the limits of human hearing; from there we can then talk about specifics like what makes a frequency response "sound different", what noise levels we need be concerned about, then the time domain parameters like phase shifts and related words like "group delay", etc...

 

I suggest making sure you read this first:

http://archimago.blogspot.com/2015/10/musings-meditations-on-limitations-of.html

 

 

Howdy, just curious whether you're familiar with Bregman's "Auditory Scene Analysis", and subsequent research?

[Archimago]

4 hours ago, Bill Brown said:

As a direct answer to the topic line I would say that we need them to continue to attempt to seek out/try to develop a body of measurements that correlate with perception (positive, negative, and euphonic), identify equipment with significant engineering flaws, and to help us identify components from the huge number that we think are worthwhile to audition.

 

Bill

 

Hi Bill,

I certainly agree that many things we measure today are either minor or do not lead to audible differences. A great example I think is JITTER. Yes, we can measure this with all those sidebands down below -100dBFS in modern asynchronous DACs, but seriously, who can hear them unless VERY severe - basically "broken", "engineering flaw" gear? (See my DEMO post awhile back.)

 

Despite this, notice how "jitter" continues to be claimed by various companies as being important... Go talk to Paul McGowan and Ted Smith and their PS Audio DAC. Isn't it more than a little suspicious that they never provide measurements/evidence to show the audible issue despite their verbal claims!?

 

Of course, this doesn't mean that jitter should not be measured. We can and I think should still look at stuff like the J-Test because it gives us an idea if the company paid attention even if not severe enough to be called a "flaw".

 

Your comment "develop a body of measurements that correlate with perception (positive, negative, and euphonic)" is insightful. Nice. I believe that it is only with objective testing and controlled listening (measurements of the subjective experience while minimizing non-sound-related biases) that we can differentiate "positive, negative, and euphonic". I think in fact we are getting to the point where of the three, "euphonic" is becoming most important.

 

Having said this, we need to remember the slippery slope of "euphonic" and what it means. For example, in my blind testing years ago, I found a group of people who preferred MP3 over lossless FLAC. I personally have no problems with this (since I don't think high bitrate MP3 sounds bad at all) but it would be very much "counterculture" to various "purist" forms of audiophilia (including strict objective ideal of "high fidelity" and absolute "transparency") that seeks less manipulation of the sound. Would be rather unexpected if audiophiles ever advocated for such a thing because for some it's "euphonic"!

 

Then again, some audiophiles already are strict vinyl listeners so maybe that's a good example where for some "euphonic" is all that really matters even if it's not to my taste personally...

[Archimago]

37 minutes ago, fas42 said:

 

Howdy, just curious whether you're familiar with Bregman's "Auditory Scene Analysis", and subsequent research?

Not familiar with this Frank.

 

I see there's a review here that seems like a good read perhaps:

https://www.frontiersin.org/articles/10.3389/fnhum.2011.00158/full

 

What are your thoughts on this?

 

[fas42]

Yes, would be worth diving in ... note a thread over on ASR, where John Kenny and I posted a lot of thoughts, and links on this area of research, https://www.audiosciencereview.com/forum/index.php?threads/auditory-scene-analysis.236/

 

[Bill Brown]

2 hours ago, Archimago said:

 

Hi Bill,

I certainly agree that many things we measure today are either minor or do not lead to audible differences. A great example I think is JITTER. Yes, we can measure this with all those sidebands down below -100dBFS in modern asynchronous DACs, but seriously, who can hear them unless VERY severe - basically "broken", "engineering flaw" gear? (See my DEMO post awhile back.)

 

Thank you for the long reply, Archimago.  It is interesting.  I agree with a lot of what you wrote, but there are some subtle differences, even just in context, or history, or something.  I'll try to explain.

 

I do agree with the above in the sense that Jitter is a "solved" problem and should not cause audible problems or be an issue in a competent design.  I can't think that I would ever consider buying a DAC that tested poorly in this regard.  I have to think back, though to the (?) 80s when this problem wasn't being addressed by designers of audio components (as opposed, possibly, to the users of digital circuits in other fields).  Objectivists at the time thought digital as it existed was "solved."  DACS, I think it is clear to see in retrospect were "tweaked," or "kludged" by "high-end" designers in the search for "better sound" (?euphony).

 

As things evolved I learned about jitter via Stereophile, saw early measurements being conducted and how poorly some DACs measured (as well as some of the tweaky boxes that were designed to fix this and didn't):

 

https://www.stereophile.com/content/2020-jitter-measurements

 

A subjectivist provocateur would say that before jitter was fixed in DACs that they were being told by objectivists that they were "crazy," that digital was "solved," and that no additional measuring techniques were needed as all important parameters could be characterized with the current state of the art......

 

But yes, I agree jitter is solved and am not holding my breath for another clear, measurable area for improvement in digital-reproduction to be found.

2 hours ago, Archimago said:

Despite this, notice how "jitter" continues to be claimed by various companies as being important... Go talk to Paul McGowan and Ted Smith and their PS Audio DAC. Isn't it more than a little suspicious that they never provide measurements/evidence to show the audible issue despite their verbal claims!?

 

Yeah, not sure I get what is going on there, to say it politely.  I have seen the measurements on Sphile and ASR and am not terribly impressed.  Some people that I respect like them, so....heck, I don't know.

 

As a consumer I don't really care if they don't provide measurements, it's up to them.  Would I buy one? No.  Would I try to "save" someone from being "fooled" into buying one?  No.  Would I take the opportunity to discuss the facets if they wanted?  Sure.  If they are happy and enjoying their music, more power to them.  I am not a crusader.  Don't think it would work anyway, people are so, so variable.

 

2 hours ago, Archimago said:

Your comment "develop a body of measurements that correlate with perception (positive, negative, and euphonic)" is insightful. Nice. I believe that it is only with objective testing and controlled listening (measurements of the subjective experience while minimizing non-sound-related biases) that we can differentiate "positive, negative, and euphonic". I think in fact we are getting to the point where of the three, "euphonic" is becoming most important.

 

Again, I agree to some degree :).  Objective testing? Yes!  I'll take all the sources of information I can get!  Controlled testing?  Sure, but.....  I like the blinded testing at Harman as an example.  I am simply not yet convinced that it is universally applicable to the testing of all audio gear, though hopefully, as a physician you won't find me too hypocritical for believing in blinded, placebo-controlled trials for medicines, etc......

 

Not sure if you recall, but decades ago JA organized a carefully controlled DBT of amplifiers.  One a tube amp, the other SS.  The tube amp had an output impedance that would have to produce frequency-response changes into the varying impedance v frequency of the speakers that were audible by anyone's criteria.  Null result.  Why?  I don't know.  I have some guesses but I don't know.

 

Yes, euphony the way you describe it is huge.  Are subjectivist reviewers now simply describing differences in euphony?  Certainly possible (thought not exclusively, my gut says).

 

2 hours ago, Archimago said:

Would be rather unexpected if audiophiles ever advocated for such a thing because for some it's "euphonic"!

Some, probably.  I hope others, though, are more self-aware and consider it a possibility.  Or maybe not

 

I suspect you are familiar with Nelson Pass' work in this regard.  He is fairly matter-of-fact that he knows 2H HD is euphonic, this informs his designs, and has even produced a gizmo that allows manipulation of its phase and level and told folks to have fun.

 

Re TTs.......In the 90s I built one using Well Tempered Reference components (arm, motor, platter, bearing).  I used multiple levels of Aluminum alternating with Isodamp for CLD, all mounted on a Newport laser table (just the base), pneumatically isolated and self-leveling.  Enjoyed lots of good sound.

 

Now, though just listen to music from an HD, having sold off most of our stuff (not just audio) in a wonderful, simplifying way.

 

I still have lots of files I recorded digitally, though

 

Thanks again,

 

Bill

 

 

[tapatrick]

12 hours ago, Archimago said:

 

Hi @tapatrick,

Yeah, very good question but a broad one also...

 

 

Thanks @archimago! I understand these are broad questions so your thoughts are appreciated. I will digest and see where that leads.

 

I must say the couple of new streams on exploring the respective value of subjective and objective perspectives are very informative with a wealth of references and take more time to absorb.

[Archimago]

15 hours ago, fas42 said:

Yes, would be worth diving in ... note a thread over on ASR, where John Kenny and I posted a lot of thoughts, and links on this area of research, https://www.audiosciencereview.com/forum/index.php?threads/auditory-scene-analysis.236/

 

Thanks Frank,

I'll take a look at the review first and consider. The thread looks pretty unwieldy and will need to have a look at the basics first to see if the topic seems to correlate significantly towards audiophilia...

[Archimago]

14 hours ago, Bill Brown said:

 

Thank you for the long reply, Archimago.  It is interesting.  I agree with a lot of what you wrote, but there are some subtle differences, even just in context, or history, or something.  I'll try to explain.

 

I do agree with the above in the sense that Jitter is a "solved" problem and should not cause audible problems or be an issue in a competent design.  I can't think that I would ever consider buying a DAC that tested poorly in this regard.  I have to think back, though to the (?) 80s when this problem wasn't being addressed by designers of audio components (as opposed, possibly, to the users of digital circuits in other fields).  Objectivists at the time thought digital as it existed was "solved."  DACS, I think it is clear to see in retrospect were "tweaked," or "kludged" by "high-end" designers in the search for "better sound" (?euphony).

 

As things evolved I learned about jitter via Stereophile, saw early measurements being conducted and how poorly some DACs measured (as well as some of the tweaky boxes that were designed to fix this and didn't):

 

https://www.stereophile.com/content/2020-jitter-measurements

 

A subjectivist provocateur would say that before jitter was fixed in DACs that they were being told by objectivists that they were "crazy," that digital was "solved," and that no additional measuring techniques were needed as all important parameters could be characterized with the current state of the art......

 

Good points there Bill. I was but a lad back in the 80's (especially early to mid 80's) before having the resources to own or explore hi-fi so can't really speak to what the debate was like between objective and subjective folks...

 

From our vantage point today,  we can say that there have been many parameters improved over the 80's when it comes to digital reproduction. For example, the first DACs were not true 16-bit even, much less the hi-res-capable low noise floor devices of today. Linearity was poorer back then. Quality of the filtering with the earliest devices would not have been as good. All these things I suspect would have been significant and IMO more important than jitter right from the start.

 

Today, we can see that some companies prefer to roll back the clock in a number of those areas. For example Audio Note CD/DACs do not apply a filter. Various less-than-stellar multibit DAC chips have been used by companies like Border Patrol, or Schiit. Discrete resistor ladder that would measure poorly used in others. Regardless of other measured performance parameters, I believe all these designs actually worsen jitter yet so many audiophiles are fine with it and pay quite a bit of $$$.

 

Among all that can be anomalous and "imperfect" about digital reproduction, I don't think jitter is or ever was a big deal. Nothing like the time domain anomalies of audible turntable wow & flutter for example.

 

Quote

But yes, I agree jitter is solved and am not holding my breath for another clear, measurable area for improvement in digital-reproduction to be found.

 

Yeah, not sure I get what is going on there, to say it politely.  I have seen the measurements on Sphile and ASR and am not terribly impressed.  Some people that I respect like them, so....heck, I don't know.

 

As a consumer I don't really care if they don't provide measurements, it's up to them.  Would I buy one? No.  Would I try to "save" someone from being "fooled" into buying one?  No.  Would I take the opportunity to discuss the facets if they wanted?  Sure.  If they are happy and enjoying their music, more power to them.  I am not a crusader.  Don't think it would work anyway, people are so, so variable.

 

Right. I say things but I too am not all that interested in the "crusade". No need to "fight" or force anyone's personal beliefs. Best to hopefully speak one's position, offer insights to consider, and even better, provide evidence where one can and invite others to partake if this could add to our collective understanding and experience (DEMO's and BLIND TESTS for example).

 

As in many other areas of life, it's important to aim for "higher ground" when it comes to these debates. The "how" and "why" are just as important as the "what" or "who" is being debated.

 

Quote

 

Again, I agree to some degree :).  Objective testing? Yes!  I'll take all the sources of information I can get!  Controlled testing?  Sure, but.....  I like the blinded testing at Harman as an example.  I am simply not yet convinced that it is universally applicable to the testing of all audio gear, though hopefully, as a physician you won't find me too hypocritical for believing in blinded, placebo-controlled trials for medicines, etc......

 

It's OK man... As above, no need to force everyone into the same mindset or methodology! In the "art" of medicine, let's just say that the placebo effect is very much a tool just as much as objective investigations and well-researched therapeutic options. Likewise, there are things we might believe in that "works" but it would be impossible to do blind testing due to ethical reasons.

 

I will say one thing about the importance of blind tests though. When we come across a claim by a "pure subjectivist" that he/she "can easily hear the difference between the two USB cables" (or whatever contentious issue), I would consider that person as a prime candidate for blind testing.

 

We know we cannot trust everyone. We know that some people are not very insightful. And yes, we know that there is a range of hearing acuity as well so it's good to keep an open mind that maybe the person truly has "golden ears" and knows what to listen for. A blind test is a tool to confirm their claim, especially when it's supposedly "easy".

 

Quote

Not sure if you recall, but decades ago JA organized a carefully controlled DBT of amplifiers.  One a tube amp, the other SS.  The tube amp had an output impedance that would have to produce frequency-response changes into the varying impedance v frequency of the speakers that were audible by anyone's criteria.  Null result.  Why?  I don't know.  I have some guesses but I don't know.

 

Yes, euphony the way you describe it is huge.  Are subjectivist reviewers now simply describing differences in euphony?  Certainly possible (thought not exclusively, my gut says).

 

We obviously cannot speak authoritatively about JA's test, and many others where DBT failed to show a difference unless we were there. All kinds of things like hearing ability of participants, room acoustics, ambient noise, music chosen, whether the procedure respected the limits of echoic memory are all significant. Also, the speakers would clearly have a huge part to play in audibility; what is the effect of the tube amp's lower damping factor on the frequency response?

 

What we can say from the result is at least that the listeners must have thought the devices sounded "equivalent" within the confines of that test procedure. Presumably this means the listeners would have been happy with either the SS or tube amp at home. I know some subjective folks will say that after 6 months, the solid state didn't give them as much joy as the tube amp or something like that based on longterm listening... Maybe, but then again, people fall out of love after 6 months all the time for all kinds of reasons not related to audio quality 😄.

 

Quote

Some, probably.  I hope others, though, are more self-aware and consider it a possibility.  Or maybe not

 

I suspect you are familiar with Nelson Pass' work in this regard.  He is fairly matter-of-fact that he knows 2H HD is euphonic, this informs his designs, and has even produced a gizmo that allows manipulation of its phase and level and told folks to have fun.

 

Re TTs.......In the 90s I built one using Well Tempered Reference components (arm, motor, platter, bearing).  I used multiple levels of Aluminum alternating with Isodamp for CLD, all mounted on a Newport laser table (just the base), pneumatically isolated and self-leveling.  Enjoyed lots of good sound.

 

Now, though just listen to music from an HD, having sold off most of our stuff (not just audio) in a wonderful, simplifying way.

 

I still have lots of files I recorded digitally, though

 

Thanks again,

 

Bill

 

 

Yup, I've had this discussion with @mitchco over the years and some local audiophiles about adding euphonic distortion including Pass' amp discussions. The cool thing is that with @pkane2001's Distort program you can easily do the same digitally by adding some low order harmonic and hear for yourself.

 

Have fun!

 

[John Dyson]

For people who aren't tied up in with strong opinons, I see the disagreement between those who tend to be 100% subjective vs. those of us who tend (but not 100%) objective partially described in the following statement:  it is mostly related to the fact that SOMETIMES it is difficult to measure certain impairments.   I think that people move towards the subjective out of frustration caused by inadequate objective information.  (There can probably be a lot of  reasons for the incomplete/inadequate objective information.)

 

Some of the problems with objective review might include: the impairments aren't adequately defined/described, sometimes the impairments are difficult to measure, and sometimes even the impairments come from multiple sources that meld together into something wrong with the sound.

 

Example: on the case of TIM, which can really exist (and used to REALLY exist in older designs), we didn't initially 1) understand what caused the impairment, and 2) it can be tricky to measure, esp with techniques used in the '60s/'70s.   TIM is a sibling of modulation distortion in gain control devices also, it happens when signals are changing character...  It can take a while to understand,define, describe an impariment well enough to quantify it.  TIM and siblings can be tricky to measure, and there more than 'one' kind of TIM in the sense it is dependent on lots of variables.   This is ONE example that might have discredited 'measurements' in some peoples minds.

 

This doesn't mean that objective measurement can be discredited, in fact when it is applicable, it must be a PRIMARY way of evaluating a design, and eventually the subjective becomes a double check.  On the other hand, complex designs can demand certain kinds of testing where there is no appropriate measurement device.  Subjective evaluation is needed until (if ever) an objective method is developed.

 

I can give parallels to the situation on my current project -- but I do not want to divert attention from the matter at hand.  The bottom line is that objective measurement and evaluation is critical for a design to be created and completed.  A pure 'design by sounds good' is only going to work for the most simple design with simple interactions.   A 'design by spec' and requirements to meet objective criteria is important for a non-trivial design to be workable.

 

NOTE:  design by finding 'sweet spots' and avoiding 'rabbit holes' is NOT the best engineering design method and can sometimes make the best of us into a sucker, wasting lots of time doing tweaking.   If the tweaking can be avoided, then tweaking MUST be avoided and it is very worthwhile to sit down and do a real design -- the worst of time wasting chasing rabbits into rabbit holes can be avoided.

 

On conventional circuitry and software, where the specifications, requirements and behavior can be accurately measured, almost pure objective design is best.  Secondary subjective review is also important, sometimes specs and measurements miss details that are unforseen.

 

On complex circuitry and software, that is, 'stuff that hasn't been done very often', then subjective review is so important, but objective focus is necessary -- the degrees of freedom and interactions could cause a 'design by sounds good' to become a random walk, falling into rabbit holes all of the time.

 

WIth my mostly objective view, sometimes I must unfortunately depend on my hearing, or accept input from other peoples perception.  Very often, the subjective feedback has been helpful in resolving actual bugs that I couldn't measure.   Subjective feedback from uncontrolled experiments is notoriously unreliable though -- and must be considered on a statistical basis and not as a measurement with negligible error.   Subjective review can become so distorted and become totally emotional.   When feedback is too distorted by emotion and/or all of the human foibles that can affect subjective review, then that data source should be ignored.  I run into that problem all of time, and must quit depending even on my own senses  -- human perception is definitely unreliable, but good information can often be derived.

Both subjective and objective review are necessary -- but each has it's limitations.

In a way, after a person really considers and truly understands the complexity of their equipment, I cannot imagine why someone would be strongly biased away from objective review.   It just doesn't make sense to disparage good objective review of any technical device.   Subjective review is important also -- it is the attitude that disparages one or the other -- doesn't make sense.

 

John

 

[pkane2001]

1 hour ago, Archimago said:

The cool thing is that with @pkane2001's Distort program you can easily do the same digitally by adding some low order harmonic and hear for yourself.

 

Have fun!

 

As has been mentioned elsewhere, you can also do a listening test with different types and amounts of jitter using DISTORT.

 

Have even more fun!

 

[Bill Brown]

2 hours ago, Archimago said:

Good points there Bill. I was but a lad back in the 80's (especially early to mid 80's) before having the resources to own or explore hi-fi so can't really speak to what the debate was like between objective and subjective folks...

 

  

 

I was 14 in 1982 when the first CD players came out(?).  I began reading everything about this hobby that became my lifelong love then, went through all the normal stages (gear-lust, reading the Audio Magazine annual equipment catalog like a kid used to read the Sears catalog before Christmas, etc.).  I would watch the mailbox for the latest issues of Audio, Stereophile, and, I must confess, Stereo Review and inhale all of them in less than a day, then resuming the month-long wait.  Can't believe this all makes me feel old at 51!  This was basically the ONLY way to get info on the hobby except for the one friend I found in college that shared my passion!  An AMAZING contrast with today.  Wow!

 

History is important to me.  Subjective reviewing grew from JGH for Stereophile, and a bit later HP from TAS as a reaction to the measurement only era preceding it, seen in magazines like High Fidelity(?).  JGH swung the pendulum towards subjectivity and I think it was needed.  Remember, though, that he used measurements as able at the time- find one of his old reviews and check out a primitive FR measurement of a phone cartridge!  He was in pursuit of "high fidelity," in, I think, the truest definition of the term, and actually decried the further swing of the pendulum towards the subjectivist-only, "how the sound makes me feel," "if it sounds good to me" type of pursuit.

2 hours ago, Archimago said:

I say things but I too am not all that interested in the "crusade". No need to "fight" or force anyone's personal beliefs.

 

Yes, I sense a healthy, pragmatic, non-judgemental approach to your writing and the engineering chops you bring to the table- whoever the heck you are!

 

I was thinking of the things this morning that would lead me to a "righteous crusade."  There are some, perhaps, but certainly none in this realm.

2 hours ago, Archimago said:

I will say one thing about the importance of blind tests though. When we come across a claim by a "pure subjectivist" that he/she "can easily hear the difference between the two USB cables" (or whatever contentious issue), I would consider that person as a prime candidate for blind testing.

 

Yes, why someone would set themselves up for that is a mystery to me.  I am confident about some stuff, but........ I try to minimize the possibility of getting even close to Hubris.

 

3 hours ago, Archimago said:

what is the effect of the tube amp's lower damping factor on the frequency response?

 

I can't remember the exact numbers, but certainly differences that should be perceived as outlined in the relevant literature, my understanding of which suggests that narrow cuts and peaks are difficult to perceive, but that (even fairly subtle) frequency response changes that affect wider frequency bands are perceptible.  Note that in this case, as suggested by JA's measurements of amps into a simulated load, that they should have been.  Again....euphony?

 

I don't want to overstate, though, the test as a "gold standard."  I found it intriguing at the time and it remains in the back of my mind, especially with my personal experiments with blind testing.

 

Bill

[fas42]

6 hours ago, John Dyson said:

 

Some of the problems with objective review might include: the impairments aren't adequately defined/described, sometimes the impairments are difficult to measure, and sometimes even the impairments come from multiple sources that meld together into something wrong with the sound.

 

 

Some of the impairments are easy to detect, and describe - my first good amplifier of 35 years ago, a supposed powerhouse, would start distorting in the treble, when its circuit reached a certain load into the speakers - I used the splashing of cymbals in a driving rock track to pick this; below the particular SPLs a very natural shimmer to the sound, above that sound level, they became saucepan lids - majorly distorted. To put this into context all other amplifiers I used this test on, that I came across, fared far worse - a Krell was particularly poor, 😜.

 

This was straightforward to resolve, though it took some time - for a couple of other reasons I suspected the amplifier's power supply, and after some major re-engineering of this area, this problem then disappeared.

 

The point is that it's not always hard - if one has the right test material, and a good understanding of electronics then the logical steps needed to resolve less than stellar SQ can be followed without too much difficulty.

[STC]

It is not without basis for audiophiles to reject measurements. I have long abandon room measurements for few years now as it did not tell what exactly sounds nice to my or visitors ears. 
 

Measurements are useful for making equipments. I buy equipments based on good measurements. I don’t judge sound based on measurements. If you let measurements, price, colour, brand, designer influence your perception than it is hard to be objective in your subjective assessment of sound quality. 

 

For a long time I was in a dilemma whether to stick to the flat room response or to go with a pool of subjective preference of the listeners. In the end, I went with what I like the best.  Of course, I can switch between the flat and my preference in a flash with just a mouse click so I can back up and provide direct comparisons between perfect flat measurements and general listeners preference. 

 

Sound quality is objective. It can be measured. But sound quality is not what makes a sound likeable. This is governed by psychoacoustics and and discussion about sound production without discussing this will be forever reach no consensus on what is desirable and what’s not. 
 

For anyone who still thinks measurements can objectively tell you about the SQ should just take a good mono recording and play them in your preferred system with ONLY ONE speaker in the centre.  Do it blind folded and you would be surprised if you could reliable tell if there is a difference between DAC, cables and others. The only exception would be speakers. And you are in for more surprises if you measure the most preferred speakers than it is very unlikely to measure perfectly. 
 

Our hearing is not flat nor consistent. It varies through out the day. Give yourself an EQ and start writing down all the settings throughout one year without any changes to the equipment and you will notice that you are actually in a big circle that the changes are repetition that your preference changes for no apparent reason. Maybe, this also explains why audiophiles are always chasing equipments. 

[tapatrick]

10 hours ago, STC said:

Our hearing is not flat nor consistent. It varies through out the day.

Certainly would explain a lot. I always wonder when listening faculties are treated as some kind of scientific instrument.  

[Archimago]

On 2/23/2020 at 12:16 PM, Bill Brown said:

 

  

 

I was 14 in 1982 when the first CD players came out(?).  I began reading everything about this hobby that became my lifelong love then, went through all the normal stages (gear-lust, reading the Audio Magazine annual equipment catalog like a kid used to read the Sears catalog before Christmas, etc.).  I would watch the mailbox for the latest issues of Audio, Stereophile, and, I must confess, Stereo Review and inhale all of them in less than a day, then resuming the month-long wait.  Can't believe this all makes me feel old at 51!  This was basically the ONLY way to get info on the hobby except for the one friend I found in college that shared my passion!  An AMAZING contrast with today.  Wow!

 

Hey Bill, cool man, we're similar in age... I think Chris here is about the same. Fellow Gen X cynical slackers 😉.

 

Yeah, those were the days with magazines and waiting a month before the next "hit". For better or worse, we have everything at our fingertips yet so much of what we argue about remain the same!

 

On 2/23/2020 at 12:16 PM, Bill Brown said:

History is important to me.  Subjective reviewing grew from JGH for Stereophile, and a bit later HP from TAS as a reaction to the measurement only era preceding it, seen in magazines like High Fidelity(?).  JGH swung the pendulum towards subjectivity and I think it was needed.  Remember, though, that he used measurements as able at the time- find one of his old reviews and check out a primitive FR measurement of a phone cartridge!  He was in pursuit of "high fidelity," in, I think, the truest definition of the term, and actually decried the further swing of the pendulum towards the subjectivist-only, "how the sound makes me feel," "if it sounds good to me" type of pursuit.

 

Yes, I sense a healthy, pragmatic, non-judgemental approach to your writing and the engineering chops you bring to the table- whoever the heck you are!

 

That's an interesting perspective Bill. And I think it does jive with JGH's latter comments like this about the lack of "honesty controls" in audio. Certainly sounds like it went too far for the man.

 

A subjective-only perspective ultimately leads us down the path of solipsism. All that matters is "me". What "I" think. How I "feel". Where I spend my money. For the most part, how a person "feels" about consumer devices isn't all that interesting to me, nor where the money is spent; this is a hobby so by all means have "fun". But if we as hobbyists are interested in moving forward beyond aesthetics and appreciate that there should be "function over form", then it's essential that we examine the actual "function" and objective methods of course will allow us to quantify this.

 

I'm ultimately just one of the audiophiles you run into at the audio show with enough time to stick an ADC to some audio gear and spent some time reading, writing and interacting with other audiophiles... 😃

 

On 2/23/2020 at 12:16 PM, Bill Brown said:

I was thinking of the things this morning that would lead me to a "righteous crusade."  There are some, perhaps, but certainly none in this realm.

 

Yes, why someone would set themselves up for that is a mystery to me.  I am confident about some stuff, but........ I try to minimize the possibility of getting even close to Hubris.

 

No need for any crusade in this realm, I agree. If we want to be "righteous", there are plenty more important and useful causes to devote time to in the real world!

 

As for why one would "set themselves up" to saying that certain things are "easy" to hear or they "clearly" can tell a difference, I think for the most part it's because this sells, and looks good for companies that want a positive review. Obviously I'm talking about stuff like cables, bitperfect streamers, bitperfect software, etc... where the chance of differences is highly unlikely (unless the device is very poorly made or broken!).

 

Cable reviews are a perfect example of this. If a magazine has already reviewed many cables in their long and distinguished run, has all kinds of advertising banners for the latest and greatest cable company, isn't it essential that a $2000/ft cable "clearly" sounds different when compared to that $1000/ft cable reviewed recently?

 

That whole construct is necessary for the confidence game. It's no good with Company C's $250/ft cable now suddenly sounds better than Company B's $2000/ft model. What if next month they "honestly" now report on a $50/ft model that trounces that $250/ft cable even!!!

 

No... It has to be "measured" (not objectively obviously!) reporting for the sake of the class of devices/components by a "respected" golden ear no less. It has to roughly at least correlate to the price structure. And it would do no good for reviews to suggest anything less than "clearly audible difference" as there would simply be no hype generated and apathy in the readership is the last thing a cable company (or other questionable device manufacturer) wants to see in print about their product!

 

The trap with this kind of writing of course is that they can't be "caught". Otherwise the whole "golden ear" facade comes crumbling down. The fantastical price structure collapses like a financial bubble that was unwisely "invested in" by those who followed. Hence... No blind tests. "It doesn't work." "It doesn't reflect actual listening situations."

 

And the related cries of: "measurements don't capture everything we hear." "You can't measure cables." So on and so forth...

 

On 2/23/2020 at 12:16 PM, Bill Brown said:

 

I can't remember the exact numbers, but certainly differences that should be perceived as outlined in the relevant literature, my understanding of which suggests that narrow cuts and peaks are difficult to perceive, but that (even fairly subtle) frequency response changes that affect wider frequency bands are perceptible.  Note that in this case, as suggested by JA's measurements of amps into a simulated load, that they should have been.  Again....euphony?

 

I don't want to overstate, though, the test as a "gold standard."  I found it intriguing at the time and it remains in the back of my mind, especially with my personal experiments with blind testing.

 

Bill

 

As suggested above, I do believe that in the academic literature we can already find all kinds of work on audibility thresholds which we in the audiophile world just do not talk about.

 

Until it becomes part of the lingo surrounding the products we consume, we often will not focus on esoteric details... For example, until around 2009/2010, we didn't talk much about digital filters, or pre-ringing, or impulse responses with DACs; not until Meridian started to market their "apodizing" filter, Ayre their "Listen" and "Measure" settings, and the magazines picked it up. Now days, it's part of our lingo even though for the most part, it's not a big deal. (Same too with stuff like "intersample overs" in the last few years.)

 

[Archimago]

On 2/23/2020 at 9:10 AM, John Dyson said:

For people who aren't tied up in with strong opinons, I see the disagreement between those who tend to be 100% subjective vs. those of us who tend (but not 100%) objective partially described in the following statement:  it is mostly related to the fact that SOMETIMES it is difficult to measure certain impairments.   I think that people move towards the subjective out of frustration caused by inadequate objective information.  (There can probably be a lot of  reasons for the incomplete/inadequate objective information.)

 

Some of the problems with objective review might include: the impairments aren't adequately defined/described, sometimes the impairments are difficult to measure, and sometimes even the impairments come from multiple sources that meld together into something wrong with the sound.

 

Example: on the case of TIM, which can really exist (and used to REALLY exist in older designs), we didn't initially 1) understand what caused the impairment, and 2) it can be tricky to measure, esp with techniques used in the '60s/'70s.   TIM is a sibling of modulation distortion in gain control devices also, it happens when signals are changing character...  It can take a while to understand,define, describe an impariment well enough to quantify it.  TIM and siblings can be tricky to measure, and there more than 'one' kind of TIM in the sense it is dependent on lots of variables.   This is ONE example that might have discredited 'measurements' in some peoples minds.

 

This doesn't mean that objective measurement can be discredited, in fact when it is applicable, it must be a PRIMARY way of evaluating a design, and eventually the subjective becomes a double check.  On the other hand, complex designs can demand certain kinds of testing where there is no appropriate measurement device.  Subjective evaluation is needed until (if ever) an objective method is developed.

 

Hi John, yes, well put!

 

Many of the fights we get into originates from the "all or none", "black or white", "100%" mindset. As mature adults, we know that the only way to handle the complexities of life (of which the squabbles of audiophilia is but a tiny microcosm) is to find the middle ground... The "shade of grey" between the subjective and objective. Some things do need to be "more subjective" just as my preference is to be "more objective" when it comes to audiophile gear.

 

It's sad to see how there was a time when at least we appreciated some of the "basics" of objective analysis (like Bill mentioned about JGH and phono frequency response). These days, the majority of online sites and at least a big proportion if not the majority of magazines have taken out even a simple frequency response; much less consideration for noise level or time-domain characteristics.

 

In part, I think that as technology has progressed, we can often "assume" that most devices are quite "good" already. Or at least good enough that most listeners will not hear a problem. While technically not great, it's fine when the subjectivist reviewer doesn't hear a problem with a DAC but yet John Atkinson measures relatively high jitter... Since jitter (IMO) was never a terrible boogeyman, the device still "sounds good" assuming the rest of the performance is reasonable. The issue then becomes one of "value" for the price (given suboptimal engineering), and whether it would be appropriate for the company to hype up a product as "one of the best DACs in the world" for example.

 

I can certainly agree with the idea that objective parameters should take primacy in design and then of course confirmed with listening. Sure, we have some products "designed by ear" and who knows, some will sound good as well. Probably best to at least double check with objective measures if something is designed by the ears of a man with many decades of life behind him 😲.

 

When it comes to audible distortions, using multiple blinded listening tests to identify and verify audible anomalies then use objective testing to measure the problem probably is the best way to effectively weed out issues, and keep findings relevant (eg. audible TIM back in the day might be an example). These days, the "golden ears" who claim to hear differences but never bother to run a blind test to prove to themselves and others that the impression is anything more than a mirage would not be helpful in this endeavor.

 

On 2/23/2020 at 9:10 AM, John Dyson said:

I can give parallels to the situation on my current project -- but I do not want to divert attention from the matter at hand.  The bottom line is that objective measurement and evaluation is critical for a design to be created and completed.  A pure 'design by sounds good' is only going to work for the most simple design with simple interactions.   A 'design by spec' and requirements to meet objective criteria is important for a non-trivial design to be workable.

 

NOTE:  design by finding 'sweet spots' and avoiding 'rabbit holes' is NOT the best engineering design method and can sometimes make the best of us into a sucker, wasting lots of time doing tweaking.   If the tweaking can be avoided, then tweaking MUST be avoided and it is very worthwhile to sit down and do a real design -- the worst of time wasting chasing rabbits into rabbit holes can be avoided.

 

On conventional circuitry and software, where the specifications, requirements and behavior can be accurately measured, almost pure objective design is best.  Secondary subjective review is also important, sometimes specs and measurements miss details that are unforseen.

 

On complex circuitry and software, that is, 'stuff that hasn't been done very often', then subjective review is so important, but objective focus is necessary -- the degrees of freedom and interactions could cause a 'design by sounds good' to become a random walk, falling into rabbit holes all of the time.

 

I think your current project with the "feral" Dolby A material is a beautiful example and some of the results I've heard certainly speaks to this. A nice melding of the objective science of how the old analogue noise-reduction system worked with its various filter bands yet there is the need for subjective listening to get the variables sounding "right" - the "art" side of things... A reflection of how folks in the studio also used their ears to tweak the music for the best sound and would have applied customized settings here and there to achieve the ultimate goal.

 

Needless to say, the moment we have real music involved (not just reproduction of "data" with high fidelity), there is no option but to integrate "art" and subjectivity into the equation.

 

On 2/23/2020 at 9:10 AM, John Dyson said:

WIth my mostly objective view, sometimes I must unfortunately depend on my hearing, or accept input from other peoples perception.  Very often, the subjective feedback has been helpful in resolving actual bugs that I couldn't measure.   Subjective feedback from uncontrolled experiments is notoriously unreliable though -- and must be considered on a statistical basis and not as a measurement with negligible error.   Subjective review can become so distorted and become totally emotional.   When feedback is too distorted by emotion and/or all of the human foibles that can affect subjective review, then that data source should be ignored.  I run into that problem all of time, and must quit depending even on my own senses  -- human perception is definitely unreliable, but good information can often be derived.

Both subjective and objective review are necessary -- but each has it's limitations.

In a way, after a person really considers and truly understands the complexity of their equipment, I cannot imagine why someone would be strongly biased away from objective review.   It just doesn't make sense to disparage good objective review of any technical device.   Subjective review is important also -- it is the attitude that disparages one or the other -- doesn't make sense.

 

John

 

 

"Subjective feedback from uncontrolled experiments is notoriously unreliable though". Yes, for sure. I suspect most audiophiles appreciate this fact even though strangely enough, it seems like some have unwavering faith in certain high profile "golden ears"... "In Mr. Purely Subjectivist Reviewer's ears we trust!" Without a fair amount of critical thinking, seems a little dangerous 😟.

 

Thanks again for the work and thoughtful insights John.

[Audiophile Neuroscience]

The relevance of measurements in audio seems to be a popular theme at the moment.

 

We know from other threads here and elsewhere that it is difficult to completely correlate objective measurements of an audio signal with subjective perceptions of what that measured signal will sound like to a human listener.

 

Measurements of the audio signal are not direct measurements of perception. They are surrogates or indirect markers. In medicine such markers are used all the time but the difference is we know how that indirect marker compares to a gold standard test of what you think you are measuring. It's just calibration of one tool against another known, and more accurate, tool.

 

In my opinion, we haven't got that far in audio yet…….( this is despite blind testing. As a test tool, in my opinion it is not adequately calibrated and without known sensitivity, specificity, true and false positives/negatives, and positives and negative predictive values. Another topic).

 

Still, in my opinion there is hope. By definition, irrespective of how complex a perception is, it must have an evoking stimulus. That is basic neurobiology. I am excluding things here like hallucinations and the huge topic of central modulation in the nervous system.

 

So, the stimulus in this case is an audio signal followed by sound waves in air. The principle is pretty straightforward in that you can study that stimulus to observe what changes in the stimulus correlate, and are concordant with, some sort of change in perception.

 

I mentioned elsewhere that I don't think it should be terribly difficult to study such things as dynamic range measurement and compare that with our subjective evaluation of compressed or dynamic sound perception. Similarly, frequency response that doesn't extend down into the bottom octaves of music is expected to sound a certain way, or at least lacking a certain sound.

 

It seems to me that where we're at with objective measurements is to verify  that a piece of audio gear is performing to spec and maybe explore better and alternative designs. There are measured levels of jitter, various distortions, frequency response and so on. Some of these measurements can give us some insight into what the gear might sound like, how it is "voiced". But frustratingly to most audiophiles there is often a disconnect between the measurement of the device's performance and how it sounds.

 

The implicit suggestion/understanding/belief that the better the device's measured performance the better the potential sound quality has strong face validity but it is not always borne out in reality. It probably holds more true in the converse, in that lousy measurements are probably going to translate into poor sound quality but even this does not hold completely true.

 

In my opinion therefore we need to establish measurements of the audio signal that not just informs us how the device is performing, as important as that is, but informs us how the sound/music will be perceived, i.e. what will it sound like.

 

What I think most subjectivists sceptically reject is the notion that excellent device specs translates into sonic transparency and the corollary, that an excellent set of specs means that the device will sound like any other device with the same specs. I don't think it is an unobtainable goal just think we're way off reaching it, in my opinion.

[jabbr]

7 hours ago, Archimago said:

 

When it comes to audible distortions, using multiple blinded listening tests to identify and verify audible anomalies then use objective testing to measure the problem probably is the best way to effectively weed out issues, and keep findings relevant (eg. audible TIM back in the day might be an example). These days, the "golden ears" who claim to hear differences but never bother to run a blind test to prove to themselves and others that the impression is anything more than a mirage would not be helpful in this endeavor.

Yes with a caveat. 
 

Measurements have multiple purposes. If you are really looking at something like TIM, you need careful testing. TIM is a nonlinearity so you can’t simply plop the sound into an AP frequency analyzer and expect a result. You need to know what you are looking for and do the measurement accordingly. That means knowing a lot about electronics theory. 
 

Similarly the relationship between power supply noise and clock phase error is a nonlinearity. 
 

Blinded listening tests can be hard to do for the engineer testing a prototype circuit on the bench. Blinded tests are needed when there is an identified subjective difference, specifically when “golden ears” hear differences that no reasonable electronics theory predicts.