Why are objective assessments important...

[Audiophile Neuroscience]

The relevance of measurements in audio seems to be a popular theme at the moment.

 

We know from other threads here and elsewhere that it is difficult to completely correlate objective measurements of an audio signal with subjective perceptions of what that measured signal will sound like to a human listener.

 

Measurements of the audio signal are not direct measurements of perception. They are surrogates or indirect markers. In medicine such markers are used all the time but the difference is we know how that indirect marker compares to a gold standard test of what you think you are measuring. It's just calibration of one tool against another known, and more accurate, tool.

 

In my opinion, we haven't got that far in audio yet…….( this is despite blind testing. As a test tool, in my opinion it is not adequately calibrated and without known sensitivity, specificity, true and false positives/negatives, and positives and negative predictive values. Another topic).

 

Still, in my opinion there is hope. By definition, irrespective of how complex a perception is, it must have an evoking stimulus. That is basic neurobiology. I am excluding things here like hallucinations and the huge topic of central modulation in the nervous system.

 

So, the stimulus in this case is an audio signal followed by sound waves in air. The principle is pretty straightforward in that you can study that stimulus to observe what changes in the stimulus correlate, and are concordant with, some sort of change in perception.

 

I mentioned elsewhere that I don't think it should be terribly difficult to study such things as dynamic range measurement and compare that with our subjective evaluation of compressed or dynamic sound perception. Similarly, frequency response that doesn't extend down into the bottom octaves of music is expected to sound a certain way, or at least lacking a certain sound.

 

It seems to me that where we're at with objective measurements is to verify  that a piece of audio gear is performing to spec and maybe explore better and alternative designs. There are measured levels of jitter, various distortions, frequency response and so on. Some of these measurements can give us some insight into what the gear might sound like, how it is "voiced". But frustratingly to most audiophiles there is often a disconnect between the measurement of the device's performance and how it sounds.

 

The implicit suggestion/understanding/belief that the better the device's measured performance the better the potential sound quality has strong face validity but it is not always borne out in reality. It probably holds more true in the converse, in that lousy measurements are probably going to translate into poor sound quality but even this does not hold completely true.

 

In my opinion therefore we need to establish measurements of the audio signal that not just informs us how the device is performing, as important as that is, but informs us how the sound/music will be perceived, i.e. what will it sound like.

 

What I think most subjectivists sceptically reject is the notion that excellent device specs translates into sonic transparency and the corollary, that an excellent set of specs means that the device will sound like any other device with the same specs. I don't think it is an unobtainable goal just think we're way off reaching it, in my opinion.

[Audiophile Neuroscience]

9 hours ago, jabbr said:

Don’t assume that the provided measurements are “specs” for an electronic circuit, We know that, for example, THD does not entirely denote “SQ”

Hi Jonathan,

not sure I completely follow what you mean when you say don't assume that the provided measurements are specs for an electronic circuit. At any rate my intention was to use the word specs or specifications in a broader sense, pertaining to materials, design and performance et cetera - the things that define what a product is and does.

Cheers

David

 

[Audiophile Neuroscience]

9 hours ago, DuckToller said:

sound like a dream to me, but I am not an expert in neuroscience ....

I would have a guess, that the full set of personal factors that determine our subjective perception could only be integrated as approximative & averaged data, which in turn may give us only a divergent perception from the ones we hold ... 
You may want to correct me here, I'd be happy to learn how our lifelong individuality & experiences can be integrated in a measuring model 
Best, Tom

Hi Tom

I believe @Iving has given an expert reply to this.

 

I agree with you that perceptions are a complex amalgam of sensory and psychological influences affected by our experiences and conditioning et cetera. I don't disagree that we have unique experiences and that we can all perceive things differently. I mean the latter both figuratively and literally (another topic).

 

The perception of pain for example is in fact defined by the International Association for the study of pain is a "sensory and emotional experience". Nobody can feel your pain and yet everybody has experienced pain and things like pain thresholds for various stimuli have been studied and defined. In other words it is predictable and reproducible that we share some features in our experience of pain and with some degree of normal variation.

 

In order to perceive anything there must be an evoking stimulus, at least in the context that we are talking about. Without going off on a tangent there are some subtle differences between evoke, illicit and emit but basically it is possible for a stimulus to produce the experience of a sensation, image, feeling, memory et cetera. While no two experiences are identical we do expect a normal range of variation especially when talking about fairly automated responses to physical percepts in neurotypical people.

 

For example, the perception of two point discrimination at the fingertips is about 6 mm in "normal people". There is an acceptable range of variance. Two point discrimination is the ability to discern being touched by one pointy object as opposed to 2 pointy objects. If the pointy objects are closer than 6 mm apart you will perceive it as a single touch, if further than 6 mm apart you will perceive being touched by two objects. Apart from anything else this is a function of the structure and physiology of the nervous system which  subserves the sensation.

 

To my way of thinking it doesn't matter how complex or unique the perception ends up it still has to be discriminated in the first place and this is where I would try to assign the dependent variable. I am not suggesting a reductionist model and believe that psychological influences are always potentially and normally present as part and parcel of the experience.I think this already happens to some extent with things like the measurement of dynamic range and the perception of compressed/loud sound.

 

Where I differ a little bit with @Iving is while I agree that the enjoyment of music is the endgame I don't think you necessarily need to make that the dependent variable. This starts to get far more complicated IMO and harks back to your original statement/question, adding degrees of difficulty for things like preferences, "lifelong individuality & experiences", cognitive associations and interpretations et cetera. It is also the area where in my opinion there is most propensity for bias as your enjoyment of music can and will be influenced by appearance, price, brand et cetera.

 

One can never assume you are not dealing with some kind of bias, even unconscious bias, but by the same token people have very little vested interest in knowing whether for example their two point discrimination at the fingertips is 6 mm or 7 mm. There is rarely a sense of performance anxiety during the test . In 40 years I have never tested a patient who particularly cared about the result. In fact over recent years I have tested both sighted and blinded and have yet to come up with a difference. This is despite the fact that sighted observation does actually influence perception in many areas as we all know and I'm not talking about bias I'm just talking about the way normal perception works.

 

So, rather than talking about preferences or better or worse sound quality per se I am saying to try and establish what part of the measured stimulus correlates and is concordant with a physical percept. I think that is within the realistic realms to achieve, maybe, outside of academia but people very familiar with research methodology and statistics would still need to be involved.

 

I mentioned dynamic range before and compressed sound. Another thing that audiophiles often talk about is "warmth". My understanding is that that spectral balance in the signal is important. Add a touch more upper bass in the EQ and it sounds a bit fuller and warmer. Some forms of distortion add warmth and I am given to believe from @sandyk that some manufacturers tailor the amount of even order distortion in some solid state amplifiers to give a vacuum tube like warmth. I think these things need to be confirmed and in some way made quantifiable. If designers are voicing their gear to sound a certain way then it would be useful to know the measurements that they have relied upon to do so. If they have added even order harmonics it is presumably measurable. The trouble is there is no look up table that tells the ordinary person what that measurement means to their listening experience.

 

 

Cheers

David

 

[Audiophile Neuroscience]

13 minutes ago, jabbr said:

Davd, 

 

Well yes, a complete specification would be the electronic circuit, firmware, drivers etc, but aside from a FirstWatt amplifier, that is never available. Even then, a SPICE simulation of the circuit is not the same...

 

A measurement is not a spec in that sense. Some of these specs are more akin to marketing. Of course you may need a certain wattage amp with certain speakers, and a DSD or PCM DAC input etc.

 

Some measurements like spectra may give an idea about how the product sounds. 
 

Tell me though, do you find that “performance specs” give you an idea about how the products sounds? If so, which specs? I’d say that some common specs give a very general idea but not at the level of detail I’m interested in — I think that some of the nonlinear behaviors give a better idea of the unique product sound but these aren’t commonly measured nor published.

 

This is where i maybe went wrong. I consider all measurements to be specs of some kind."The purpose of a specification is to provide a description and statement of the requirements of a product, components of a product, the capability or performance of a product, and/or the service or work to be performed to create a product."

 

Jonathan (I think) I totally agree with you regarding performance specs and we need better indicators of how products sound in a much greater level of detail than presently exists. Whether that's measuring or quantifying or somehow communicating "nonlinear behaviours" I cannot say. I would much prefer to be informed by objective data even at the end of the day if I reject it and still make my own subjective choice. At least I would have an option, a choice.🤔

Cheers

David

[Audiophile Neuroscience]

10 hours ago, Iving said:

If I I'm not mistaken, @Audiophile Neuroscience is a psychiatrist (medically-trained practitioner)

 

I probably need to see one, call me crazy🤣, but I am not a psychiatrist. I updated my profile with some details of my background just prior to your post I think. I am definitely not a neuroscientist and regret actually including the term in my moniker. When I joined AS years ago I literally wanted to be known as musicophile  but the name was not available. Neuroscience reflects an interest not a qualification.

[Audiophile Neuroscience]

50 minutes ago, jabbr said:

Yes absolutely. I believe in a combination of understanding the electronic circuit, measuring the circuit to ensure that it is behaving as intended, as well as listening!

 

This is the crux of the question: which measurements predict great sound?

 

Hard to say but if a product claims low jitter, then let’s see the measurements. If a power cord or supply supposedly improves a DAC then let’s see the measurement! If a product claims to block leakage current then let’s see the effect n a DAC!

Totally agree. My only qualification would be obviously provided that said measurement exists...and exactly as you say, which measurements predict great sound?

[Audiophile Neuroscience]

18 minutes ago, jabbr said:

Ok how about these measurements — as some examples, there are others:

 

1) isolation from common mode noise (CMRR) — common mode noise, whether through the power supply or inputs, leads to leakage currents, hum etc

2) input rise time sensitivity

3) impedance mismatch sensitivity 

4) power supply noise sensitivity

 

(I’ve heard very audible issues with several components which otherwise measure very well)

Okay, great, thanks for the examples. For many (most?) non technically minded audiophiles only Number 1) isolation from common mode noise (CMRR) has a meaningful correlate people can understand .....it leads to hum.

Don't get me wrong, I'm not saying the other measurements don't affect sound quality it's just that it is not specified in what way.

 

It is as you say I believe the most audiophiles are looking for the measurement that explains what they hear or why two components sound different....... if indeed they are actually hearing something that is measurable.

 

If they are hearing something that is not measurable then there are two possibilities. It is not actually there to be heard in the first place or alternatively, it's some kind of failure of the testing process such as the wrong test looking in the wrong place or other reason for a false negative.

 

 

[Audiophile Neuroscience]

 

Hi Archimago

thanks for the very considered reply. I doubt that you and I will come to the same conclusion at the end of the day but I am okay with that.  I appreciate learning your perspective

 

 

1 hour ago, Archimago said:

I think we can make a case with examples of almost anything we want to argue about depending on which tests we're talking about! 😃

 

This is what I want to know. If you're making a case that the test reveals an abnormality how is that manifest to the listener. If it is not manifest to the listener then why is it important? This brings us full circle to the title of your thread.

 

I fully accept things like frequency response and sufficiently high noise levels can be correlated with audibility and at least in the case of spectral anomalies it can be characterised with subjective descriptions like "warmth" or "air". I would like to learn how each of the other measurements translate to the listening experience.

 

I get your point that THD+N and jitter may no longer be the culprits that they used to be thanks to clever engineers and designers. As I understand it noise induced jitter like in the PHY of Dacs was not even recognised until relatively recent times and thanks to the work of engineers like John Swenson.

 

1 hour ago, Archimago said:

All that we can ever recover or "hear" is in that data. If we can completely, transparently reproduce that "source" with a perfect turntable, perfect CD player, perfect DAC... No losses in the cabling... No losses in the preamp... "Perfect" amplifier to a perfect speaker/headphone that can "faithfully" (as in "high fidelity") reproduce what the data encoded in that source is, then that is all we can hope to do. "Transparency" to the source content is all that we can ever achieve.

 

I agree that being faithful and transparent to the source is all that a perfect reproduction system can do. At this stage I haven't seen your evidence for the other statements. With great respect it almost echoes the claims of CDs when they first came out of "perfect Sound forever". The trouble is it sounded terrible. Then Julian Dunn and others started talking about jitter and after this noise.

 

 

1 hour ago, Archimago said:

 

What happens in the mind neurobiologically is of no direct concern to high-fidelity audio reproduction or to hi-fi companies even though it would be very interesting academically.

 

This is where we have the greatest disconnect but yes I understand your perspective. You are dealing with making a signal perfect to the extent that you can gauge that the signal is perfect. The trouble is we disagree as to the ability to ensure the signal is perfect without listening. It's like building a perfect aeroplane but never doing a test flight in reality, in my opinion.

 

 

 

1 hour ago, Archimago said:

 this is in the realm of perceptual enhancement rather than as an audiophile who wants the cleanest, uncolored, most direct, "shortest signal path" type of performance.

 

I'm not asking for perceptual enhancement

 

1 hour ago, Archimago said:

 I typically listen to the gear, then put it on the test bench, then have another listen afterwards to see if I can hear significant anomalies found. More often than not, I agree with the measurements and where I am surprised, it's usually with the areas where I believe it makes little difference (again, jitter is a good example, and low level harmonic distortion and TIM might be another).

 

This of course can be confirmation bias

 

1 hour ago, Archimago said:

Then there's no frustration at all because one can accept that the anomaly is there but it makes no difference because one's ears/brain/listening ability would not be able to pick it up.

 

This sounds a bit like speculation

 

 

1 hour ago, Archimago said:

In time, as one accepts these limitations of the self, then some things become less important.

 

Alternatively one can accept that "limitations of the self" means we don't have an answer for everything at this present time and there are more discoveries to come with attendant improvements in transparency. Certainly this would appear to be the way of science.

 

1 hour ago, Archimago said:

The question is, does that man have a means to prove that his rejection is not based on biased opinion and variables outside of whatever governs the topic at hand?

 

Science really isn't about proof, it's about probabilities and evidence, test and retest and setting conditions to reject the null hypothesis. I totally agree that properly controlled trials that eliminate bias and confounders and using demonstrably valid tests is the way to go. This is a huge topic but would take us off topic for this thread.

 

Archimago, for the sake of some brevity I have chopped down your responses and am aware that sometimes doing so can make it appear out of context. This was not my intention if it has occurred.

 

Cheers

David

[Audiophile Neuroscience]

32 minutes ago, STC said:

Don’t you think this falls under psychoacoustics? It has been extensively researched and still being discovered. 
 

 

Hi ST, I think there is something of a chicken and the egg argument going on. I get that you can just work on improving fidelity of the signal without any reference to a human being and to the extent that you can satisfy yourself that it is faithful. The problem is people have been told it is faithful before and yet it doesn't sound right to human beings. Now either the human beings are flawed or the stuff they're being served up is flawed. Due to listening reports, and engineers that listen, engineers have gone back to the drawing board and discovered that it wasn't really faithful at all. Thus improvements in transparency are made.

 

[Audiophile Neuroscience]

25 minutes ago, STC said:

(Hi David, nice to see active here again. )
 

Referring to the quote above, I suppose by equipment manufacturers, recording engineers and high end reviewers.

 

i don’t see why the measurements are relevant for listening music. Technically, a vinyl is inferior to digital and yet a vinyl system can sound as good as a digital system if not better to some. That itself is enough to convince me that measurements is not going to tell everything. 
 

When you say engineers, there are many out there but what are they actually measuring?  It is only somewhere in 1978 the new standard of object and subjective assessment for sound quality was conceived by taking psychoacoustics aspect into considering for sound quality evaluation. The standard is specified in the ITU.  Emotional response to music is more important when compared to objective measurements. 
 

If you were to play a jazz DSD in a dead room and compare the same in MP3 (320) in a lively room; the preferred one would be the MP3 sound. Measurement wise the DSD is better in every aspect but at emotional level the MP3 in a live room is more natural and believable. 
 

Our own judgment changes with time. As a young boy, I was mesmerized by the Poetry of the sea album. Every night after my parents were asleep, I would sneak in the hifi set to my room and place the speakers to the side of the bed and I will lie in between. It was so real sounding to me at that time. As time goes by, I have forgotten about the cassette until about 10 years ago when I saw the CD. When I played them in my system which is far superior than the player I had 45 years ago, the emotional connection wasn’t there. And like all audiophiles I too thought the analogue tape was a superior medium compared to digital. (Actually, this is also one of the thing why I abandon hirez and high end but that story is for another day).

As you can see from my experience, it wasn’t the measurements that brought out the best sound to my ears when I was a kid. It was the emotional connection of hearing the sea gulls flying from left to right, the boat engine, the crashing waves and the orchestra trigged something in me. 
 

As music lovers, measurements are not relevant. It is about making the sound of your system good so that you would appreciate the music more. Is it measurable? Yes but not with the conventional measurements we use to design equipments. 

 

p.s. Just Google to find out the correct name of the CD and was pleasantly surprised to notice that it was a Quadraphonic recording. glad that I was on the right track at that tender age.   

 

 

 

Hi ST

 

coming at this from a purely scientific point of view I don't have a problem with the theory that engineers (or other skilled professional) might be able to produce a signal that is perfectly faithful to the source.

 

As many have pointed out there is currently a poor correlation between many audio measurements and audible sound quality or sound characteristics. I think there is potential for this to change if there are people willing to look (and listen).

 

If you understand how sound and music is perceived it seems reasonable to me, armed with this information, that you might be able to find measurements to reflect this. We might even find out why some people prefer vinyl over digital or vice versa

 

 

[Audiophile Neuroscience]

4 minutes ago, kumakuma said:

31 minutes ago, fas42 said:

Until someone!!! takes the business of measuring a complete system in operation, in every possible area that may be relevant, seriously - this will go absolutely nowhere.

 

I have to say that I've never seen anyone!!! use exclamation marks in the middle of a sentence before.

 

I'm!! starting to wonder about your !! observation skills! Tom!! Is your sreen! resolving !! enough!? 🤔

[Audiophile Neuroscience]

5 hours ago, Iving said:

and an object lesson in the kind of "humility" I've mentioned once or twice.

 

I think there is an inherent humility and honesty in the simple statement "I don't know'. This is in contrast to what I see as theory served up as dogma by some subjectivists and objectivists.All one can really do is look at all the available evidence (in the case of Audio Truth that would include measurements and listening) and decide what fits best into their current paradigm.