Jump to content
IGNORED

Why are objective assessments important...


Recommended Posts

6 hours ago, Audiophile Neuroscience said:

What I think most subjectivists sceptically reject is the notion that excellent device specs translates into sonic transparency and the corollary, that an excellent set of specs means that the device will sound like any other device with the same specs.


Don’t assume that the provided measurements are “specs” for an electronic circuit, We know that, for example, THD does not entirely denote “SQ”

Custom room treatments for headphone users.

Link to comment
7 hours ago, Audiophile Neuroscience said:

In my opinion therefore we need to establish measurements of the audio signal that not just informs us how the device is performing, as important as that is, but informs us how the sound/music will be perceived, i.e. what will it sound like.

sound like a dream to me, but I am not an expert in neuroscience ....

I would have a guess, that the full set of personal factors that determine our subjective perception could only be integrated as approximative & averaged data, which in turn may give us only a divergent perception from the ones we hold ... 
You may want to correct me here, I'd be happy to learn how our lifelong individuality & experiences can be integrated in a measuring model 
Best, Tom

Link to comment

Thanks for another good post, Archimago.

 

Very happy you found the article with JGH.  He was quite the curmudgeon, but I liked him.  We have to remember that at the time he began publishing measurements were VERY primitive by today's standards (the HW and SW available to even hobbyists is astounding!).  They certainly wouldn't have been conducive to the extensive work to correlate measurements with perception that now seems at least somewhat possible (perhaps by you!). He did his best in this context, and I think did it quite well.  I agree with his references to acoustic music performed in a real place.  Of course the circle of confusion can be evoked, but he recorded extensively, knew the sound of live music and the character of the mics he used.

 

I hope you read the link to AD's article as well.  It certainly elucidates the other possibilities, potential faults that you alluded to later.....  It is certainly possible that we "lost our way."  Maybe the pendulum will swing back.  AD is interesting, he is a skilled writer.  I still read his stuff based on that (and am always on the lookout for music to seek out).  Perhaps there is a lot of "synergy" and "euphony" involved, this providing the sound he prefers.  I have a lot of thoughts about this in the context of the equipment he likes.  And the room he listens in!  I am frequently fairly disappointed with the rooms subjectivists reviewers describe....so much that can be done in this regard!

 

I agree a lot with your response to John's as well, very nice!  But also very strongly with "Neuroscience's" that followed.  Seems like there has to be a synthesis of Neuroscience (the field, not the man) and the engineering to move forward.  I think we also have to remember that the former, while strides are being made, is a bear!  So much we don't know yet!  So complex!

 

I was waxing philosophic in my thoughts last night, but my posts are already cumbersome in length so will list bullets of my thinking (my office network is down this morning....I still miss paper charts):

 

-  I am not interested anymore in ridiculously expensive equipment of any sort; if I had unlimited resources I still wouldn't purchase it.  It is not a part of my ethos. I grew up poor, lived overseas as a child, and spent 3 years in the "developing world" as a volunteer (not hyping myself in any way, just background).  I feel guilty enough sometimes with my current equipment.

 

-  I rarely read entirely subjective reviews.  Their value has decreased for me over time.  With the amount of information available to us now in so may fields that interest me I feel like I have to be selective.

 

-  I do read subjective reviews that are followed by measurements in my ongoing attempt to correlate them with perception, though frequently go introduction -> conclusions-> measurements.

 

-  Most of my audio interests for a long time don't involve the equipment discussed in depth on this forum.  I have a Mac, never learned enough about Windows to be effective (the first time I used a computer at all was age 30!) running Audirvana, to an RME DAC with the USB cable that came with it (and no interest in exploring others).  I taught myself vacuum tube design (though have only built for my son a guitar amp), have read "The Master Handbook of Acoustics," Geddes ""Audio Transducers" (mainly the waveguide stuff, the math is more up your alley!), and others.  I dream of DIY loudspeaker design, tube amplifier topology, current transmission between components, other stuff.  As to the last I built a DAC/headphone amplifier using the "Twisted Pear" Buffalo DAC from diyaudio powered by SLAs and connected the DAC chip outputs to a fancy transformer, to headphones (sort of using the voice coils as the I/V resistor) and used parametric EQ to correct the FR variations calculable from their impedance curve.  Good stuff!  Some of the best (subjective) sound I have ever experienced.

 

- I found this recent AES article timely http://www.aes.org/e-lib/browse.cfm?elib=20455, it reflecting my thoughts:  It isn't the dynamic range that increased bit-depths allows or the high-frequency extension that higher-sampling rates allows (though I wonder about some of the studies on possible other mechanisms that may provide benefit in "Neuroscience's" realm), but instead to allow different filtering options.  I like and buy high-res material (though am too often very disappointed when, from old recordings, they squash the dynamics in the re-mastering). :(

 

- I was intrigued by Ayre's "Listen" filter and was excited to learn the settings in iZotope provided by "Audinventory" (?) in the long iZotope thread on this site that would closely replicate it (with measurements).  I liked the sound, eventually blind-testing my "golden ear" son who picked them out and described it subjectively in the exact terms I had in mind (that I hadn't biased him with by sharing).  I upsample in Audirvana and apply this filter, bypassing the RME's.

 

"Many of the fights we get into originates from the "all or none", "black or white", "100%" mindset. As mature adults, we know that the only way to handle the complexities of life (of which the squabbles of audiophilia is but a tiny microcosm) is to find the middle ground... The "shade of grey" between the subjective and objective. Some things do need to be "more subjective" just as my preference is to be "more objective" when it comes to audiophile gear."

 

Amen!!!

 

Best, Bill

Labels assigned by CA members: "Cogley's ML sock-puppet," "weaponizer of psychology," "ethically-challenged," "professionally dubious," "machismo," "lover of old westerns," "shill," "expert on ducks and imposters," "Janitor in Chief," "expert in Karate," "ML fanboi or employee," "Alabama Trump supporter with an NRA decal on the windshield of his car," sycophant

Link to comment

In the 35 years I've been looking at getting subjectively convincing SQ, I have never once looked at it from the point of reducing THD, IMD, or TIM - yes, if I do my own design, then I very carefully assess what would be the best use of electronic parts to get measurably optimum numbers; but in the real world I have been startled too many times at hearing equipment that I know has very ordinary figures in this sphere perform way above what most would expect, to believe that worrying about such is going to be vital.

Link to comment
9 hours ago, jabbr said:


Don’t assume that the provided measurements are “specs” for an electronic circuit, We know that, for example, THD does not entirely denote “SQ”

Hi Jonathan,

not sure I completely follow what you mean when you say don't assume that the provided measurements are specs for an electronic circuit. At any rate my intention was to use the word specs or specifications in a broader sense, pertaining to materials, design and performance et cetera - the things that define what a product is and does.

Cheers

David

 

Sound Minds Mind Sound

 

 

Link to comment
55 minutes ago, Audiophile Neuroscience said:

Hi Jonathan,

not sure I completely follow what you mean when you say don't assume that the provided measurements are specs for an electronic circuit. At any rate my intention was to use the word specs or specifications in a broader sense, pertaining to materials, design and performance et cetera - the things that define what a product is and does.

Cheers

David

 


Davd, 

 

Well yes, a complete specification would be the electronic circuit, firmware, drivers etc, but aside from a FirstWatt amplifier, that is never available. Even then, a SPICE simulation of the circuit is not the same...

 

A measurement is not a spec in that sense. Some of these specs are more akin to marketing. Of course you may need a certain wattage amp with certain speakers, and a DSD or PCM DAC input etc.

 

Some measurements like spectra may give an idea about how the product sounds. 
 

Tell me though, do you find that “performance specs” give you an idea about how the products sounds? If so, which specs? I’d say that some common specs give a very general idea but not at the level of detail I’m interested in — I think that some of the nonlinear behaviors give a better idea of the unique product sound but these aren’t commonly measured nor published.

Custom room treatments for headphone users.

Link to comment
13 minutes ago, jabbr said:


Davd, 

 

Well yes, a complete specification would be the electronic circuit, firmware, drivers etc, but aside from a FirstWatt amplifier, that is never available. Even then, a SPICE simulation of the circuit is not the same...

 

A measurement is not a spec in that sense. Some of these specs are more akin to marketing. Of course you may need a certain wattage amp with certain speakers, and a DSD or PCM DAC input etc.

 

Some measurements like spectra may give an idea about how the product sounds. 
 

Tell me though, do you find that “performance specs” give you an idea about how the products sounds? If so, which specs? I’d say that some common specs give a very general idea but not at the level of detail I’m interested in — I think that some of the nonlinear behaviors give a better idea of the unique product sound but these aren’t commonly measured nor published.

 

This is where i maybe went wrong. I consider all measurements to be specs of some kind."The purpose of a specification is to provide a description and statement of the requirements of a product, components of a product, the capability or performance of a product, and/or the service or work to be performed to create a product."

 

Jonathan (I think) I totally agree with you regarding performance specs and we need better indicators of how products sound in a much greater level of detail than presently exists. Whether that's measuring or quantifying or somehow communicating "nonlinear behaviours" I cannot say. I would much prefer to be informed by objective data even at the end of the day if I reject it and still make my own subjective choice. At least I would have an option, a choice.🤔

Cheers

David

Sound Minds Mind Sound

 

 

Link to comment
19 hours ago, Audiophile Neuroscience said:

The relevance of measurements in audio seems to be a popular theme at the moment.

 

We know from other threads here and elsewhere that it is difficult to completely correlate objective measurements of an audio signal with subjective perceptions of what that measured signal will sound like to a human listener.

 

Measurements of the audio signal are not direct measurements of perception. They are surrogates or indirect markers. In medicine such markers are used all the time but the difference is we know how that indirect marker compares to a gold standard test of what you think you are measuring. It's just calibration of one tool against another known, and more accurate, tool.


Hmmm, let's hold off on the medical lab test analogy because I think we can make a case with examples of almost anything we want to argue about depending on which tests we're talking about! 😃

 

The way I see it, the measurements we do in audio are actually very correlated to what we hear already! Other than the rare birds like jitter and TIM which I agree can be surrogate measurements, key measured parameters like frequency response is highly correlated to sound quality of speakers and listener preferences (like the Harman research). Noise level likewise when high enough is clearly correlated to audibility and preference.

 

Let's talk about something like THD+N which commonly gets brought up. We have the luxury in 2020 to be blessed routinely with <0.01% THD+N DACs and through our vantage point say: "Oh, THD+N is meaningless and doesn't correlate with sound quality!"

 

Imagine if we lived in a time when source components went as low as 0.5% and often even went up to 5+% THD+N (still the case with low quality vinyl - anyone care to measure a Crosley turntable for example?). Back in those days, I imagine the objective number was seen as way more than a theoretical construct or questioned as to whether it made a difference! Of course it could be audible.

 

Every one of the distortions we typically use for measurements would lead to unwanted audible issues when severe enough. Even the dreaded but most likely toothless jitter boogeyman when high enough (as demonstrated) ruins the sound. To a real degree, it is thanks to technological progress that we have the luxury of subjectively expressing our preferences!

 

When devices often measured poorly say in the 40's to the early 80's, surely the focus on objective performance can be understood.

 

Quote

In my opinion, we haven't got that far in audio yet…….( this is despite blind testing. As a test tool, in my opinion it is not adequately calibrated and without known sensitivity, specificity, true and false positives/negatives, and positives and negative predictive values. Another topic).

 

I think implied in my response above is the idea that we have actually gone far enough already in many ways - especially for 2-channel audio! While there may be some things here and there to find, I think the technology is well mature. To go beyond might not be the domain of the hardware audiophile hobby as I'll discuss below.

 

Quote

Still, in my opinion there is hope. By definition, irrespective of how complex a perception is, it must have an evoking stimulus. That is basic neurobiology. I am excluding things here like hallucinations and the huge topic of central modulation in the nervous system.

 

So, the stimulus in this case is an audio signal followed by sound waves in air. The principle is pretty straightforward in that you can study that stimulus to observe what changes in the stimulus correlate, and are concordant with, some sort of change in perception.

 

I mentioned elsewhere that I don't think it should be terribly difficult to study such things as dynamic range measurement and compare that with our subjective evaluation of compressed or dynamic sound perception. Similarly, frequency response that doesn't extend down into the bottom octaves of music is expected to sound a certain way, or at least lacking a certain sound.

 

Actually, my hope for high fidelity products need not go this far. Beginning at the source, all we really have are our CDs, vinyl, and digital downloads (tapes, cylinders, etc...). All that we can ever recover or "hear" is in that data. If we can completely, transparently reproduce that "source" with a perfect turntable, perfect CD player, perfect DAC... No losses in the cabling... No losses in the preamp... "Perfect" amplifier to a perfect speaker/headphone that can "faithfully" (as in "high fidelity") reproduce what the data encoded in that source is, then that is all we can hope to do. "Transparency" to the source content is all that we can ever achieve. What happens in the mind neurobiologically is of no direct concern to high-fidelity audio reproduction or to hi-fi companies even though it would be very interesting academically.

 

To go "beyond hi-fi" would be to research psychoacoustics, which I think is what you're starting to suggest. How do we measure or even create mental effects such as "virtual 3D sound", "being in a stadium", expand perceived dynamic range, enhance subtle details, etc... All of which likely is more than what's on the CD or vinyl or downloaded file (especially 2-channel material). But as a purist "high-fidelity audiophile", this is in the realm of perceptual enhancement rather than as an audiophile who wants the cleanest, uncolored, most direct, "shortest signal path" type of performance.

 

Whereas we can objectively measure and know "fidelity" to the source recording (what goes in is exactly what comes out of the transducers and enters the ear), what constitutes as perception of the sound (neurobiologically measured so we can know that the person perceives it as "good" or even "better"), can only be defined based on individual preferences and that person's unique perceptual machinery.

 

Suppose we even have that kind of capability to peer into the mind with some detail (IMO not possible in this generation including with EEG, MEG, and neuroimaging), what do we think DAC manufacturers or speaker manufacturers can do about it?

 

Quote

It seems to me that where we're at with objective measurements is to verify  that a piece of audio gear is performing to spec and maybe explore better and alternative designs. There are measured levels of jitter, various distortions, frequency response and so on. Some of these measurements can give us some insight into what the gear might sound like, how it is "voiced". But frustratingly to most audiophiles there is often a disconnect between the measurement of the device's performance and how it sounds.

 

To be honest, I don't feel this way at all! I typically listen to the gear, then put it on the test bench, then have another listen afterwards to see if I can hear significant anomalies found. More often than not, I agree with the measurements and where I am surprised, it's usually with the areas where I believe it makes little difference (again, jitter is a good example, and low level harmonic distortion and TIM might be another).

 

Like I said above, performance of gear is already very good and if we have a concept of the threshold of our own hearing limitations, the magnitude of the anomaly on the measurements will make sense. Then there's no frustration at all because one can accept that the anomaly is there but it makes no difference because one's ears/brain/listening ability would not be able to pick it up.

 

In time, as one accepts these limitations of the self, then some things become less important. People still ask me all the time if I prefer USB or SPDIF. To me it doesn't matter any more because the only difference unless there are bit errors or audible noise is jitter and even though TosLink has worse jitter, I believe it's inaudible. A person like Ted Smith can come along and claim his DAC sounds different because of "jitter"; well, unless he can show me evidence that his DAC has good jitter performance (which it does not), then it's probably fair to suggest he's wrong.

 

Does it matter if my Emotiva amp has -80dB THD+N compared to my Hypex with less than -90dB? No, because both Class AB and D amps are very good now, that's not what matters most anymore... Rather the difference in damping factor to control my speakers, the lower cost and the fact that the Class D uses much less electricity are objective parameters that matter to me now within normal listening levels. In terms of anomalies found, does it matter that the Hypex has a 400kHz switching noise? No, because I know from experience that I cannot hear it and there are easy ways to ameliorate the effect in the system...

 

This IMO is the "offer" and power of having an objective perspective - this is an opportunities to experience, develop awareness, and understanding for oneself. It's simply part of the hobby. Subjectivists IMO should engage with the objective side because they will learn something about audio gear and themselves; in the process I suspect many will not be as influenced by peer pressure 😉 of what they're "supposed" to hear or like. Likewise, objectivists should recognize that our perceptual abilities are unique (personal, subjective) and embrace the emotional side including pride of ownership and luxury as the case might be.

 

Quote

The implicit suggestion/understanding/belief that the better the device's measured performance the better the potential sound quality has strong face validity but it is not always borne out in reality. It probably holds more true in the converse, in that lousy measurements are probably going to translate into poor sound quality but even this does not hold completely true.

 

I agree this is not true that perfect measurements mean "better sound quality" to everyone. And this is because we can respect people's subjective opinions. There are those among us who love vinyl sound and will only partake of a purely analogue/vinyl feast 😀. Are they "wrong"? Of course not. But this doesn't mean that they are not listening to turntables with temporal distortions way "worse" than a few hundred picoseconds of jitter off the CD player. Or that their systems don't have relatively high noise levels and >1% THD. If a person considers this subjectively "better sound", so be it... No need to engage in a crusade.

 

This is why to me it doesn't matter what neurobiology is saying when it comes to the hardware goal. For some, what I might call distortion is heard as being euphonic. I don't mind knowing about it or even modelling the sound quality to see if I might enjoy the coloration... These preferences are idiosyncratic in nature and not something that we need to chase. Some like beer, others wine, others whiskey. But if ya wanna get intoxicated, make sure the EtOH content is objectively adequate :-). It all comes down to what we're aiming for.

 

Quote

In my opinion therefore we need to establish measurements of the audio signal that not just informs us how the device is performing, as important as that is, but informs us how the sound/music will be perceived, i.e. what will it sound like.

 

As above. I think that's important for neuroscience, not "hi-fi" audio engineering as per the hobby I believe I'm engaging in for the hardware sound reproduction side. Maybe one day when we can plug our brainstems into the system and hit the vestibulocochlear nerve, cochlear nuclei, inferior colliculi, or maybe directly to Heschl's gyrus, then we better know what's happening with neuronal firing patterns!

 

"How the sound/music will be perceived" IMO is not the job of this hobby but yours, the neuroscientist 😉.

 

Quote

What I think most subjectivists sceptically reject is the notion that excellent device specs translates into sonic transparency and the corollary, that an excellent set of specs means that the device will sound like any other device with the same specs. I don't think it is an unobtainable goal just think we're way off reaching it, in my opinion.

 

A man can reject all kinds of notions. The question is, does that man have a means to prove that his rejection is not based on biased opinion and variables outside of whatever governs the topic at hand?

 

Suppose we have 2 amps to listen to. A subjectivist might reject that these two well-designed, well-measured amplifiers "sound the same" within normal operating parameters. I can listen for myself and walk away disagreeing with him. Since it is only his opinion, unless he shows me that this is indeed the case with "honesty controls" (ie. controlled trials, blind testing), otherwise with what faith am I to have that his opinion reflects reality?

 

Archimago's Musings: A "more objective" take for the Rational Audiophile.

Beyond mere fidelity, into immersion and realism.

:nomqa: R.I.P. MQA 2014-2023: Hyped product thanks to uneducated, uncritical advocates & captured press.

 

 

Link to comment
50 minutes ago, jabbr said:


Yes absolutely. I believe in a combination of understanding the electronic circuit, measuring the circuit to ensure that it is behaving as intended, as well as listening!

 

This is the crux of the question: which measurements predict great sound?

 

Hard to say but if a product claims low jitter, then let’s see the measurements. If a power cord or supply supposedly improves a DAC then let’s see the measurement! If a product claims to block leakage current then let’s see the effect n a DAC!

Totally agree. My only qualification would be obviously provided that said measurement exists...and exactly as you say, which measurements predict great sound?

Sound Minds Mind Sound

 

 

Link to comment
21 hours ago, Audiophile Neuroscience said:

In my opinion therefore we need to establish measurements of the audio signal that not just informs us how the device is performing, as important as that is, but informs us how the sound/music will be perceived, i.e. what will it sound like.


Don’t you think this falls under psychoacoustics? It has been extensively researched and still being discovered. 
 

Blind tests are useful to determine if the difference can be detected by human hearing. If not than whatever specs becomes unnecessary. 
 

I don’t think there was any dispute whether objective or subjective 5.1 is perceived to be more pleasurable and real than 2.0.  What kind of measurements can tell this?  

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...