Misleading Measurements

[Audiophile Neuroscience]

Okay, I see a few posts have been deleted as presumably OT. If understanding, the premise is that of a double standard - that objectivists might be publishing misleading measurements deemed to be inaudible (below differential, absolute thresholds whatever) while criticizing subjectivists for pondering the possibility of hearing such things.

 

I would say it is only misleading/disingenuous/ of a double standard if those measurements are said to be in some way relevant to performance.One would expect that, if they believed in what they say elsewhere, they should point out that whatever measurement is irrelevant.

 

The question still remains then, why publish measurements of "inaudibility" in the first place. Being objective (but not an objectivist) I would simply say you do the test/s and the results fall where they fall. It is objectively helpful to know that any flaws revealed might be (allegedly are) inaudible.

 

 

[Audiophile Neuroscience]

 

2 hours ago, kumakuma said:

 

The tendency of subjectivists to reduce objectivists to a caricature is more in tune with the former philosophy rather than the latter.

 

Caricatures of objectivists ? I dunno about anyone else but i see them as funny little bears holding cups !

[Audiophile Neuroscience]

Trust my measurements and trust my ears can both mislead -

IMO the problem in the case of measurements, presuming they are accurate, is in their interpretation. Double standards may come into play if *some* people manipulate that interpretation to serve their agenda or belief system.

[Audiophile Neuroscience]

1 hour ago, John Dyson said:

A simplistic interpretation of an objective measurement can be very misleading, and bias the evaluation.   Likewise, depending on ones hearing for comparisons, especially when *accurate* aural memory is pretty short, can also be misleading and bias the evaluation.  This is why subjective measurements should be done with proper scientific/statistical methods.  Likewise, comparing measurements without knowing & controlling the conditions will likely be misleading.  (Even certain OBJECTIVE measurements might require experimental/statistical discipline.)

 

It seems in the audio realm -- much of the misunderstanding/disagreement comes from the *understandable* lack of knowledge about the math, the electronics to the physics level, lack of understanding of human hearing, understanding the value of statistics -- and lack of realization of personal variabilities in hearing.   Most importantly, not respecting the how/why to control experiments.   As a collection, being such a wide expert is NOT easy, and this is NOT an undergraduate subject -- this is a field that requires understanding, study and lack of bia$$$.  Otherwise, erroneous info WILL be presented.   * (I am not claiming to be such an expert -- just throwing in some REALISM about the variables we are dealing with.)   Usually, in a commercial situation, there is a group of people that MUTUALLY provide the needed skills and checks and balances.

 

One lifesaving discipline for 'us little people' who have limited resources and perhaps not enough knowledge in all of the fields:  *controlled experiments* with proper application of simple statistics.

 

* Knowing and believing  the potential mistakes about comparisons and measurements would make claims a lot more measured.

 

'My opinion' is a good thing.   Or measured suggestions for improvements, those are also good things.  There are LOTS of variables in the real world -- and they sometimes scare me away from participating unless I know exactly the design/environment that we are talking about.

 

John

 

 

I agree John. I think to some degree we all may have difficulty seeing beyond our own conditioned myopia or way of looking at things......

 

[Audiophile Neuroscience]

I could weigh in on audibility thresholds, differences for absolute vs differential and interesting results for  non periodic steady-state noise but as I see it, what the thresholds actually are is not the point.

 

The point is *if* someone declares a threshold for inaudibility (whether they be right or wrong), then uses it to further their opinion on another topic, but then elsewhere appears to use the very same data in a contradictory manner. A double standard is said to exist or at least is implied.

[Audiophile Neuroscience]

3 hours ago, pkane2001 said:

I see that Alex is getting upset that we are veering off topic, so I'll stop here

 

 

I seem to recall it was about

 

On 6/29/2020 at 10:42 AM, The Computer Audiophile said:

I would love to know why objective people are interested in anything below the threshold of human hearing and why they are interested in this info given that it can cause the same issues they rail against with respect to subjective opinions. 
 

Perhaps I should ask @Archimago why he publishes measurements below the threshold of human hearing. 

 

[Audiophile Neuroscience]

12 hours ago, pkane2001 said:

12 hours ago, The Computer Audiophile said:

I hear what you're saying, but a DAC that measures below the threshold of human hearing must also be in that "should just work" category correct?

 

Correct. But there are many more opportunities to mess up a DAC than a USB cable.

 

Hold the phone, is not the number of opportunities to mess up irrelevant? If in fact the DUT measures below the threshold of human hearing it "should just work" in that range below the threshold of human hearing, isn't that the assertion? So why report it is the question being asked. Things like price, aesthetics, build quality, and performance in the audible range are independent factors that people can use to guide purchase.

[Audiophile Neuroscience]

29 minutes ago, pkane2001 said:

Already answered all of that.

 

Sorry, I missed this bit....

18 hours ago, pkane2001 said:

 

 As long as the consumers are aware that there's no audible difference, they can concentrate on other qualities that matter to them.

 

Hi Paul and yes I do agree. I already said something similar too 😜

 

The crucial part, as you say, is that the (allegedly) inaudible part not be represented as a better performance marker. So, If the measurement is offered and not specifically stated as irrelevant it may well be misleading to those less technically inclined readers. Presumably @Archimago provides this interpretation in those cases.

 

In either case it would be nice to hear from @Archimago as he was specifically mentioned by the OP.

[Audiophile Neuroscience]

6 hours ago, John Dyson said:

A few tweaks are okay, but tweak tweak tweak isn't instructive.  Studying a bit of technical background and learning why the attempt at 'design' requires so many tweaks, is MUCH MUCH more important.

 

Tweaking doesn't create learning -- ask Mr Edison about that.  I doubt that 100yrs of Mr Edison tweaking would have created Tesla's new ideas.   Tweaking is an intuititve physical activity -- it is only an adjunct to the more important learning.   Or, most ideally -- a finishing touch.

 

John

 

 

I think you may be being a bit bold about this John ! 🤣🙄 ^^^

[Audiophile Neuroscience]

6 hours ago, sandyk said:

6 hours ago, pkane2001 said:

Because I tested one [Lush cable].

Other than a Subjective check  , are you able to post some of your measurements, including it's measured impedance, which is possibly the most critical for a high quality USB cable , which could illustrate why so many members favour this cable over most generic USB cables ?

 

Being somewhat OT, @pkane2001could these measurements be misleading?

[Audiophile Neuroscience]

8 minutes ago, semente said:

They've been Tooled to accept

 

Yes perhaps they've been Tooled in the wrong tool box.

[Audiophile Neuroscience]

10 hours ago, The Computer Audiophile said:

Measurements are misleading just like statistics.

 

10 hours ago, semente said:

 

This particular case is not so much that measurements are misleading but whether a couple of measurements are enough to characterise audible performance; I defend that they're not.

 

 

9 hours ago, Summit said:

Measurements and statistics are not misleading per se, therefore better don't throw out the baby with the bathwater.

 

 

 

 

Measurements tell you no more or less than the test tool and its human operator are capable of. Errors can occur at both these levels. If entirely accurate the measure can only tell you what the particular tool is designed to look for within its limitations of sensitivity, specificity etc.

 

If you don't know how good the tool is that is used to make the measurement you don't know how good the measurement is.IOW you don't know how the tool measures up.

 

An absence of an accurate and meaningful measurement can sometimes just mean that a different tool (or operator) is required.

 

The meaning of the measurement is open to interpretation and thus subjective conclusion and open to errors whether intentional or not.

 

For all these reasons IMHO one must be skeptical of measurements, acknowledge they can indeed be misleading or meaningless, and not slavishly and blindly accept measurements over observation.

 

 

 

 

[Audiophile Neuroscience]

7 hours ago, The Computer Audiophile said:

Why do you want to see measurements that make no difference to any human ears?

 

Simply because I do not trust the 'measurement guy' to tell me what is and is not audible, or otherwise in some way influence audibility. It is logically apparent that "measurements that make no difference" are meaningless and may well be misleading. It is a double edged sword tho because i want to see all the data and not have someone tell me what is relevant or not because they say so.

[Audiophile Neuroscience]

9 hours ago, pkane2001 said:

If I ever need a stent, I'll spend my time to educate myself on what matters and interpret "facts" and measurements published by a manufacturer rather than just trust someone else to tell me

 

I would suggest it is difficult to interpret measurements published from a source with vested interests. For similar reasons I have issues trusting measurements or studies published by audio manufacturers and that includes Harman, irrespective of the lead name on the paper.

[Audiophile Neuroscience]

5 hours ago, Jud said:

Is anyone aware of papers measuring hearing thresholds that use something like an fMRI rather than conscious verbal responses? Experimenters do take account of the difference between tests that use instrumental measurements and those that require a conscious verbal response, but I don't recall seeing academic papers on this topic in the area of human hearing.

A couple of more recent papers but probably off topic for discussion attached here

 

2020 Assessment of temporal resolution of bone-conducted ultrasonic hearing using neuromagnetic measurements

2017 High-Resolution Audio with Inaudible High-Frequency Components Induces a Relaxed Attentional State without Conscious Awareness

 

 

2020 Assessment of temporal resolution of bone-conducted ultrasonic hearing using neuromagnetic measurements.pdf2017 High-Resolution_Audio_with_Inaudible_High-Frequenc.pdf

[Audiophile Neuroscience]

1 hour ago, PeterSt said:

But of course a better measuring product is better engineered etc. There are no coincidences here.

 

A better measuring product measures better with a particular tool and FWIW. That may or may not mean it is "better" in whatever way depending on what the goal is and how validly the measure reflects that goal ...if at all.

 

53 minutes ago, PeterSt said:

 

The point of my post from yesterday was that there is virtually no threshold of audibility. You (and millions more) seem to think this threshold exists.

 

I think there are obviously auditory perceptual thresholds and they are at least potentially measurable with the right tools and the right method. I think the "millions" you refer to Peter are those individuals who mistake results of test tone audiometry for perception of complex and dynamic musical passages. They mistake acuity at one level to complex perception at other levels and/or alternative ways of perceiving. 

 

 

Quote

- Improve your listening skills;

- Improve your system where needed (but it is my estimate this will be the least issue for most);

- get the proper tools to vary the results (this may impede your imagination). Now iterate to the first step.

 

+1

[Audiophile Neuroscience]

22 minutes ago, semeniub said:

I would like to propose a different position on the relevance of measurements, from a sound and vibration engineering, rather than EE perspective.

 

The last items in the audio chain before your ears are either the loudspeaker or headphone drivers. The measurement that I am then most interested in is the displacement and phase of the driver surface, as this is what is radiating sound. I can measure this very precisely using a laser vibrometer, at single points, or map over the entire surface of the driver.

 

The proposal then is if I have a reasonably accurate system and keep it constant, I can change out components such as DACs which differentiate themselves via “superior” EE measurement parameters, and see if there is any measurable difference in terms of driver displacements. The premise then is if you have not changed the driver displacement, you have not introduced a significant audible change with the introduction of the superior EE measuring component in the system.

 

I realize that this would be a new way of looking at the problem, and not everyone has access to laser vibrometer measurement facilities, but at least this would be a type of measurement parameter that would be guaranteed to have an effect on audibility. Maybe someone with the right connections to a lab could have a system set up there where 5 different DACs, having low to high SINAD numbers, could be tested as a feasibility study?

Nice. At least we are moving downstream, from the output of the DAC (comparing signals with signals) to the output of the sound wave transducer.

One day we should be able to use a vibrometer at the tympanic membrane level, even further downstream. One day it may be even possible to tune into the cochlear transduced signal. Oh wait, that's what the brain is for ...😁🤣

 

Just kidding, sounds like a very cool idea. The caveat would be the same as other measurements - how valid is the result (is it sensitive, is it reliable etc)

 

 

[Audiophile Neuroscience]

1 minute ago, semeniub said:

This concept just has to be tested out in a small feasibility study first. Some colleagues have used laser vibrometers to measure nano scale vibrations in materials, does that sound sensitive enough? I'll check with them sometime to see if they have any suggestions on how this could be done.

Intriguing. I suppose you'd attempt some kind of Fourier transform of component frequencies? and somehow try and correlate changes in vibrations or frequencies with perceived listening experiences. At the end of the day it would come down to how well you could analyze the vibrations to meaningful spectral or timing or spatial characteristics that match/predict perception. Could we measure changes for example that correlate with soundstage. Fascinating stuff

[Audiophile Neuroscience]

3 minutes ago, PeterSt said:

 

I am pretty sure that would not work. Showing differences, however, will be easy.

So it can't be qualified.

If you can demonstrate clear differences you can establish independent and dependent variables for a research hypothesis.Its a start.

[Audiophile Neuroscience]

4 minutes ago, PeterSt said:

 

Not even that, unless you fall back to test signals. So do notice that I was not talking about test signals with the sines and the squarish mess etc. - it is to be about real music, e.g. containing a nice long organ pipe. So it is the lot which changes a particular "instrument" / sound.

If such a thing would allow measurement from a single test signal (or 10 or 50 (IMD)) it would be easy. But it does not work like that at all.

 

So you have, for example, a choir in a church loudly singing and let's say that the baritones do not run into your woofers. What I say is that the quality of your woofer's output (implied by the organ) is influenced in the electronic's path by the other frequencies. Change a few XXHighEnd settings et voilá.  Or change to a different USB cable. Or change the configuration of it (oops). Change the computer processor's frequency.

Everything matters. In this example it could be the impedance response of the power amp which is influenced by higher frequencies (and the higher frequencies may change by whatever the change you applied), now allowing for better (or worse) bass control. ... Like I said earlier, test your imagination on what all could happen in order to explain things (which is thus what I always try).

I never said it would be easy 😁

[Audiophile Neuroscience]

39 minutes ago, semente said:

 

I am not sure what you mean but I agree with @PeterSt that "a better measuring product is better engineered", as long as you are considering a product that performs better across a comprehensive set of measurements.

There is no point in having a time-coherent speaker if the response curve is completely ragged, the dispersion is all over the place and nasty driver break up resonances are very audible...

Hi Ricardo, I simply mean that (in your words) " very low resolution unfit-for-purpose" measurements are meaningless = do not mean better engineering (or anything else). So it comes down to the quality, validity and interpretation of the measurement/s.

[Audiophile Neuroscience]

7 hours ago, Bill Brown said:

just let them have their fun. 

 

But Bill, they need to be saved from themselves !🤣🙄

[Audiophile Neuroscience]

12 hours ago, The Computer Audiophile said:

If you’re going to argue for things that don’t matter, why stop anywhere. Let’s include the color of the DAC in the measurements. It has a much to do with the sound quality as a measurement that doesn’t impact the DAC’s performance to human ears. 

 

Yellow and Red Dacs have a warmer tone, blue and white a cooler sound coloration. I think this is the main reason Toole advocated blind testing😄

[Audiophile Neuroscience]

How is WOW measured i wonder ?

[Audiophile Neuroscience]

1 minute ago, Jud said:

Just about the same way as flutter, but slower.