Misleading Measurements

[Summit]

First of all not all objectivists or subjectivist are the same. I know objectivists that strongly believe that equipment that measures better also will sound better. Objectivists which truly believes that every significant aspect can be measured, and that people which prefer inferior measuring gear does it solely because “they like a bit of coloration”. The correlation between measured noise and actual SQ is very strong according to them. We of course have other types of objectivists like the naysayers and anti-audiophiles, but I believe those are small subgroup, even if they tend to be quite vocal.

 

I don’t get why presenting measurements "below level of audibility" would be consider misleading. To do the opposite and not show how a gear actually measured OTOH would be far more misleading and confusing IMO.  

[Summit]

27 minutes ago, The Computer Audiophile said:

Great point. 
 

If, as objective leaning people believe and according to studies, there is something below the level of audibility, what is the benefit to consumers to know this information? What is the possible harm to consumers?

 

Not all objectivists believe that. Many objectivists and subjectivist know that noise, jitter etc despite not being directly audible can have negative affect on other electrical devices and other audio gear downstream.

 

What’s the point to manipulate the measurements, it is literally like open a can of worms?

[Summit]

Threshold of human hearing is only significant if actually measured after the transducers.

[Summit]

Measurements and statistics are not misleading per se, therefore better don't throw out the baby with the bathwater.

 

Measurements and statistics can of course be presented in an attempt to mislead, but that’s another thing, don’t you think?

 

I get that you think that measurements that can't be heard by humans only serve to mislead consumers. Though many audiophiles believes that Signal integrity, Jitter, Phase noise, Leakage current, Rise and fall time, Signal reflections, Electrical conductivity, Resistance and many other characteristics is of great importance and like to know how they measures.

[Summit]

I believe that a better measuring product is not inevitably a better engineered product. First we have to establish which type of design, circuits and products that sound good and truthful. After we have establish that we can see if we can find some measurements (and amount) that correlate with good and truthful sound. Measurements are always about the amount of something and never about the quality. Because we can only measure the quantity of different aspects, it’s difficult and almost impossible to establish which of all those parameters that generally matters and if these parameters are depended or independent of circuit design.   

 

It is well known that common-mode rejection is higher in a differential balanced circuit and transmission than in a SE. The differential balanced circuit will always measure better (everything else held equal). Still not all audiophiles consider that balanced gear sound better, which in turn means that we have not really establish that lower common-mode noise ALWAYS is better from a sound good and truthful perspective.

 

Negative feedback is widely used in electronics because many believe it to improve the linearity, frequency response etc. Apparatus with a lot of Global Negative feedback are measuring better than apparatus that don’t use any (zero) Global Negative feedback. Like in the example above there is no conscious that apparatus that use Global Negative feedback even if they have clearly lower THD sound better that than apparatus with Zero Global Negative feedback.

 

My point is that as long as we don’t know and have establish which parameters and measurements that really matters in different type of circuits and keep lumping them together like they all are the same, and independent of type or tech design we will never be able to use measurements in a meaningful way.

[Summit]

10 hours ago, Audiophile Neuroscience said:

 

 

 

 

Measurements tell you no more or less than the test tool and its human operator are capable of. Errors can occur at both these levels. If entirely accurate the measure can only tell you what the particular tool is designed to look for within its limitations of sensitivity, specificity etc.

 

If you don't know how good the tool is that is used to make the measurement you don't know how good the measurement is.IOW you don't know how the tool measures up.

 

An absence of an accurate and meaningful measurement can sometimes just mean that a different tool (or operator) is required.

 

The meaning of the measurement is open to interpretation and thus subjective conclusion and open to errors whether intentional or not.

 

For all these reasons IMHO one must be skeptical of measurements, acknowledge they can indeed be misleading or meaningless, and not slavishly and blindly accept measurements over observation.

 

 

 

 

 

How good the measuring tool is of course important. A more common problem is related to how the devices has been set up - shit behind the levers and the method used to measure.

 

Measurements are data that have to be interpreted and interpretation depends on theories and knowledge and can be open to bias. Measurements, if conducted correctly, are however not misleading per se.

 

I agree with you in not blindly accept measurements as being more reliable than the observations of many many thousands of people.

[Summit]

2 hours ago, The Computer Audiophile said:

Serious question. 
 

DAC ABC has a noise floor of -150 dB

DAC XYZ has a noise floor of -140 dB

 

Will the average consumer be mislead into thinking something about these two DACs that the measurements don’t say?

 

I respect whatever opinion you have. 

 

I would not be surprised if some consumers actually would believe that the difference in noise floor from 140 to 150 dB is of importance. I would personally not put any value in just this difference. Especially if we are measuring DACs of different type/design. I don’t think it’s a straight correlation between very low noise floor or low THD and excellent sound quality. I consider some other measurement aspects to correlate better to how I actually previse difference in SQ.  

 

Presenting measurements that is below what I/we normally consider to be of importance for SQ, like a very low noise floor, is however not by itself misleading. It just show how one (of many) parameters actually measures.

 

It is also important to remember that I/we don’t know all the mechanisms and things that affect SQ and better not close any doors or minds.

 

[Summit]

32 minutes ago, pkane2001 said:

 

There's a reason for that, and we both know it.

 

 

Why yes, I've looked at audiophile switch measurements, at least those I could find. And found them to be well below audibility, in fact, below any measurement capability. This is as I'd have expected based on my understanding of networks. I've done a lot of testing and measurements in my own system, using different routers, switches, wi-fi networks, usb converters, optical networks, etc. I found no measurable differences (not just audible differences) that could be attributed to network equipment. 

 

As explained before, I prefer well-engineered products that improve transparency, even when I can't hear it. Audiophile switches do nothing to improve transparency of the system, cost a lot more, some generate a lot of heat and lower network speed. So, explain to me why I should be looking at one again? None of this sounds like good engineering.

 

And yes, measurements helped confirm all of this, so I'm still in favor of having them published for everyone to see and not hidden, as you seem to want, because someone might get the wrong impression. At the very least, the measurements confirm that the switch isn't causing harm to audio and that's already useful information.

 

Paul I’m curious, which source, DAC, amp and speaker do you have?  

[Summit]

15 hours ago, fas42 said:

 

Which tells me a thing or two ... 😁.

 

First thing I would do if I visited you to listen, seriously, to your "best rig", would be to go throughout your home, and pull the plug out on every single item that wasn't actually needed to make that primary rig work - and see what that gave me ... that would give me a baseline for assessing what the potential in SQ was, 🙂.

 

It’s not a bad advice Frank. We tested this in in my local HIFI shop many years ago and the difference was palpable. We also moved out all other speaker from the listening room and that change was even bigger.

[Summit]

22 hours ago, pkane2001 said:

This forum isn’t big enough for me to describe all my equipment

 

Three systems (one is 5.1) plus a ton of DACs audio interfaces, headamps, amps, headphones.
 

WiFi, Ethernet and USB are all in the digital audio path between the three floors. No PC in the main listening room.

 

You don’t need to name them all by model just your favorite source, DAC, amp and speaker.

 

Favorite = best measuring (ok or sounding).

[Summit]

13 hours ago, Audiophile Neuroscience said:

 

Yellow and Red Dacs have a warmer tone, blue and white a cooler sound coloration. I think this is the main reason Toole advocated blind testing😄

 

You would be surprised how much different colors actually affect us . Seriously much more than many would think. Many studies have been conducted on this topic, to get the best suited color for a creative workplace, best color to get people to relax in hospital and many other places. I prefer a dark room with only smooth spotlights on the wall behind the speakers. Also no TV or other obstacles between the speakers.

[Summit]

10 hours ago, Audiophile Neuroscience said:

 

Was it @Summit or @semente or someone else who brought this up earlier. i didn't read the full article but the previous poster referred to the propensity to initially choose a strong quality like the increased sweetness of Pepsi is often overturned by longer exposure to the product. I think the suggestion was that even well conducted short burst ABX tests may be misleading.

 

OT

Yes in a test there we are listening only for a few seconds to one product and they quickly change to another product for a few seconds, before changing again and again it’s common to choose the ones which have a sound that “stands out”. In this type of tests we are completely focused on noticing the difference between two products instead of the sound quality, which can’t be evaluated and decided after just a few seconds. This favouring of a type of sound that stands out is not very evident in the beginning of a test, but steadily increase by the amount of switching back and forward rapidly.

 

Related to this type of test method is the problem called forced choice bias. Forced choice means that we are forced to pick one of two even if we can't hear any difference between them, or worse don’t feel comfortable that the difference we hear also is desirable.

  

A well conducted short ABX test is good for distinguishing if there are any audible difference between two products or not. Or if we can hear a difference between sample A and B and many other things that can be easily distinguished.

 

It’s not very hard to identify Coke from Pepsi one time or two, but after 7 sips of Pepsi and 7 sips of Coke it’s much harder to distinguishing them. In food and beverage test we are not forced to do that 10 times, but in audio.

[Summit]

3 hours ago, manisandher said:

 

Hey Jud, when I conducted the ABX test with Mans, each sample lasted for about 15 seconds. There was then a 5-10 second delay before the next sample started. The whole 10x ABX took around 15 minutes to undertake.

 

 

In the ABX, I was primarily picking up an ever-so-slight edginess in the piano in one and not in the other sample (both bit-identical to each other).

 

 

By the end of the ABX, I was totally exhausted (having conducted a couple of non-ABX tests beforehand too). The experience was absolutely nothing like listening to music for pleasure. I suspect this is the main reason why ABX tests often 'fail'.

 

I'm reading, and very much enjoying, 'The Master and His Emissary' by Iain McGilchrist at the moment. Maybe an ABX test is more of a left-brain activity, whereas listening to music for enjoyment is more of a right-brain activity? Not easy to reconcile the two.

 

Mani.

 

What sounds different or to recognize spacial patterns (in a familiar system) is the easy part and can be tested by discrimination tests like you did. What “sounds right” is also relatively easy if we are to choose between a play back system and real musicians in the flesh playing live.

 

To determine which of two product, in a whole audio system, which sounds most correct is much harder because the audio system and recordings can never sound exactly like real musicians on stage. We are therefor in reality to determine, not which product that sound like real musicians on stage, but which that sounds closest to how it should sound, or more exact which gear that fool us best.

 

Few things are more difficult to establish as if one gear in an audio system actually “sounds right” or not. To determine if one specific part of a chain sounds right is next to impossible, because how do we know that the rest of the audio chain “sounds right” and the diversion from “sounds right” is to be blamed on the gear we are examining and not any other gears?

 

The main problem (as I see it) is that we have no real reference system in which we can compare gear. What we have is many thousands of audio system in as many homes that in practice are used as reference systems. Many systems also seems to be in a constant state of change.

 

Measurements remove many of the problems associated to lack of universal reference system, different in preference, the quality of the recordings etc. The only problem is that the correlation between measurements and what audiophiles contemplate “sounds right” is weak for high end gear.  

[Summit]

9 hours ago, sandyk said:

 

I doubt that Mani needed measurements to tell him that something wasn't quite right.

 

He probably didn't need it to observe that something was odd, but with measurement he could clearly point out to others that something wasn't quite right and that it probably was the zero crossing glitch.

[Summit]

18 hours ago, The Computer Audiophile said:

Getting back on topic.

 

The underlying physical reality of the ear/brain is that we can’t pick and choose what we can hear. We can however pick and choose which unhearable measurements we like to champion and which ones we write off as snake oil. That’s what interests me. 

 

I believe that it paramount to understand that the human hearing is not liner which means that we can detect some imperfections very easy while not others. The other thing that is equally important to understand is that measurements taken at the output of a DAC or amp is never directly audible and the signal from the DAC and amp has to be converted to sound waves before we can hear it.

 

If for example we are talking about audio noise measurement and the amount of THD or distortion that can be heard taken after the speakers/HP, yes the ability of human hearing is known and set the limits for what can be heard. If we are talking about the amount of THD or distortion taken on the output of a DAC or amp, the ability of human hearing doesn’t set any audible limits since that signal has to be converted to sound vibrations that travel through the air and that can be heard first when they reach a person's ear.

 

Many measurements are measuring (e.g. quantifying) aspects that we can’t hear directly like jitter, phase noise, RFI and EMI. That doesn’t mean that we can’t hear their negative effects on other electrical circuits/devices. This type of noise are measured for this reason, which is that they have a negative effect on sensitive electronic circuits. The correlation between measurements taken at speaker/HP is much stronger than measurement taken at electronic devices because of that.  

[Summit]

5 hours ago, pkane2001 said:

 

A DAC doesn't have soundstage height, width and depth.

 

I believe it’s safe to say that different DACs is not equally good at presenting a realistic sound-stage just as they differ in other SQ characteristics.

 

If lower THD or distortion would actually correlate to bigger 3D sound-stage people would not choose tube gear for that exact reason.

[Summit]

22 hours ago, pkane2001 said:

As discussed, it is not THD but interchannel differences that determine soundstage “quality”. These are caused by differing levels and amounts/types of distortion between channels.

 

All “things” has to be very good to record and reproduce a realistic 3D sound-stage and measurements that correlate with a realistic 3D sound-stage is not simply the level of inter-channel differences or THD. In fact a big and flat left right sound-stage is easy to record and reproduce while am talking about a realistic 3D sound-stage with good image.

[Summit]

1 hour ago, pkane2001 said:

“Everything matters” is a possibility, but the question is why?  
 

Left to right position is defined by differences between channels.

 

Depth is detected primarily through  reverb. It’s possible that some distortions will destroy very low level reverb that our ears may find useful, but that should be measurable, as these will affect any low level signal, not just reverb. The question is then, at what level can we still hear reverb, and at what level does it still help the brain determine distance?

 

I didn’t mean to say that “everything matters”. The opposite. I do not think that low distortions and THD correlate with a realistic 3D sound-stage. If low distortions and THD really would show a relationship tube gear wouldn’t be selected for their big 3D and lifelike sound-stage.

 

Reverb is made at the site and recorded. It is not more difficult to reproduce than any other SQ aspect. Some distortions can create an illusion of more reverb than that is recorded thou.

 

I don’t believe for a second that the minimal measured inter-channel differences in modern DACs and amps make much difference when it comes to left and right positions. Vinyl measure much much worse and still can do left and right and 3D sound-stage as good as a lot of digital gear.

 

Many people use sound-stage as an example for characteristics there measurements of electronics won’t tell you anything of value. Recordings, speakers and room acoustic is another thing and measurement of them matters a lot more. 

[Summit]

7 hours ago, fas42 said:

All that really matters in this whole conversation is whether there is some mysterious, "third force", that adds content to what one hears when listening to a musical instrument - or not. I say, the instrument, and possibly objects which are not air around it are the sole contributors - which means that a decent microphone will pick up purely what matters, which is the sound generated by the playing of the instrument, in the particular environment. Replay of the capture, to an adequate accuracy, will not be additive - because it can't be ...

 

I see @bluesman as having an agenda, to ascribe the veiling of conventional playback to something which can't be dealt with - IME this is completely incorrect; suitably '"debugged" reproduction doesn't have veiling issues - the problem simply evaporates, just like a morning mist 😉.

 

It’s like UFOs 😄. Many observations, many investigations, many theories but in the end the vast majority of UFOs can be identified as ordinary objects or phenomena.

[Summit]

2 hours ago, Patate91 said:

 

It seems that you are misusing sceptism and science method for an unknown reason.

 

You can simply use and verify the app to see if it works.

 

Maybe you don't know how a plane works and can fly. You don't have access to all the data and how everything works. But still those planes are flying around the world.

 

I don’t know about airplanes but we don’t like UFOs flying around the world without validated independent testing that they do exactly what they were designed to do.

 

The ministry of extraterrestrial spacecraft has a form (you got to love bureaucracy) for obtaining independent testing, but so far none on an unidentified spacecraft has been able to verify their identity, which is required to submit the application (a classic catch 22 situation).

[Summit]

1 hour ago, Patate91 said:

 

You'll have to learn how to code an application to understand how it works, this is out of scope of the thread and "irrevelant" for now. 

 

Like it has been said earlier the app has already been "validated" and verified . Unless you have the expertise too objective data about how it bas been code will be "useless" for this goal of this thread.

 

And sure I'll change my position whenever a rationnals arguments will be provided. It includes things that I can think of in the future.

 

So for OP, objective data are always welcome.

 

 

 

Welcome to audiophile style! I think it’s a great capability to be able to consider other perspectives and to change position if sensible evidences are provided. Opened minded is the word for it.

 

To be blunt, I don’t think that your examples with planes and hammers are related to the issue/phenomenon that is discussed here, and more important doesn’t make your perspective stronger or easier to grasp. I would even say that it has the opposite effect.  

[Summit]

16 minutes ago, Patate91 said:

 

Distort app is a tool, hammer is a tool. All objective data for a tool cannot be published.

 

Same thing goes for plane, we don't have access to all the objective data before we buy a ticket. There's no doubt or skeptism to have about the plane's abilities to fly and bring you from point A to B.

 

But people can still ask for it if needed. Manufacturers or creators will provide it or not. For our situation feel free to ask everything toi want, we'll  see what Pkane anwser.

 

Again objective data can only be misleading if you don't have the knowledge to interpret it.

 

Which type of objective data that’s relevant varies greatly between a plane, a hammer and an app for measuring. Which data that are of importance depends, and I think it’s safe to say that the data/measurements that we needed or want to know for a simple device like a hammer and a complex thing like an airplane is not comparable.  

[Summit]

14 minutes ago, Patate91 said:

 

In fact I don't even need the hammer's data to use it and find that it does what's supposed to do.

 

Now what objective data do you expect from a tool like Distort? Seems that no one want to anwser it.

 

The reason you and I don’t need to know everything is because much is regulated and tested one way or the other, some directly by governments others by the branches.

 

I haven’t asked for objective data from a tool like Distort. I don’t think anyone have actually asked for anything besides the validated and verified independent tests that has been conducted.

[Summit]

32 minutes ago, The Computer Audiophile said:

That's the rub. Nobody has the knowledge of everything.

 

Knowledge is a familiarity, awareness, or understanding of someone or something, such as facts (propositional knowledge), skills (procedural knowledge), or objects (acquaintance knowledge). Knowledge can be acquired in many different ways and from many difference sources, including but not limited to experience, reason, memory, testimony, scientific inquiry, education, and practice. The philosophical study of knowledge is called epistemology.

 

https://en.wikipedia.org/wiki/Knowledge

 

[Summit]

For me, it is not the measurements that are misleading, it is what they are said to have for actual (SQ) effect that can be misleading. All measurements must be interpreted by someone and interpretations are based on preconceived ideas of what is audible and what is not.