Misleading Measurements

[John Dyson]

8 hours ago, opus101 said:

 

As another engineer here I disagree that measurement is the foundation of science. This would appear to be a dogma held by 'objectivists'.

 

I completely agree that 'my own listening is not a very reliable measurement method'. When I listen, I am making an observation, not a measurement. An observation is fundamentally a qualitative thing, not a quantitative thing.

Measurement isn't the 'foundation' of science, but is certainly needed to do science more reliably and more efficiently.

 

Measurement can take all kinds of forms -- some more reliable than others.   When a measurement system has poor accuracy/reliably, it will almost guaranteed lead one down rabbit holes.   I have been chasing rabbit down rabbit holes for 9yrs on my project, and eventually I have found a good solution.  If I had (or could find) a reliable measurement mechanism and/or a good design spec, then there would have been much less pain.   Of course, someone else would have figured out the solution if the problem was easy to solve.   Few engineers/individuals are as persistent as I am -- but that is also why I got the 'good/difficult' projects at the labs.

 

I am not espousing dogma -- just the opposite, my opinions come from and have been re-enforced by hard earned results and lots of learning experiences.

 

John

 

[John Dyson]

7 hours ago, opus101 said:

 

I'm not saying measurements don't play an important role in science, of course they do. But I am saying they're not foundational, observation is foundational.

 

'Hard science' is used here because you believe other (non-hard) kinds of science aren't science? I'd say some kinds of science are characterized as 'hard' because of their heavy reliance on measurement so the intersection of 'hard science' with 'measurements being not a key method...' is indeed the empty set.

Measurement is a critical subset of observation.   Biases/imprecision creep in with the typically less disciplined, relatively ad-hoc  'observation', therefore it is best/easiest to depend on more reliable, intrinsically more accurate objective measurement when at all possible.   If one really reads what I write, my recent experience includes both measurement and the more general observation.   Voluntarily choosing a more general, less solid form of 'observation' over and above accurate and reliable objective measurement can (will) be wasteful of time and effort.   I do suggest that *probably* because of a lack of expertise/knowledge, many people will choose the looser 'subjective' observation instead of the more clean, accurate 'objective' measurement.   In fact, I have used my own experience as an example, where there was NO OTHER choice but to use 'subjective' observations because the 'objective' was simply not available at all, to ANYONE.

 

If easier, better measurement methods were available, my project would have been done/completed almost 20yrs before, by someone else.   Using tools beyond that of our native senses requires more thinking and analysis that is often much less intuitive.  It is sometimes INITIALLY more difficult to be disciplined, but in the longer term the results can often come more quickly.  (Believe me -- a lot of math is very UN-intuitive, and some of that non-intuitive math is just under the hood of a lot of electronics design - but also overlooked by hobby level and even some pro developers.   Some of those non-intuitive aspects can make problem-solution much easier in the long term, and that is why I was sometimes used as an answer-person back at the labs.)

 

I give my input as a result of painful experience, even though as a very effective engineer, I already knew that it is best to avoid time wasting exercises.  An engineer doesn't get good feedback in their job if they dawdle too much.   Just because it looks like someone is doing a 'science project' doesn't mean that it really needs to be one.  ('Science project' here is used in a prejudicial way.)

 

 

[opus101]

2 minutes ago, John Dyson said:

Measurement is a critical subset of observation.   Biases/imprecision creep in with the typically less disciplined, relatively ad-hoc  'observation', therefore it is best/easiest to depend on more reliable, intrinsically more accurate objective measurement when at all possible.  

 

Yes, I understand that but I disagree - substituting measurement for (admittedly subjective) observation leads to reductionism. The solution to biasses in observation is more impartial observation, not measurement. 

[pkane2001]

5 minutes ago, opus101 said:

 

Yes, I understand that but I disagree - substituting measurement for (admittedly subjective) observation leads to reductionism. The solution to biasses in observation is more impartial observation, not measurement. 

 

I can use my tongue on the battery terminals to get a subjective observation of the remaining charge, but a voltmeter is going to produce a much more useful and accurate "observation". As long as the measurement device is properly calibrated and the error margins are known, there's no reason to keep relying on the difficult to perform and hard to validate subjective results.

[opus101]

1 minute ago, pkane2001 said:

 

I can use my tongue on the battery terminals to get a subjective observation of the remaining charge, but a voltmeter is going to produce a much more useful and accurate "observation".

 

The voltmeter doesn't produce an 'observation', its a measurement. I would of course agree, using a tongue to 'observe' the state of a battery is inapt.

[John Dyson]

8 minutes ago, opus101 said:

 

Yes, I understand that but I disagree - substituting measurement for (admittedly subjective) observation leads to reductionism. The solution to biasses in observation is more impartial observation, not measurement. 

Huh?  Did I write that?   I am for both, but you'll waste lots of time (per experience) if you live in the world of the subjective.   Maybe some people like tweaking -- I don't.

Observation INCLUDES measurement, but observation using normal human perception is naturally VERY susceptable to bias.   If you have a choice, you'll generally waste a lot less time with objective measurement.    I have visited WAY TOO MANY rabbit holes because of work on the edge of solid technical knowledge, and it is a good idea to avoid those visits.

Therefore -- when you have a choice, be careful about your human observation when it doesn't match a measurement.   We see this problem all of the time in the mostly hobby audio world -- too many 'observations' that don't match mathematical/physical reality IN ANY WAY -- EVER.

Sometimes, once in a long while, a human observation might be a good hint that there MIGHT be a problem with the measurement system, but that VERY SELDOM happens in this realm of 'very established' science.  (again, we aren't talking about 'sounds good', but more like objective equipment evaluation.)   Most often, the biases in observation overwhelm the perception of reality.   Do your best to avoid biases.   In this hobby audio realm, we are so far away from the edges of scientific knowledge that, if there is a choice at all,  measurements will be the winner.

 

I don't think that we are talking abstract theory on the edges of science or truly difficult problem solving, are we?  Aren't we talking about estabilished science with good measurement techniques?   Also, we are not talking about 'sounds good' -- that is purely subjective and outside of the scope of objectivism.

 

John

 

 

[John Dyson]

6 minutes ago, opus101 said:

 

The voltmeter doesn't produce an 'observation', its a measurement. I would of course agree, using a tongue to 'observe' the state of a battery is inapt.

The act of reading an accurate, good quality voltmeter used competently is also an observation.   A tongue makes a very inaccurate voltmeter, and if you try to use it for measurement of something that needs accuracy, the you'll be visiting a lot of rabbits.

 

[opus101]

1 minute ago, John Dyson said:

 I am for both, but you'll waste lots of time (per experience) if you live in the world of the subjective.   Maybe some people like tweaking -- I don't.

 

I certainly have spent (I wouldn't say 'wasted' myself) a lot of time in the world of the subjective. There aren't any short cuts to putting in the hours in the world of the subjective. I conjecture (but have no evidence for) that substituting measurement for subjective observation is an attempt to find a short cut, an attempt to avoid 'wasting' time. 

[John Dyson]

8 minutes ago, opus101 said:

I certainly have spent (I wouldn't say 'wasted' myself) a lot of time in the world of the subjective. There aren't any short cuts to putting in the hours in the world of the subjective. I conjecture (but have no evidence for) that substituting measurement for subjective observation is an attempt to find a short cut, an attempt to avoid 'wasting' time. 

When you use a well considered 'measurement', it often does require more of an intellectual understanding of what is going on.   A quickie human observation is easy, but not always so accurate.   A subjective observation can be a double-check, but not much else, that is-  otherwise can be useful, if there is no other resource.   Measurements are not just on the exterior/outside of a design, but also in the interior.   Using subjective listening (for example) limits the domain where you can observe.   Not all designs are linear througout -- a measurment will give much more information.   Processing that greater amount of info does require more intellect,  and IS often an important shortcut to finding an answer.   (I am not using the term 'SHORTCUT' in a prejudicial way, but instead using it as a synonym to efficiency.)

 

[opus101]

12 minutes ago, John Dyson said:

When you use a well considered 'measurement', it often does require more of an intellectual understanding of what is going on.   A quickie human observation is easy, but not always so accurate.   

 

I think it makes sense also to turn this around. To wit :

 

When you use a well considered observation it often does require more of an intellectual understanding of what's going on. A quickie human measurement is easy, but not always so accurate.

 

It seems (subjectively of course) to me you're saying that it matters how a measurement is done. Absolutely no disagreement there - the quality of a measurement is always going to be important. Just as the quality of an observation is. But that's my point really - quality matters, numbers don't.

[John Dyson]

2 minutes ago, opus101 said:

 

I think it makes sense also to turn this around. To wit :

 

When you use a well considered observation it often does require more of an intellectual understanding of what's going on. A quickie human measurement is easy, but not always so accurate.

I agree --in our discussion a quicky human 'measurement' is the 'tongue' technique.

 

So, it all depends on how you define measurement -- I tend to see that (measurement) as with some clear 'measure'.   The using the 'tongue' as a measurement device, 'tongues' measure isn't very precise or stable.

 

Likewise, an observation of a meter isn't very useful, without competent use of the meter.  (Using the term 'observation' as the result of human perception instead of a more general term that includes measurement as a verb.)   It is this general inability to competently use the meter, understand the measurement context/circuit where missing knowledge can give troubles.   Just reading a meter seldom needs much training though, even understanding what is going on 'underneath the needle' isn't very intellectually challenging.   It is this lack of understanding (not just reading a meter) that pushes a non-technical (or even inexperienced techie type) into an expedient subjective 'listen'.  I am all for helping people to understand what is really going-on under the hood, but that kind of learning can be time consuming (or even all-consuming.)  Electronics has so many layers, and it takes a few layers down to really understand what is going on in a most general and adaptable way.

 

It is all about trying to find the best reliaiblity or resulting stability & quality.   In the equipment realm, there are few places where a well considered measure/observation of an aid (device/meter/data-processor) isn't more accurate and reliable than just using the subjective hearing/listening mechanism.   This is especially true because FEW people who even consider themselves experts have actually calibrated their hearing and trained it based on hard core physical references.   Those who have, and trained themselves carefully to avoid biases, which would also include testing, could POSSIBLY attain some level of reliability, but feelings will almost always creep in.

 

John

 

[opus101]

2 minutes ago, John Dyson said:

I agree --in our discussion a quicky human 'measurement' is the 'tongue' technique.

 

So, it all depends on how you define measurement -- I tend to see that (measurement) as with some clear 'measure'.   The using the 'tongue' as a measurement device, 'tongues' measure isn't very precise or stable.

 

I define a measurement as a subset of observation that produces a purely quantitative result. I.e. a number (or series of numbers). If the result is qualitative then the operation wasn't a measurement it was an observation.

 

The use of a tongue on a PP3 doesn't produce a number hence can't be classed as a measurement, at least to my way of seeing things.

[pkane2001]

47 minutes ago, opus101 said:

Yes, I understand that but I disagree - substituting measurement for (admittedly subjective) observation leads to reductionism. The solution to biasses in observation is more impartial observation, not measurement. 

 

It is the goal of science to reduce what appear to be random or unpredictable phenomena in the natural world to a simpler, predictive, explanatory model. All scientific theories are reductionist in nature. The better models have a more encompassing predictive/explanatory power, covering more of the observations, current and future. Without "reductionism" there's no science, IMHO.

 

[opus101]

It looks like we are using 'reductionist' (or 'reductionism') in rather different ways.

 

I would agree theories do simplify, or reduce things to simpler things but they're not 'reductionist' in so doing.

[John Dyson]

21 minutes ago, opus101 said:

 

I define a measurement as a subset of observation that produces a purely quantitative result. I.e. a number (or series of numbers). If the result is qualitative then the operation wasn't a measurement it was an observation.

 

The use of a tongue on a PP3 doesn't produce a number hence can't be classed as a measurement, at least to my way of seeing things.

I try to make sure that people who depend on their observations or measurements truly understand the quality of their results.   Low quality results can result in many rabbit chases.

 

Just depending on the human body's stability doesn't bode well for the results.   Subjective evaluations result from many variables, therefore making it difficult for any kind of serious precision.   Subjective is okay for verifcation, but not precision.   Geesh, subjective viewpoints are even somewhat dependent on metaphysics.

 

[John Dyson]

3 minutes ago, opus101 said:

It looks like we are using 'reductionist' (or 'reductionism') in rather different ways.

 

I would agree theories do simplify, or reduce things to simpler things but they're not 'reductionist' in so doing.

Reductionist seems prejudicial...   There are many times that one cannot 'divide and conquer' in a design -- but that often becomes MORE of an intellectual challenge rather than over simplifying into subjectivism...   Being subjective when not absolutely necessary  is like 'giving up' on actual engineering.

[opus101]

2 minutes ago, John Dyson said:

Geesh, subjective viewpoints are even somewhat dependent on metaphysics.

 

To me a 'viewpoint' is not at all the same as an observation. Rather its more akin to an opinion.

[opus101]

2 minutes ago, John Dyson said:

Being subjective when not absolutely necessary  is like 'giving up' on actual engineering.

 

Well yes - a tongue in place of a voltmeter would seem to be an example of this. Though if a voltmeter isn't available, then a tongue could be the only way to attempt a 'measurement'. 'Necessity' in my experience is typically a subjective determination.

[John Dyson]

5 hours ago, John Dyson said:

Reductionist seems prejudicial...   There are many times that one cannot 'divide and conquer' in a design -- but that often becomes MORE of an intellectual challenge rather than over simplifying into subjectivism...   Being subjective when not absolutely necessary  is like 'giving up' on actual engineering.

My comment about 'divide and conquer' not being operative is very important.   Some designs are easily divided and conqured, but make a minor architectural change or a structural change, then the result can be a morass of interdependencies.  Sometimes these kinds of changes are more of an intellectual challenge, but there is great benefit in  'solving the problem' from a totally rigorous standpoint.   Absolute rigor is sometimes not practical, so I claim that the answer is NOT to give up and devolve into a subjective 'tweak until it works', because this becomes the 'roomful of monkeys' scenario.   There are often hybrid approaches, where the structure can be predicted, but the math is impossible (or nearly so.) 

 

With the subjective design using the roomful of monkeys, then the project will never be completed.

With the requirement of total rigor, then the project will not be completed for different reasons.

 

Sometimes a hybrid approach is a good thing, but in my own necessarily hybrid approach, avoiding the subjective has been wise.   When trying to use other people as data inputs, using their subjective abilities, then I often ended up with more noise than data.    Dunning-Kruger effect is very operative in the audio world, probably other psuedo-engineering and psuedo-science activities also.

 

My resistance to ad-hoc, tweakabily approach is well considered and comes from REAL experience.   Resistance doesn't mean total rejection, I tend more to wisdom instead of religion* in my approaches.   My only 'unwise' decision was to do my project at all, but the result has been well worth it.   Sometimes doing an impractical thing is worthwhile, because in the end-game, it wasn't really impractical -- it was invention.

 

* There are better applications to 'religion' other than psuedo-science and psuedo-engineering!!!   That is, I am not anti-religion in general!!!

 

John

 

[fas42]

As being discussed, observation is the start of achieving understanding - unfortunately, philosophical as well as ego factors cause some to downplay, or ridicule other people's observations ... and science is always the loser here. Insistence on measurement is used as a weapon by one side, in the hope that 'uncomfortable' observations can be made to go away ...

[John Dyson]

1 hour ago, fas42 said:

As being discussed, observation is the start of achieving understanding - unfortunately, philosophical as well as ego factors cause some to downplay, or ridicule other people's observations ... and science is always the loser here. Insistence on measurement is used as a weapon by one side, in the hope that 'uncomfortable' observations can be made to go away ...

The problem that I have recently seen -- ego is often messed up with the subjective observations.   Getting away from the subjective also helps to mitigate the expectation biases and the Dunning-Kruger effect that seems to be so associated with  subjective observations.   The only real answer is proper scientific and statistical methods, sometimes even simple applications of scientific method can expose the personal and ego biases that are oh so associated with being subjective.   I have recently done some simple experimental controls  to expose a severe problem with certain subjective reviews.    Subjective must not be the 'default', but instead objective -- then use subjective as a cross check.   Unchecked subjective review is almost the same as relying on a high priest, and is even less valuable than a random choice -- because there are sometimes negative biases.   In actual development and invention, ubjective evaluations really need a competent statistical wrapper, or they are just opinion -- and we all know about opinions.

 

If someone else wants to chase rabbits and play whack-a-mole, then so be it.   I strongly counsel that using subjectivism with double checks and controls should be secondary.   Objective methods with double checks and controls (e.g. controlled experiments) should be the first choice to avoid wasting time.  Subjectivism without controls (something like scientific method) should only be used in the realm of 'sounds good' and that is it.

 

Some people, however enjoy tweaking as a hobby -- being a tweakabilly isn't my goal, but my goals have usually been pragmatic results.   I won't take anything away from people who enjoy the endless 'tweak', but I have useful things that I prefer doing.

 

 

[sandyk]

1 hour ago, John Dyson said:

I strongly counsel that using subjectivism with double checks and controls should be secondary.   Objective methods with double checks and controls (e.g. controlled experiments) should be the first choice to avoid wasting time

 It hasn't always worked out that way in your PM group though, has it ? 😉

I will leave it at that, and not reply further, as we are so far off topic already. with virtually no poster  giving OBJECTIVE illustrations , whether by later refuting measurements or the results of DBT sessions that showed the measurements in the example given were misleading instead of the philosophical discussions  currently  .

 

 

  

[fas42]

7 minutes ago, sandyk said:

Will somebody please point me to the conforming posts that I must have missed, as I would really like to see some examples of Misleading Measurements instead of the philosophical discussions  currently . 

  

 

Measurements are misleading, if they are used to 'prove' that the performance of whatever is beyond reproach - in audio, it doesn't work this way; the ear/brain can easily detect anomalies in a complete system, during playback of a recording - it ain't pleasant to listen to  🙂.

 

Of course, some then turn around and say, "Well, it's a crappy CD you're playing!"  - now, you can't get more objective than that, as a reason, can you? ... 😝

[opus101]

1 hour ago, John Dyson said:

Unchecked subjective review is almost the same as relying on a high priest, and is even less valuable than a random choice

 

Quite the opposite (though I am assuming 'review' here simply means 'reporting one's subjective observations') - the checking function historically was always the role of the priesthood. 'Go, show yourself to the priests' was said by Jesus after a healing for example.

[sandyk]

50 minutes ago, opus101 said:

 

Quite the opposite (though I am assuming 'review' here simply means 'reporting one's subjective observations') - the checking function historically was always the role of the priesthood. 'Go, show yourself to the priests' was said by Jesus after a healing for example.

I agree, and John may not be aware that you are also well qualified technically, not just anotherSubjective type.