Why are objective assessments important...

[Iving]

32 minutes ago, DuckToller said:

sound like a dream to me, but I am not an expert in neuroscience ....

I would have a guess, that the full set of personal factors that determine our subjective perception could only be integrated as approximative & averaged data, which in turn may give us only a divergent perception from the ones we hold ... 
You may want to correct me here, I'd be happy to learn how our lifelong individuality & experiences can be integrated in a measuring model 
Best, Tom

 

I went a long way towards answering this question here:

.

If I I'm not mistaken, @Audiophile Neuroscience is a psychiatrist (medically-trained practitioner) whereas my career background was in academic psychology with an emphasis on Research Methods and Statistics. We will have slightly different perspectives on this no doubt!

The key parts of my notes in the post linked above are:

- It's true that we can then proceed with measurements and conduct rigorously designed studies to see whether reliable experience reports are significantly correlated with the variables we have harnessed for that purpose.

- Even with such procedures we cannot attribute cause without intervention studies (change the level of an IV to see whether a change in the DV accompanies that change).

- As I see it, that is putting the cart (measurements) before the horse (musical enjoyment) anyway - so moving along - we are all interested in music and enjoyment - so why not see whether our enjoyment (choose your DV - take your pick - discuss) can be explained by plausible IVs (take your pick - discuss).

- Then you need good ways of measuring material *and psychological* variables combined with lots of subjects/participants and (M)ANOVA and/or regression etc with suitable statistical control and evaluation of interactions (between IVs) in order to draw fitting conclusions.

 

So there are *many* ways one could approach this in terms of developing one or more research questions.

 

For myself, "individual differences" or "personality" represents a fascinating realm of measurement possibilities on the "Independent Variable" (IV) side of the equation. Any hedonistic experience could qualify as an interesting "Dependent Variable" (DV)!

 

It's quite possible to keep things focussed more on equipment and measurement of equipment on the IV side of things, and more on "perceptual" reports rather than "enjoyment" ones on the DV side. Either way we are missing at least half the picture by failing to consider psychological variables.

[Iving]

At last - edifying debate!

 

I'm aware that this thread is in "Objective-Fi". Here and recently on "When do measurements correlate with subjective impressions" I have demonstrated that I am neither Subjectivist nor Objectivist; indeed, I argued that these are limiting categories if "truth" is our goal. I am neither troll nor disrupter. I will keep my few quote-wise remarks to "response" vs. opening up new fronts. To me, "Objective" considers the merits of and possibilities surrounding that which is *measurable*. Generally we don't consider psychological matters here at AS. A psychological angle is not necessarily a Subjective angle - quite the contrary if it resorts to an empirical approach. There is a strong Behaviourist tradition in modern psychology - usually associated with the laboratory-based conditioning experiments of Pavlov (Classical Conditioning) and Skinner (Operant Conditioning) - not to mention Thorndike and of course Watson. Scientific psychology also embraces as core elements of its BPS-endorsed curriculum in the UK inter alia Developmental Psychology, Personality aka Individual Differences and of course Hearing, Speech and Language. Just like any ordinary degree in the physical sciences, it takes three years of full-time effort to navigate merely an undergraduate qualification. Most of us - even if postgraduates or academics - are Jacks of All Trades when it comes Audiophilia - i.e., Masters of None. If we all chip in a bit - remaining open-minded except to the truly preposterous - we can hope for a little enlightenment.

 

20 hours ago, Iving said:

If I I'm not mistaken, @Audiophile Neuroscience is a psychiatrist (medically-trained practitioner)

 

9 hours ago, Audiophile Neuroscience said:

I am not a psychiatrist ... I am definitely not a neuroscientist ... Neuroscience reflects an interest not a qualification.

 

Many apologies. Inculcated with thoughts of statistics, I added 2 + 2 and made 5. Mea culpa. [Actually I blame @Samuel T Cogley's humorous departure-flourish - I think I got you and @Bill Brownmixed up as one nebulous and of course imaginary character.]

 

10 hours ago, Audiophile Neuroscience said:

I believe @Iving has given an expert reply to this.

 

Thank you

 

20 hours ago, Iving said:

It's quite possible to keep things focussed more on equipment and measurement of equipment on the IV side of things, and more on "perceptual" reports rather than "enjoyment" ones on the DV side.

 

10 hours ago, Audiophile Neuroscience said:

Where I differ a little bit with @Iving is while I agree that the enjoyment of music is the endgame I don't think you necessarily need to make that the dependent variable ... rather than talking about preferences or better or worse sound quality per se I am saying to try and establish what part of the measured stimulus correlates and is concordant with a physical percept. I think that is within the realistic realms to achieve, maybe, outside of academia but people very familiar with research methodology and statistics would still need to be involved.

 

I think we are on the same page!

 

Measuring perception [self-reports] as DV in relation to "Equipment" manipulations as IV(s) is bound to constitute the short-run game and appeal more to hard-nosed folks with a Scientific bent. One could conduct experiments with psychological or "non-Equipment" variables *only* - e.g. Extraversion as IV to perceptual thresholds in the context of a "Stimulus Intensity Modulation" model say - not too tough on an "Objectivist" mindset.

 

4 hours ago, Archimago said:

Ultimately, understanding human perception, cognition, consciousness, and sentience is no doubt one of the many "ultimate questions" which humankind will explore in science in the measure of time.

 

I am an advocate for mysteries! Yes - these "ultimate questions", particularly consciousness (although human language, subjective experience of pain, aesthetic appreciation including that for beauty, mathematics, music etc may be regarded philosophically as mutually bound with human consciousness - and thus all these ideas may be considered equally elusive) may be explored "in science" - but I doubt - at least for argument's sake - that science, as we understand science since the Scientific Revolution, is the paradigm in which humans will appreciate these things as fully as we may ever be able to do that.

 

10 hours ago, Audiophile Neuroscience said:

I don't disagree that we have unique experiences and that we can all perceive things differently. I mean the latter both figuratively and literally (another topic).

 

8 hours ago, Archimago said:

If we can completely, transparently reproduce that "source" with a perfect turntable, perfect CD player, perfect DAC... No losses in the cabling... No losses in the preamp... "Perfect" amplifier to a perfect speaker/headphone that can "faithfully" (as in "high fidelity") reproduce what the data encoded in that source is, then that is all we can hope to do. "Transparency" to the source content is all that we can ever achieve. What happens in the mind neurobiologically is of no direct concern to high-fidelity audio reproduction or to hi-fi companies even though it would be very interesting academically ... to me it doesn't matter what neurobiology is saying when it comes to the hardware goal... These preferences are idiosyncratic in nature and not something that we need to chase.

 

 

These are the teasers of the day for me. In other words I'll probably find myself reflecting for a while on the psychology of individual differences in music appreciation given the *hypothetical* proposition of a perfectly transparent system. On the one hand I accept that proposition (i.e. "perfectly transparent") as a logical foundation for discussion - it's like twin studies in psychology - if DNA is identical then DV differences cannot be attributed to genetic differences and may plausibly be attributed elsewhere instead - on the other hand - I ask, "Where does that perfectly transparent system end?" And answer rhetorically, "At the DAC output?"; "Just in front of the speakers?"; "At the human eardrum?"; "In the afferent nerves?" - "In the yawning and unexplored - even unfathomable [?] - chasm between those impulses and the subjective experience of music?"

 

But the OP's remarks suggest we may be straying beyond the scope of the Thread Topic here and so I shall stop any moment.

 

5 hours ago, Audiophile Neuroscience said:

Science really isn't about proof, it's about probabilities and evidence, test and retest and setting conditions to reject the null hypothesis. I totally agree that properly controlled trials that eliminate bias and confounders and using demonstrably valid tests is the way to go. This is a huge topic but would take us off topic for this thread.

 

Well said.

 

3 hours ago, STC said:

Our own judgment changes with time. As a young boy, I was mesmerized by the Poetry of the sea album. Every night after my parents were asleep, I would sneak in the hifi set to my room and place the speakers to the side of the bed and I will lie in between. It was so real sounding to me at that time. As time goes by, I have forgotten about the cassette until about 10 years ago when I saw the CD. When I played them in my system which is far superior than the player I had 45 years ago, the emotional connection wasn’t there. And like all audiophiles I too thought the analogue tape was a superior medium compared to digital. (Actually, this is also one of the thing why I abandon hirez and high end but that story is for another day).

As you can see from my experience, it wasn’t the measurements that brought out the best sound to my ears when I was a kid. It was the emotional connection of hearing the sea gulls flying from left to right, the boat engine, the crashing waves and the orchestra trigged something in me.

 

Beautiful. Compelling. Beguiling.

 

Things are not always as they seem.

[Iving]

 

21 minutes ago, DuckToller said:

The Long Read  at The  Guardian's webpage called "Why your brain is not a computer"

My short resume:  There seems to be a lot unclear in that area ...😉

 

It's a good article ... and an object lesson in the kind of "humility" I've mentioned once or twice.

 

Honestly ... some things just are. Science hates circular arguments. But that's where we're going (maybe necessarily!) with this one.

 

I just mean - @Archimago has been very lax and kind about scope on this thread lately - but even I'm thinking we're getting a bit wild for "Objective-Fi"!

 

All meant in good humour!

[Iving]

1 hour ago, tapatrick said:

Agree wholeheartedly but I do not have the training or experience so this might be a good point to ask:

• what are the measurements that can be made, or should be made in relation to analysing performance of equipment?

• and secondly overall sound quality

• what equipment can be used/bought for home use?

• how do you make and record these?

• how can we 'view' them?

• what should we be looking for in the measurements?

 

These are great questions, and in a few of my more recent posts (other than e.g. "Moon Music") I've attempted to approach some of them from a stance of logic / principle / research methodology.

Anything that is *measured* is in the domain of "Objective-Fi". This is also aka the "empirical approach". Its primary advantage is that it permits agreement based on the reproducibility of the Results of any given experiment / data exercise. The extent to which we may agree should not be based on our *biases* - but rather on the extent to which the hypothesis has been framed well, the Method unassailable in terms of its logic, the Design and statistics sound, the interpretation correct - again mainly a matter of straight thinking and logic - and the Conclusions drawn appropriate. This is *the* empirical/scientific approach to researching questions to which we would like answers. It is used in *all* the mainstream sciences. In fact, without empirical support for our position we are expressing mere opinion. Any strong assertion should always be supported with reference to empirical precedents. The problem for social media Forums like this is that we converse casually about this and that - and quite naturally many of us are not familiar with the Scientific Literature germane for any given conversational assertion. If we are speculating - we should say so really. If we speculate with strong assertions then we are being a little unfair on ourselves and others - and stifling reasonable debate. If we speculate with strong assertion and also ridicule - then we are trolling - and that prevents anything useful happening (except satisfying the ego of the troll in what is only ever a selfish and empty or hollow victory).

To keep things ever so simple, we could for example:
1. Hypothesise that adding component X to piece of equipment Y might result in better SQ. We find that SQ scores reported by an adequate pool of Ss are (or are not) significantly higher (in the statistical sense) for (X+Y) vs. Y. The Method / Procedure would have to be rigorous (controlling for nuisance variables). The stats would depend on whether between or within Ss, and the p value would have to be less than 0.05 for sig. If it were say <0.0001 we would be *very* convinced (and manufacture accordingly). This is sort of a bottom-up approach.
2. We could wonder whether a thick cable was better for SQ than a thin cable. We conduct an experiment. Again rigorous *where only thickness of cable differentiated levels of the IV* and Method / Procedure sound. If Ss reported that the thicker cable sounded better at p<0.0001, it would be *undeniable* that *something* about the cable was accounting for SQ - but unlike 1. above we wouldn't know what it was (without conducting further experiments). This is sort of top-down.

 

Both of these research approaches are *equally legitimate*. There is nothing unduly philosophical about the latter approach in 2. Absolutely, both approaches are *pragmatic for audiophiles* as much as for Scientists at large.

 

I know I am not addressing all your bullets @tapatrick. I am trying to make a simple contribution to "Objective-Fi".

"Subjective-Fi" is where we assert an experience - but we *don't* have direct evidence for it - except our personal experience of it. *There is nothing wrong with that* - and for all anybody knows - a person could be "right" about their assertion - and it is difficult (if not impossible) to *prove* a subjective experience false - unless the preceding premises for the assertion are illogical [but even then someone who believes they are Napoleon is in fact for all psychiatric intents and purposes Napoleon]. But that is not what we see [or used to see anyway] typically here on AS - we see people ridiculing a person for reporting an enjoyable tweak - when the logic for contradicting the subjective experience germane *has far from been established*. For example - and I am not advocating Myrtle Wood or whatever it is - but if a person hears a difference raising a cable on Myrtle Wood - it *could* be true that they *do* here a difference - whether that experience is a subjective "illusion" - or whether there is something *that we do not know about Myrtle Wood* that accounts for that SQ difference. Just because basic knowledge of physics combined with comparatively crude measuring exercises suggest that Myrtle Wood cannot have such an effect, doesn't mean that there isn't something *unknown* about Myrtle Wood that is affecting the person's "ears". I say all this "leading with the chin" I suppose - but please - I am talking about principles of logic - not Myrtle Wood.

Because we have fallen short in the rigour of our thinking and in our "humility" - both intellectual and social - we have ended up with two playgrounds here at AS. But that is far more likely to result in enlightenment than one in which trolls (from either camp) and ribaldry reign supreme (such as at certain other Forums) - and earnest people stay away.

 

I think I have provided a fair synopsis-answer to the thread Title.

Notes prepared before breakfast - please excuse any minor mistakes.

I repeat - I'm not on about Myrtle Wood OK? 🙂 [nice smile]

[Iving]

17 minutes ago, tapatrick said:

Yes to be convincing there would have to be this level of rigour applied. Any studies I have come across (which aren't that many) which depend on data from listeners preferences reveal that there seem to be very few people who are consistently accurate. The study on wine tasting experts not being able to tell good from bad was similar in this respect.

 

The whole point of Design/Method/Procedure is to contest these problems. The stats only demonstrate (or deny) reproducibility. That is their fruit!

 

Honestly I have no idea the extent to which manufacturers carry out rigorous performance studies prior to launch - but I don't want to open a can of worms.

 

For we consumers, the logic of our choices should afford us proportionate confidence or doubt. If we are right-thinking we are more likely to enjoy our online social environment (and less likely to wind each other up).

 

I presume that in the end we just want musical satisfaction. I'm only ever looking to improve the octane quality of my musical kicks.

 

Just love your post.