The problem with subjective impressions

[Summit]

The problem with subjective impressions is not that they are made by listening and comparing different devices. At least not in my opinion. The problem is that when we describe the difference between diverse hi-fi stuff, we are often too subjective in our descriptions. By subjective, I mean that we often use exactly the same terms and adjectives to describe the difference, whether the difference is relatively small or if it is more significant.

 

Now, I'm pretty sure some people disagree with me on this, and think that they describe their subjective observations in an objective way. Okay there are exceptions, of course, but I would say they are in the minority.

 

I think that no matter what parts of a hi-fi system, placement and the room itself we believe are most important, it is often difficult when reading reviews or impressions to understand the reviewer's gradation difference between a little but observable - to that sounds like a completely different hi-fi system. If you do not already know from your own experience that the speakers and their placement in the room almost always have a much greater impact on the sound quality than a power cable, frame memory or a switch, it can be difficult to interpret how much impact we are really talking about. Now, I think most people here know the difference I described, but maybe not between all other parts of our hi-fi chain.

 

Anyway I've read a large number of reviews and impressions and BELIEVE that the “problem” is because we often describe subjective differences too subjective and in more or less the same way regardless of whether we describe more significant differences from two completely different types of amplifiers or two external power supplies of the same type. What I want to say is that although I think there are relevant differences between different cables, hard drives, PSUs, external clocks and so on, but because of how we usually describe our observations in the same way, it is often difficult to know how big a difference it actually is at the system level and compared to other upgrades we can do for the same money.

 

I am all for subjective observations, but think that we can all benefit if we can get a little more objective in our descriptions in our reviews and impressions. I have also observed that the reviews that are more moderate written do not become as "popular" as those where it seems that the difference is of a more revolving nature. Yes, I actually think there has been a steady inflation of superlatives generally in the hi-fi world, and who really benefits from that? Not one of us I would think. If someone buys something and believes now my audio system will sound superb, but does not, well then the person can be burnt and not captivate observations describing the more significant differences of other gear, because the exact same superlative has been used, and reused. What I wish for is that we all would try to describe the subjective differences more nuanced and in a more objective way and not as if all parts of the audio chain play the same role for SQ, although of course many small individual differences together make more significance.

 

The question is how can I / we describe the difference between devices in a detailed way but still not make it sound like the difference is bigger than it really is?

 

I know that some reviewers wait several months before writing and publishing their review. Maybe we can get a better perspective on the actual sound difference that way? Less FOTM and talk about burn in.

 

TD; LR I think that we would all benefit on some sort of categorizing of how big the difference is in relative terms and in the grand scheme of things instead of as it was an isolated part of the audio chain.

 

And rant

[John Dyson]

Much of the time, subjective impressions are substitutions for the inability to make a clear, clean objective compaison.  However, sometimes the measurement criteria isn't stable/high enough quality that objective measurements are very meaningful.

When possible, objective measurments are the best first bet, but secondary subjective compaisons can be helpful.  Also, like mentioned above, sometimes a pure objective measure might be impossible, difficult or beyond the state of the art.

 

(The above opinion comes from someone who might be pre judged to be wholly dependent on objective measurements.)

 

John

 

[STC]

I think the problem with subjective impressions is they are mostly made up to justify their choice of purchases.  
 

I grew up with a reasonable collection of good music and it is always been about music. I remember the gramophones, TT, cartridge, cassettes and CDs. It was never about equipments.   
 

Audiophiles created a non existent problem and then attempted to remedy them. A music lover only needed clarity and loudness. They use treble and bass to address room acoustics and placements. It worked and works well for 99% of the population. 
 

Then they wanted something better without understanding what they want. They buy, say a powerful amplifier, it didn’t turn out to be the best purchase. Without treble and bass, the sound wasn’t what they were accustomed too. Then after few amps they change the speakers. Then they spend hours on speakers position. Then they realize they need to to treat the room to address the bass boom. And the the vicious cycle continuos....

 

The hobby from getting a better system to enjoy the music now becomes the hobby of listening to the equipments. Then the time will come where your system will be talk of your friends. You by virtue of having a more expensive system naturally will become an expert. Your subjective words than becomes another person’s reference. 
 

Slowly you go out and listen to other systems. Some of them going to sound much better than yours. That persons opinions and subjective description of sound quality now becomes your guide. The vicious cycle continues....

 

Audio is not difficult to understand. Good sound production is not complicated. Not one person in the world would argue that the sound of the 60s cinemas were better than current cinemas. But we have audiophiles still believe a 60s amplifiers or speakers supposedly to be the holy grail of the ultimate SQ. 

[Rexp]

All us subjectivists can get a little carried away sometimes, I think its important to have a reference sound to benchmark against. What hi-end systems can do is allow recorded music to trigger an emotional response in the same way a live performance can. So for me its fairly black and white, either a component/system delivers the emotion in the music or it doesn't. Beyond that it's personal preference. 

[Kimo]

Subjective impressions may be problematic in that they generally will reflect the listener's subjective impression of what a good system should sound like,  which may not align with your own. 

 

For example, I have a strong dislike for almost all metal dome tweeters to the point that I find most systems difficult to listen to when I first sit down with them.  They typically sound harsh and flat to me.  I am aware that the Raal ribbons I prefer probably don't measure as well as some dome tweeters, but they also don't annoy me.  For those who don't share my affliction, the metal dome may be the superior choice, and my subjective view isn't worth squat to them.

 

If you do happen to find someone who shares your impressions in general, I believe that their subjective impressions can be quite valuable.  

[Summit]

1 hour ago, Kimo said:

Subjective impressions may be problematic in that they generally will reflect the listener's subjective impression of what a good system should sound like,  which may not align with your own. 

 

For example, I have a strong dislike for almost all metal dome tweeters to the point that I find most systems difficult to listen to when I first sit down with them.  They typically sound harsh and flat to me.  I am aware that the Raal ribbons I prefer probably don't measure as well as some dome tweeters, but they also don't annoy me.  For those who don't share my affliction, the metal dome may be the superior choice, and my subjective view isn't worth squat to them.

 

If you do happen to find someone who shares your impressions in general, I believe that their subjective impressions can be quite valuable.  

 

Yes it’s true that subjective impressions generally will reflect the listener's subjective impression and preference, it is why they are subjective. I am not talking about preference per se, but that most reviews and impressions are written the same and with the same words. I mean you can almost take any review and how the gear they liked the most is described and just change the name of the gear for another gear and voila. The problem as I see it is this type of review will not reflect the SQ of the gear in the grand scheme of things.

 

Even if a good USB cable for example is important for obtaining really good SQ I would not say that the difference from one good USB cable to better USB cable to be as big as the difference between a good amp and a better amp. I think that it would be great if we would get some sort of classifying of how big the difference is in relative terms (1-10), so that our subjective listening impressions would be more objective. Not objective as in measurement or DBT, but more like how big was the overall SQ effect by changing gear A to gear B in their reference audio system. Most impressions focus too much on the reviewed gear IMO and in the effort to describe the gear in depth we may not know how big the upgrade/change really is as part of a system and together with the other equipment in the audio chain. Some does, but most not.

[Archimago]

55 minutes ago, Kimo said:

Subjective impressions may be problematic in that they generally will reflect the listener's subjective impression of what a good system should sound like,  which may not align with your own. 

 

For example, I have a strong dislike for almost all metal dome tweeters to the point that I find most systems difficult to listen to when I first sit down with them.  They typically sound harsh and flat to me.  I am aware that the Raal ribbons I prefer probably don't measure as well as some dome tweeters, but they also don't annoy me.  For those who don't share my affliction, the metal dome may be the superior choice, and my subjective view isn't worth squat to them.

 

If you do happen to find someone who shares your impressions in general, I believe that their subjective impressions can be quite valuable.  

 

Curious Kimo, have you ever tried blind-testing a dome tweeter speaker with similar type of speaker (be it 2-way monitor or maybe floor stander) using ribbons at the same volume level to confirm that impression?

 

On the one hand, the issue with subjectivism is about the language used as you suggested. Perhaps a more standardized way to describe the sound as I have seen tried over the years can help - this includes common language for the adjectives used, and also core characteristics of the sound.

 

What I think is more important is the lack of "honesty controls" put into place so that the subjective descriptions are just that and not tainted by name brand, knowledge of price tag, being buddies with the designer, the grace shown by the company in allowing long-term loans, trips to the manufacturing facility, dining experiences, ad support, etc...

 

The only way the above issue can be addressed is with blind-testing. Unless one can do that effectively, I believe it is essentially impossible not to be affected by these biases. The problem of course is that most subjective reviewers do not do this. And often will fight against those who suggest controlled listening tests. And even ban discussions, IMO of the most important procedural factor to add value to listening tests!

 

As per JG Holt's response:

Quote

 

Do you see any signs of future vitality in high-end audio?

 

Vitality? Don't make me laugh. Audio as a hobby is dying, largely by its own hand. As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me, because I am associated by so many people with the mess my disciples made of spreading my gospel. For the record: I never, ever claimed that measurements don't matter. What I said (and very often, at that) was, they don't always tell the whole story. Not quite the same thing.

 

 

[Kimo]

1 minute ago, Summit said:

 

Yes it’s true that subjective impressions generally will reflect the listener's subjective impression and preference, it is why they are subjective. I am not talking about preference per se, but that most reviews and impressions are written the same and with the same words. I mean you can almost take any review and how the gear they liked the most is described and just change the name of the gear for another gear and voila. The problem as I see it is this type of review will not reflect the SQ of the gear in the grand scheme of things.

 

Even if a good USB cable for example is important for obtaining really good SQ I would not say that difference from one good USB cable to better USB cable to be as big as the difference between a good amp and a better amp. I think that it would be great if we would get some sort of classifying of how big the difference is in relative terms (1-10), so that our subjective listening impressions would be more objective. Not objective as in measurement or DBT, but more like how big was the overall SQ effect by changing gear A to gear B in their reference audio system. Most impressions focus too much on the reviewed gear IMO and in the effort to describe the gear in depth we may not know how big the upgrade/change really is as part of a system and together with the other equipment in the audio chain.  

 

I don't think that we are going to be able to set an objective scale for subjective impressions for much of anything in life.  How much hotter is vintage Jennifer Garner than vintage Pam Anderson?  We are still stuck with subjectively even when talking about different kind of 10s.   

 

[Summit]

4 hours ago, STC said:

I think the problem with subjective impressions is they are mostly made up to justify their choice of purchases.  
 

I grew up with a reasonable collection of good music and it is always been about music. I remember the gramophones, TT, cartridge, cassettes and CDs. It was never about equipments.   
 

Audiophiles created a non existent problem and then attempted to remedy them. A music lover only needed clarity and loudness. They use treble and bass to address room acoustics and placements. It worked and works well for 99% of the population. 
 

Then they wanted something better without understanding what they want. They buy, say a powerful amplifier, it didn’t turn out to be the best purchase. Without treble and bass, the sound wasn’t what they were accustomed too. Then after few amps they change the speakers. Then they spend hours on speakers position. Then they realize they need to to treat the room to address the bass boom. And the the vicious cycle continuos....

 

The hobby from getting a better system to enjoy the music now becomes the hobby of listening to the equipments. Then the time will come where your system will be talk of your friends. You by virtue of having a more expensive system naturally will become an expert. Your subjective words than becomes another person’s reference. 
 

Slowly you go out and listen to other systems. Some of them going to sound much better than yours. That persons opinions and subjective description of sound quality now becomes your guide. The vicious cycle continues....

 

Audio is not difficult to understand. Good sound production is not complicated. Not one person in the world would argue that the sound of the 60s cinemas were better than current cinemas. But we have audiophiles still believe a 60s amplifiers or speakers supposedly to be the holy grail of the ultimate SQ. 

 

I do not share your belief that subjective impressions mostly are made up to justify choice of purchases.

 

I believe it’s true that in the effort to get better and better audio system to enjoy the music it can become the hobby of listening to the equipment.

[Kimo]

4 minutes ago, Archimago said:

 

Curious Kimo, have you ever tried blind-testing a dome tweeter speaker with similar type of speaker (be it 2-way monitor or maybe floor stander) using ribbons at the same volume level to confirm that impression?

 

 

Nope.  I got tired of trying to "fix" the treble on my former Vandersteen, Dynaudio, Triangle, etc. housemates and banished them.  I am okay with most Tannoys though, for whatever reason.  I used single drivers for many years, my dislike of most tweeters being so strong.

 

I now use objective response.  My narrowing eyes.  The sneer of disapproval of my wife provides my confirmation bias.  

[Summit]

8 minutes ago, Kimo said:

 

I don't think that we are going to be able to set an objective scale for subjective impressions for much of anything in life.  How much hotter is vintage Jennifer Garner than vintage Pam Anderson?  We are still stuck with subjectively even when talking about different kind of 10s.   

 

 

No the scale is subjective, but reflects the importance of the audio gear in the audio chain and not only as one individual equipment.  

[Kimo]

4 hours ago, STC said:

But we have audiophiles still believe a 60s amplifiers or speakers supposedly to be the holy grail of the ultimate SQ. 

 

We also have respected and successful tube amp manufacturers that would say the same thing, though not about Scott and Fisher.

 

I also know of speaker manufacturers that will tell you the greatest sounding driver ever built was the RCA.

[Kimo]

1 minute ago, Summit said:

 

No the scale is subjective, but reflects the importance of the audio gear in the audio chain and not only as one individual equipment.  

 

I would agree with you on the importance of amps and speakers, but others would not.  I know people that will tell you that a good preamp is the foundation of great sound in any system, and others that would say no preamp is the best preamp.

[Summit]

14 minutes ago, Kimo said:

 

I would agree with you on the importance of amps and speakers, but others would not.  I know people that will tell you that a good preamp is the foundation of great sound in any system, and others that would say no preamp is the best preamp.

 

I am all for people having different thought and subjective preference. To make ones preference known is the best way to share one’s subjective impressions. To be objective is to reduce subjective factors to a minimum, but we can never eliminate them, and most people will tell it like they hear it. Preamp or not. 

[plissken]

The issue for me is where everything is subjectively different including things that can't make a difference.

 

It's a credibility issue. I miss Julian Hirsch

 

" Here’s a revealing quote from his October 1977 “Technical Talk” column: “A purely subjective reaction to an audio product tells a reader only what that particular reviewer thought of the product when he ‘heard’ it in a given room, with certain associated equipment, at a particular time, and under particular conditions. Such an evaluation is bound to be highly personal and limited in scope.”

[Kimo]

2 minutes ago, Summit said:

 

To be objective is to reduce subjective factors to a minimum, but we can never eliminate them, and most people will tell it like they hear it. Preamp or not. 

 

You would think so, but that is not what I have experienced.  One time about dozen audiophiles all older than myself were mulling around in a shop and the owner, who was messing around with some test equipment asked us to raise our hands as he played test tones rising in response with each subsequent play.  As he ran through ascending tones, one fellow who admitted his hearing was damaged dropped out rather quickly.  I was next at 16k.  

 

The rest of the golden ears kept popping up their hands as he advanced well in the mid 20K region.  After the test I told him that I was surprised that my hearing dropped off so quickly compared with the group.  He told me that the last tone he actually played was the 16k tone, and that he was just saying he played the higher tones for a laugh.

 

I guess that story doesn't do much for the subjective cause.

[Summit]

21 minutes ago, Kimo said:

 

You would think so, but that is not what I have experienced.  One time about dozen audiophiles all older than myself were mulling around in a shop and the owner, who was messing around with some test equipment asked us to raise our hands as he played test tones rising in response with each subsequent play.  As he ran through ascending tones, one fellow who admitted his hearing was damaged dropped out rather quickly.  I was next at 16k.  

 

The rest of the golden ears kept popping up their hands as he advanced well in the mid 20K region.  After the test I told him that I was surprised that my hearing dropped off so quickly compared with the group.  He told me that the last tone he actually played was the 16k tone, and that he was just saying he played the higher tones for a laugh.

 

I guess that story doesn't do much for the subjective cause.

 

We all know subjective impressions has its disadvantage. I started this thread to discuss how subjective impressions (although that) can be made better and more objective in the sense that the reviewed gear somehow can be categorized in how big the SQ difference is in relative terms and in the grand scheme of things.

 

Yes I believe most audiophiles and music lovers will tell it like they hear it.

[Ralf11]

The real problem with subjective impressions has been known for many decades: it is that many things (besides the sound itself) influence the perception of the sound, including visuals, knowledge of money spent, mood, etc. etc.

 

As a subjectivist, I find that one can use objective measurements to narrow down choices but not to decide between two or more units in the final group.  That requires a properly done blind test.  But those who want to buy based on esthetics, ergonomics, etc. can certainly do so.  I consider such things myself.  The problem arises when prper testing is not done and people claim they are free of all bias.

 

They aren't.

 

Besides that, there are a number of fads and fallacies in the audiophile world.  A Noise Fetish seems very au current... so to speak.

[sandyk]

3 hours ago, Archimago said:

The only way the above issue can be addressed is with blind-testing. Unless one can do that effectively, I believe it is essentially impossible not to be affected by these biases. The problem of course is that most subjective reviewers do not do this. And often will fight against those who suggest controlled listening tests. And even ban discussions, IMO of the most important procedural factor to add value to listening tests!

 Thanks for the good laugh .

 In my experience, the hard line Objective side will only accept the results of a series of correctly performed  Blind Tests when the results go the way they expect them to.

[plissken]

2 minutes ago, sandyk said:

 Thanks for the good laugh .

 In my experience, the hard line Objective side will only accept the results of a series of correctly performed  Blind Tests when the results go the way they expect them to.

 

There are a lot of disciplines that won't accept incorrectly performed testing. I know, right?

[sandyk]

2 hours ago, Summit said:

 

We all know subjective impressions has its disadvantage. I started this thread to discuss how subjective impressions (although that) can be made better and more objective in the sense that the reviewed gear somehow can be categorized in how big the SQ difference is in relative terms and in the grand scheme of things.

 

Yes I believe most audiophiles and music lovers will tell it like they hear it.

 A problem here, is that when an improvement,(or a degradation) is first noted, that the differences heard initially may be quite marked, but when revisited later, the differences although still remaining, are likely to be only minor, not HUGE differences .

[crenca]

5 hours ago, Kimo said:

Subjective impressions may be problematic in that they generally will reflect the listener's subjective impression of what a good system should sound like,  which may not align with your own.....If you do happen to find someone who shares your impressions in general, I believe that their subjective impressions can be quite valuable.  

 

4 hours ago, Archimago said:

On the one hand, the issue with subjectivism is about the language used

 

What I think is more important is the lack of "honesty controls" 

 

The only way the above issue can be addressed is with blind-testing.

 

 

I would say that a (the?) problem is the need for an objective (repeatable) subjectivity.  So IMO Kimo is correct in that an "alignment" of subjective descriptors/language would allow the audiophile consumer to read a review and correlate it with his preferences/expectations.  A standard use of language would be part of this.

 

IMO, some kind of further/advanced objectivity such as "honesty controls" would be nice but are largely unattainable in a niche industry/hobby.  That said, it would be nice if more folks would go this far with the controversial and nonsensical stuff such as voodoo encodings and digital cables...

 

edit:  rigorous "honesty controls" are difficult as they take a valid methodology, real time and more often than not real $money$.  Even in large establish industries/markets such tests/controls are only done on a small subset...

[sandyk]

11 minutes ago, crenca said:

That said, it would be nice if more folks would go this far with the controversial and nonsensical stuff such as voodoo encodings and digital cables...

 

Many members do not agree with you in this area, and furthermore you are highly unlikely to be ever able to disprove what they report hearing using current measurement techniques.

[fas42]

The 'problem' of audio is in fact extremely simple, if you use a certain perspective on it. What one should be after, IMO, is what's on the recording - the equipment, irrespective of how expensive or blingy it is, is always merely a means to an end. Of course, many people don't see it that way - the method of procuring what one is theoretically after is dominated by a deep fascination with the equipment used, and largely subsumes the nominal goal ... the photography person who is obsessed with lenses and camera bodies, the fisherman who agonises on precisely the perfect tackle to use - the fish caught is largely irrelevant.

 

If one can dump a fetish about "the means to the end" then all one has to worry about is whether one is getting closer to the sound event captured on the recording - worrying about differences is like obsessing about which is the 'right' shape of a camera body, to get a better grip of the thing ... I just want to know whether the apparatus falls into place, everywhere, in doing the job of allowing me to take great pictures, 🙂..

 

[Ralf11]

Just now, sandyk said:

 

Many members do not agree with you in this area, and furthermore you are highly unlikely to be ever able to disprove what they report hearing using current measurement techniques.

 

Many many qualified members do not agree with you in this area

 

It is easy to disprove impressionistic "reports"