The problem with subjective impressions

[Summit]

The problem with subjective impressions is not that they are made by listening and comparing different devices. At least not in my opinion. The problem is that when we describe the difference between diverse hi-fi stuff, we are often too subjective in our descriptions. By subjective, I mean that we often use exactly the same terms and adjectives to describe the difference, whether the difference is relatively small or if it is more significant.

 

Now, I'm pretty sure some people disagree with me on this, and think that they describe their subjective observations in an objective way. Okay there are exceptions, of course, but I would say they are in the minority.

 

I think that no matter what parts of a hi-fi system, placement and the room itself we believe are most important, it is often difficult when reading reviews or impressions to understand the reviewer's gradation difference between a little but observable - to that sounds like a completely different hi-fi system. If you do not already know from your own experience that the speakers and their placement in the room almost always have a much greater impact on the sound quality than a power cable, frame memory or a switch, it can be difficult to interpret how much impact we are really talking about. Now, I think most people here know the difference I described, but maybe not between all other parts of our hi-fi chain.

 

Anyway I've read a large number of reviews and impressions and BELIEVE that the “problem” is because we often describe subjective differences too subjective and in more or less the same way regardless of whether we describe more significant differences from two completely different types of amplifiers or two external power supplies of the same type. What I want to say is that although I think there are relevant differences between different cables, hard drives, PSUs, external clocks and so on, but because of how we usually describe our observations in the same way, it is often difficult to know how big a difference it actually is at the system level and compared to other upgrades we can do for the same money.

 

I am all for subjective observations, but think that we can all benefit if we can get a little more objective in our descriptions in our reviews and impressions. I have also observed that the reviews that are more moderate written do not become as "popular" as those where it seems that the difference is of a more revolving nature. Yes, I actually think there has been a steady inflation of superlatives generally in the hi-fi world, and who really benefits from that? Not one of us I would think. If someone buys something and believes now my audio system will sound superb, but does not, well then the person can be burnt and not captivate observations describing the more significant differences of other gear, because the exact same superlative has been used, and reused. What I wish for is that we all would try to describe the subjective differences more nuanced and in a more objective way and not as if all parts of the audio chain play the same role for SQ, although of course many small individual differences together make more significance.

 

The question is how can I / we describe the difference between devices in a detailed way but still not make it sound like the difference is bigger than it really is?

 

I know that some reviewers wait several months before writing and publishing their review. Maybe we can get a better perspective on the actual sound difference that way? Less FOTM and talk about burn in.

 

TD; LR I think that we would all benefit on some sort of categorizing of how big the difference is in relative terms and in the grand scheme of things instead of as it was an isolated part of the audio chain.

 

And rant

[Summit]

1 hour ago, Kimo said:

Subjective impressions may be problematic in that they generally will reflect the listener's subjective impression of what a good system should sound like,  which may not align with your own. 

 

For example, I have a strong dislike for almost all metal dome tweeters to the point that I find most systems difficult to listen to when I first sit down with them.  They typically sound harsh and flat to me.  I am aware that the Raal ribbons I prefer probably don't measure as well as some dome tweeters, but they also don't annoy me.  For those who don't share my affliction, the metal dome may be the superior choice, and my subjective view isn't worth squat to them.

 

If you do happen to find someone who shares your impressions in general, I believe that their subjective impressions can be quite valuable.  

 

Yes it’s true that subjective impressions generally will reflect the listener's subjective impression and preference, it is why they are subjective. I am not talking about preference per se, but that most reviews and impressions are written the same and with the same words. I mean you can almost take any review and how the gear they liked the most is described and just change the name of the gear for another gear and voila. The problem as I see it is this type of review will not reflect the SQ of the gear in the grand scheme of things.

 

Even if a good USB cable for example is important for obtaining really good SQ I would not say that the difference from one good USB cable to better USB cable to be as big as the difference between a good amp and a better amp. I think that it would be great if we would get some sort of classifying of how big the difference is in relative terms (1-10), so that our subjective listening impressions would be more objective. Not objective as in measurement or DBT, but more like how big was the overall SQ effect by changing gear A to gear B in their reference audio system. Most impressions focus too much on the reviewed gear IMO and in the effort to describe the gear in depth we may not know how big the upgrade/change really is as part of a system and together with the other equipment in the audio chain. Some does, but most not.

[Summit]

4 hours ago, STC said:

I think the problem with subjective impressions is they are mostly made up to justify their choice of purchases.  
 

I grew up with a reasonable collection of good music and it is always been about music. I remember the gramophones, TT, cartridge, cassettes and CDs. It was never about equipments.   
 

Audiophiles created a non existent problem and then attempted to remedy them. A music lover only needed clarity and loudness. They use treble and bass to address room acoustics and placements. It worked and works well for 99% of the population. 
 

Then they wanted something better without understanding what they want. They buy, say a powerful amplifier, it didn’t turn out to be the best purchase. Without treble and bass, the sound wasn’t what they were accustomed too. Then after few amps they change the speakers. Then they spend hours on speakers position. Then they realize they need to to treat the room to address the bass boom. And the the vicious cycle continuos....

 

The hobby from getting a better system to enjoy the music now becomes the hobby of listening to the equipments. Then the time will come where your system will be talk of your friends. You by virtue of having a more expensive system naturally will become an expert. Your subjective words than becomes another person’s reference. 
 

Slowly you go out and listen to other systems. Some of them going to sound much better than yours. That persons opinions and subjective description of sound quality now becomes your guide. The vicious cycle continues....

 

Audio is not difficult to understand. Good sound production is not complicated. Not one person in the world would argue that the sound of the 60s cinemas were better than current cinemas. But we have audiophiles still believe a 60s amplifiers or speakers supposedly to be the holy grail of the ultimate SQ. 

 

I do not share your belief that subjective impressions mostly are made up to justify choice of purchases.

 

I believe it’s true that in the effort to get better and better audio system to enjoy the music it can become the hobby of listening to the equipment.

[Summit]

8 minutes ago, Kimo said:

 

I don't think that we are going to be able to set an objective scale for subjective impressions for much of anything in life.  How much hotter is vintage Jennifer Garner than vintage Pam Anderson?  We are still stuck with subjectively even when talking about different kind of 10s.   

 

 

No the scale is subjective, but reflects the importance of the audio gear in the audio chain and not only as one individual equipment.  

[Summit]

14 minutes ago, Kimo said:

 

I would agree with you on the importance of amps and speakers, but others would not.  I know people that will tell you that a good preamp is the foundation of great sound in any system, and others that would say no preamp is the best preamp.

 

I am all for people having different thought and subjective preference. To make ones preference known is the best way to share one’s subjective impressions. To be objective is to reduce subjective factors to a minimum, but we can never eliminate them, and most people will tell it like they hear it. Preamp or not. 

[Summit]

21 minutes ago, Kimo said:

 

You would think so, but that is not what I have experienced.  One time about dozen audiophiles all older than myself were mulling around in a shop and the owner, who was messing around with some test equipment asked us to raise our hands as he played test tones rising in response with each subsequent play.  As he ran through ascending tones, one fellow who admitted his hearing was damaged dropped out rather quickly.  I was next at 16k.  

 

The rest of the golden ears kept popping up their hands as he advanced well in the mid 20K region.  After the test I told him that I was surprised that my hearing dropped off so quickly compared with the group.  He told me that the last tone he actually played was the 16k tone, and that he was just saying he played the higher tones for a laugh.

 

I guess that story doesn't do much for the subjective cause.

 

We all know subjective impressions has its disadvantage. I started this thread to discuss how subjective impressions (although that) can be made better and more objective in the sense that the reviewed gear somehow can be categorized in how big the SQ difference is in relative terms and in the grand scheme of things.

 

Yes I believe most audiophiles and music lovers will tell it like they hear it.

[Summit]

13 hours ago, crenca said:

 

 

I would say that a (the?) problem is the need for an objective (repeatable) subjectivity.  So IMO Kimo is correct in that an "alignment" of subjective descriptors/language would allow the audiophile consumer to read a review and correlate it with his preferences/expectations.  A standard use of language would be part of this.

 

IMO, some kind of further/advanced objectivity such as "honesty controls" would be nice but are largely unattainable in a niche industry/hobby.  That said, it would be nice if more folks would go this far with the controversial and nonsensical stuff such as voodoo encodings and digital cables...

 

edit:  rigorous "honesty controls" are difficult as they take a valid methodology, real time and more often than not real $money$.  Even in large establish industries/markets such tests/controls are only done on a small subset...

 

Interesting and the issue I wished to discuss. Can you elaborate on what you mean by “the need for an objective (repeatable) subjectivity” and how this can be done?

 

I believe that subjective impressions can be good and useful, but can be made much better and less subjective in how they are conducted and how they are reported. I see very few discussions on how subjective evaluations can be made better and more standardized. I think that’s unfortunately when most audiophiles select their equipment, or at least which to listen to, based on those subjective observations. If for instance the measurements we use and know isn’t capturing what we wishes to measure (Higgs particle for example) we don’t just rest, we will try to find a way to measure it. I don’t get why not more effort is made to get more reliable sound quality evaluations by using our ear isn’t a subject that is evolving. It even has gotten worse IMO.  

 

Everyone, including die hard subjectivists, know that impressions has many disadvantage. Some are almost impossible to do much about without control groups and blind tests, others I believe can be improved rather easily if we wanted to.   

[Summit]

27 minutes ago, semente said:

For me, building or developing a higher fidelity system is a long-term project which results from identifying shortcomings in the different components/equipment and dealing with them.

This requires a combination of observational listening and measurements.

Put a new equipment in the system, listen for a week or two then go back to the original setup for a bit. This will highlight the differences. In my opinion and experience short duration A/B comparisons are mostly worthless.

 

Observational listening is not tasting. To listen critically one must focus on the shortcomings. It's easy to lose track (of sound) and start enjoying a new presentation of your recordings (music).

 

Most commercial reviews are little more than tasting.

 

How do you identify shortcomings in different components/equipment in an audio system by listening?

 

How are you dealing with the shortcomings that has been identified? like @fas42 .

 

Can you show me a review that you have made?

 

[Summit]

On 12/31/2019 at 11:57 PM, crenca said:

 

 

TLDR answer:  I am not exactly sure

 

I have found folks who do reviews that have an "objective" subjectivity - they use language consistently and they have enough consistency in what they hear to use comparative/descriptive language such that I know what they mean when they say "this transducer is 'warm' but has greater 'detail/transparency' than this other transducer".  @JoshM, @firedog, and to a lesser extent @The Computer Audiophilehere for example are examples of folks I have been able to correlate my experience with and understand what they mean.  Tyll Hertsens of Inner Fidelity (now retired) was part of the way there, but for some reason at times he would deviate (in other words, a handful of his reviews did not seem consistent with the majority to my ears).  There are some folks over at SBAF whom are consistent, and SBAF consciously promotes this consistency and use of language, but then there are some folks over there whom seem to deviate as well.  

 

I have noticed that those who do this, or even just try to do this, are not "radical" subjectivists nor are they "radical" objectivists.  They to a person seem to have a balance and realism when it comes to electronics and engineering (and are able to admit voodoo when they see it), yet are also able to admit that gross measurements such as THD+N is not the sum total of the differences between equipment...

 

I have also found that I prefer reviewers that use well-known sound quality descriptions and are consistent in their observations and reviews about what they hear. Though it takes time and many reviews to get to know a reviewer and how he/she use those well establish words/terms to describe various sonic alterations, and remember that it is easy to mix those reviewer up. How many reviews does it takes before I will know if I can trust and relate to the reviewers sonic view and wording for describing what he/she has heard subjectively 5, 6, 7 or 10?

 

Would we use recipes with very vague and subjective measures? No, small can sometimes be 5 gram and in another recipe it can be 20 gram.

 

Even if I get to “know” and trust a reviewer so much that I will take their impressions as of any real value at all we still have the problem of gradation.  What I mean and think is a big problem is that we lack ways to describe how big or small one identified sonic alterations really is in one gear VS another gear. What am trying (hard) to explain is that besides that we need universal terms/vocabulary to describe different sonic attributes, like warm, full, transparency, rich, harsh and so on, we also need a way to quantify those difference better. I have read many impressions and reviews and found that that even the best ones use just a few different words/adjectives to quantify the difference.

 

We use such subjective wordings as the DAC A had a clearly more harsh upper minds or the amp had more grunt and better low bass punch. How much is more or clearly more? I mean compared to what reference, and are we only comparing to other DACs or the final sound? I would like to know how small/big the difference is in reality as well.

 

To me it doesn’t matter which quantifying adjective we use they are always very vague and subjective. It can help if they compare the reviewed gear to another similar audio gear, but it is uncommon that the reviewer also tell how big the SQ difference is in the grand scheme of things. Can the much more transparent DAC for example be describes as 2 or 4, if the gradation could be everything from 0-10 (0=no difference 10= night and day difference)?

 

Yes the gradation is subjective and not truly objective, but with them we would at least know how big/small the sonic difference the reviewer think they have and doesn’t need to guess about that. Today it is possible to write a very long review where every word has to be interpret, and often in a context which is not well known if you haven’t read many other reviews by the same author and you also know how the rest of his gear in his reference audio system sound like.   

 

TL; DR Good reviewer that use well-known sound quality descriptions consistently and that compare the SQ characteristics to other well-known gear, and that at least are trying to be objective about how big/small the impact is in their audio chain (preferably by gradate the influence they have).  

[Summit]

16 hours ago, The Computer Audiophile said:

Interesting rant and one that I don’t necessarily disagree with but obviously don’t agree with all points. 
 

When I write I am always cognizant of which words I use. If I say something is the best, it must be the best or I’ve just mislead people. I frequently use the terms good and great to describe two different levels of quality. There are gradations within these as well unfortunately. 
 

I think subjective writers must understand they can’t please everyone with their style and choice of words. I write to please the mythical reader that is myself. In other words, would I like the review and get something out of it if I was in the reader’s shoes. For the most part this goes a long way as opposed to writing for other people or trying to please others by writing what I think they want to read. 
 

 

 

Thank you for responding on the topic of this thread. I don’t know how reviews should be written, only how I like them. I agree that not everyone can be pleased. To me the problem is not the choice of words for describing SQ, the problem is to interpret the scale of those sound performance and the value of the product. By quantify in numbers in the summery, like in the end of those two linked reviews I believe it is easier as a reader to know the reviewer’s verdict than if the same is only written in text. It is also easier to directly compare two gear that has been reviewed by the same auteur if the verdict of some of the parameters are quantified somehow. The points is of course always subjective, but with them we would at least know how the reviewer quantify different SQ aspects and don't need to interpret how someone use the words like better, good, great and so on. I think that the value aspect of audio gear is very important and something I would like all reviews to reflect about more.

 

 

https://www.stereolifemagazine.com/reviews/item/1333-klipsch-heresy-iii-70th-anniversary-edition

 

https://dekoniaudio.com/dekoni-blue-featured-in-the-hifi-ultimate-headphone-guide-2018/

[Summit]

16 hours ago, Allan F said:

 

I don't know that there is an answer to this question. Part of the problem is that we all don't necessarily hear the same way and what I may experience as a subtle difference, you may experience as a major one. A possible answer is to simply describe the difference as audible, and limit the detailed description to the nature or characteristics of the difference as opposed to its magnitude. But that makes it more difficult for someone to decide, based on the description, if the difference is significant enough to justify the expenditure of changing the device.

 

Ultimately, I suppose one can only use these experiences as guides to whether a device is of sufficient interest to investigate further.

 

Thank you for responding on the topic of this thread. I only use the reviews as guides to whether a device is of sufficient interest to investigate further. The problem is that we have so many audio gear nowadays and I feel that many reviews are too vague for me to get a good grip of the actual SQ performance. I like a review to describe the sonic characteristics of the gear as well as the scale of some of the performance aspects.

[Summit]

5 hours ago, Samuel T Cogley said:

 

As long as we're getting all philosophical, the universe doesn't have a particular need for humanity.  Or to paraphrase George Carlin:

 

"Save the planet? You mean save the people!  The earth will just shake us off like a bad case of fleas.  The earth is fine."

 

I always look at this hobby through the lens of consumerism.  Audiophilia can't be more virtuous than consumerism, because it is consumerism.

 

By "saving the planet" we do not only mean humans, as there are many other organisms on earth that is much less adoptable to environmental changes.

[Summit]

11 minutes ago, Samuel T Cogley said:

 

Sure, but typically, "save the planet" for most people means "save the humans" was my point.

 

If we lose a lot of organisms and if many ecosystem would decline or collapse the humans and our society will be affected negatively as a result.

[Summit]

6 minutes ago, The Computer Audiophile said:

 

Thanks for the links and graphic. 

 

I'm glad you found some information that you like and that suits your needs. 

 

For the most part I dislike these score based assessments. They often lead consumers to a score race like a specs race. If one headphone gets a 7/10 for Resolution and another gets an 8/10, consumers with nothing else to go on will select the 8/10 thinking it's better. With so many variables involved there needs to be a plus/minus 3 for every score.

 

Anyway, thanks again for the graphic. 

 

I agree that numbers can result a "score race" and have seen it may times. Hifi choice is good example on that. Still the same is true for reviews there the scale is only described with words IMO.

[Summit]

10 minutes ago, The Computer Audiophile said:

Put another way, if all the bugs die, all the people die. If all the people die, all the bugs will be just fine :~)

 

More or less, but I do not which for all humans to die and think we can coexist .