Why you can't trust measurements

[Allan F]

15 hours ago, kumakuma said:

“Against stupidity we are defenseless....{R}easons fall on deaf ears; facts that contradict one’s prejudgment simply need not be believed – in such moments the stupid person even becomes critical – and when facts are irrefutable they are just pushed aside as inconsequential, as incidental. In all this the stupid person...is utterly self satisfied and, being easily irritated, becomes dangerous by going on the attack..."

-Dietrich Bonhoeffer

 

As edited, the above is a perfect description of the public position of a majority of both elected members and supporters of a particular political party regarding a former POTUS, both of which shall remain nameless, of course.

[JoeWhip]

Do not forget the flat earthers, anti vaxxers, 5G conspiracy theorists, etc. 

[fas42]

10 hours ago, kumakuma said:

 

I'm having a hard time seeing how this relates to the subject being discussed in this thread. Can you elaborate? 

 

Groupthink is everywhere, in audio. Anyone who strays from the narrow thinking of the two fraternities, objectivists and subjectivists, is usually hounded out from the area, because the variation from the mindset of the majority there makes too many people uncomfortable - it can't be allowed to remain!

 

And here we see it again ... @Rexp points to a single article where there weren't measurements to support an audible difference - and what do we get? Not an interesting discussion of why this may be the case, because of the technical aspects of the particular situation; but rather the usual argy-bargy between the two camps of thought. And not a single, tiny bit of movement is made forward, in terms of better understanding of what matters ... 

[kumakuma]

58 minutes ago, fas42 said:

 

Groupthink is everywhere, in audio. Anyone who strays from the narrow thinking of the two fraternities, objectivists and subjectivists, is usually hounded out from the area, because the variation from the mindset of the majority there makes too many people uncomfortable - it can't be allowed to remain!

 

 

This is a convenient way to you to blame others for a situation of your own making. 

[fas42]

1 hour ago, kumakuma said:

 

This is a convenient way to you to blame others for a situation of your own making. 

 

A situation of my own making? ... Clarify, please.

[Allan F]

1 hour ago, fas42 said:

A situation of my own making? ... Clarify, please.

 

In legal speak, it would be called res ipsa loquitur and, consequently, no need for clarification. 🙂

[fas42]

 

1. The principle that the mere occurrence of some types of accident is sufficient to imply negligence.

    

    

   Yep, clear as mud ...

[Jud]

11 minutes ago, fas42 said:

 

1. The principle that the mere occurrence of some types of accident is sufficient to imply negligence.

    

    

   Yep, clear as mud ...

 

Simple. The doctrine originated with a case where the plaintiff was walking along the street and a barrel rolled out of an upstairs warehouse door and fell on him. He couldn't see what happened upstairs at the warehouse, but the ruling in the case was that he didn't need to.  If a barrel flies out of a second story warehouse door and falls on a passing pedestrian, it's evident that someone in the warehouse screwed up.  These days the principle is used in cases where, for example, you wake up after surgery with a big surgical clamp inside you.  You weren't awake to see it, but you didn't need to be - it's obvious someone has screwed up.

 

As for how this might apply to your situation, I haven't read the thread other than the last couple of posts, so I'll venture no opinion.

[Josh Mound]

Following up on the earlier discussion of reliability and review ethics:

 

I’m having a hard time seeing how a 30% failure rate for the ~140 Topping PA5 units bought by ASR member hasn’t caused Amir to update his review of the PA5, in which he raved about its superb engineering. Buyers have reported that the company has asked them to disassemble and fix the amp themselves and generally given them the runaround. Amir’s response has been to defend the company and assure ASR readers that it will fix things. Meanwhile, people are reporting that their replacement amps are failing, too. 
 

“It is abundantly clear that ton of great engineering has gone into the design of Topping PA5.” 🤭

 

 

 

 

 

 

[Confused]

6 hours ago, fas42 said:

 

Groupthink is everywhere, in audio. Anyone who strays from the narrow thinking of the two fraternities, objectivists and subjectivists, is usually hounded out from the area, because the variation from the mindset of the majority there makes too many people uncomfortable - it can't be allowed to remain!

 

And here we see it again ... @Rexp points to a single article where there weren't measurements to support an audible difference - and what do we get? Not an interesting discussion of why this may be the case, because of the technical aspects of the particular situation; but rather the usual argy-bargy between the two camps of thought. And not a single, tiny bit of movement is made forward, in terms of better understanding of what matters ... 

My posts were specifically trying to move away from the narrow thinking of the two fraternities, objectivists and subjectivists. 

 

My frustration is with Stereophile. They have such an advantage over us mere mortals posting on a forum. They have access to the actual equipment being discussed, they have a reviewer who states he can hear clear differences when a specific piece of equipment is used in the chain, they have a measurement specialist armed with an Audio Precision SYS2722. Yet all they do is publish a subjective review which states that the DAC being reviewed has "less resolution and timbral accuracy and created a spatially smaller, less lifelike sound" when the USB clocking device is not used in the chain, and then publish measurements that do not reveal why this might be the case.

 

Also note that when the DAC was used without the Master Time Black Star. it was being fed by a Sonore opticalRendu.

 

So lets take the subjective review at face value. Why did the Master Time Black Star make such a profound difference? A reduction in noise? Maybe, but this should be measurable. Also the opticalRendu is a relatively low noise device, the entire point of the device is to produce a low noise USB feed, with optical isolation of the Ethernet feed eliminating any upstream noise.

 

Maybe there is a benefit from the USB clocking? Although as @Superdad pointed out there are no details of what the clock actually is, so this would just be speculation.

 

We could of course speculate that very many other things created the audible differences, but it would be just that, more speculation.

 

We could also speculate that the reviewer imagined the audible differences or was otherwise mistaken. That too is just speculation.

 

My frustration is that Stereophile are happy to publish such stuff without making any apparent effort to did a little deeper when basically they are the only entity with actual access to the kit, a reviewer who can hear the difference, and measurement kit. Yet all they do is publish something almost deliberately aimed at fuelling the objective subjective debate. Frankly I think it is lazy and unscientific , from a publication that takes great pride in the fact that they stated publishing measurements decades ago.

[Confused]

I can think of more....

 

The review chain was a Sonic Transporter i5 running Roon Core, feeding an optical Rendu. In practical terms, most people would consider this to be a fairly decent digital "front end".

 

So what would the DAC sound like being fed direct from the Sonic Transporter? What if the Sonic Transporter was used to to feed the Master Time Black Star directly, taking the opticalRendu out of the chain? A few experiments like this this might provide some data as to exactly what was producing the profound subjective improvements that the reviewer could here, and give some insight into why. What if the DAC is fed from another different source, maybe something other than Roon? Could they not have tried something else?

 

But all we have is a review which basically states "with the $4k extra box in the chain I heard much better sound".

 

Then take a look at Ideon's own webpage. Even they can't be bothered to provide any technical data or reasons as to why this device improves sound quality:

 

https://ideonaudio.com/3r-master-time-black-star/

 

Although they do state this: 

Based on Ideon Audio’s proprietary technologies, dramatically improves the sound from digital audio computer files, and music streaming services.

It achieves amazing sound performance by uncovering lost detail, enhancing dynamics, re-clocking the signal using ultra-low jitter oscillators, and minimizing transmission losses.

[fas42]

35 minutes ago, Confused said:

So lets take the subjective review at face value. Why did the Master Time Black Star make such a profound difference? A reduction in noise? Maybe, but this should be measurable. Also the opticalRendu is a relatively low noise device, the entire point of the device is to produce a low noise USB feed, with optical isolation of the Ethernet feed eliminating any upstream noise.

 

Should be measurable - but what specifically is to be measured? And if a variation is seen, by some "new" method of assessing the performance, how can we be certain that there is strong correlation between these numbers, and what was heard? There may be some other difference, which is the crucial one as far as the subjective assessment is concerned, but it's completely missed. ...

 

35 minutes ago, Confused said:

 

We could of course speculate that very many other things created the audible differences, but it would be just that, more speculation.

 

Like the cables used ... I'm trying optimising a digital link between components for the first time ever, right now; and it's a can of worms! Okay, standard quality optical rather than USB, but the situation from reports I read is equivalent: how much does the precise way the cable is organised between any digital out and any digital in matter ... so far, the answer, for me, is a lot!

 

35 minutes ago, Confused said:

We could also speculate that the reviewer imagined the audible differences or was otherwise mistaken. That too is just speculation.

 

My frustration is that Stereophile are happy to publish such stuff without making any apparent effort to did a little deeper when basically they are the only entity with actual access to the kit, a reviewer who can hear the difference, and measurement kit. Yet all they do is publish something almost deliberately aimed at fuelling the objective subjective debate. Frankly I think it is lazy and unscientific , from a publication that takes great pride in the fact that they stated publishing measurements decades ago.

 

The problem is that to really, really do it properly the whole exercise would gravitate to a full blown scientific analysis; with every possible confounder evaluated. Which just won't happen. If lucky, some clues will emerge, to give buyers a bit of guidance - but I wouldn't bet on it!

[Confused]

38 minutes ago, fas42 said:

The problem is that to really, really do it properly the whole exercise would gravitate to a full blown scientific analysis; with every possible confounder evaluated. Which just won't happen. If lucky, some clues will emerge, to give buyers a bit of guidance - but I wouldn't bet on it!

The first step would be to establish if the reviewer really did hear the differences that he reported. As mentioned earlier in this thread, this would be very easy to do.

 

The above could be repeated with and without the opticalRendu in the chain.

 

Assuming this provided a positive result, this would provide some reason to explore further in the measurements.

 

None of this would be difficult.

 

I agree that we are never going to get a a full blown scientific analysis; with every possible confounder evaluated, but the Stereophile review and measurements offer nothing, when they could very easily have at least offered something.

 

Or put it this way, don't let perfection be the enemy of good. What Stereophile offered was not good.

[Rexp]

2 hours ago, Confused said:

The first step would be to establish if the reviewer really did hear the differences that he reported. As mentioned earlier in this thread, this would be very easy to do.

 

The above could be repeated with and without the opticalRendu in the chain.

 

Assuming this provided a positive result, this would provide some reason to explore further in the measurements.

 

None of this would be difficult.

 

I agree that we are never going to get a a full blown scientific analysis; with every possible confounder evaluated, but the Stereophile review and measurements offer nothing, when they could very easily have at least offered something.

 

Or put it this way, don't let perfection be the enemy of good. What Stereophile offered was not good.

@John_Atkinson did you take a listen, are the sound differences real and not measureable or?

[fas42]

14 hours ago, Confused said:

The first step would be to establish if the reviewer really did hear the differences that he reported. As mentioned earlier in this thread, this would be very easy to do.

 

The above could be repeated with and without the opticalRendu in the chain.

 

Assuming this provided a positive result, this would provide some reason to explore further in the measurements.

 

None of this would be difficult.

 

The industry has had decades to pull its finger out, and try and do things better. Do you think there is any chance that anything is going to change any time soon, based on how far things have progressed, to this point?

 

Enthusiastic audiophiles are seen by most to be in the same category as model train buffs - a peculiar bunch of people who spend lots of money on something that is of little relevance to anything else in life, apart from the pleasure of fiddling with things. No-one spends any time, energy, or money analysing the bigger picture in model trains - and the same applies in audio. Only companies who create products that are interesting to enthusiasts are making efforts; and of course this will be devoted to ensuring that their output is seen to be superior ...

[Confused]

7 hours ago, fas42 said:

 

The industry has had decades to pull its finger out, and try and do things better. Do you think there is any chance that anything is going to change any time soon, based on how far things have progressed, to this point?

 

I do not really have a view on this. 

 

Generally I tend to side with the words of Physicist  Niels Bohr "Prediction is very difficult, especially if it's about the future". I am sure we could have a thread on this forum where we all give our very many views on what the future will bring, a lot of it will be contradictory and most will be wrong. Maybe even this view will prove to be wrong?

 

Meanwhile I intend to travel through life, trying to make a positive impact with respect to anything that matters to me.

 

7 hours ago, fas42 said:

Enthusiastic audiophiles are seen by most to be in the same category as model train buffs - a peculiar bunch of people who spend lots of money on something that is of little relevance to anything else in life, apart from the pleasure of fiddling with things. No-one spends any time, energy, or money analysing the bigger picture in model trains - and the same applies in audio. Only companies who create products that are interesting to enthusiasts are making efforts; and of course this will be devoted to ensuring that their output is seen to be superior ...

 

An interesting view.

 

In my experience, when I tell most people that I am an Audiophile, they mostly tend to respond with something along the lines of "oh, I could never tell the difference between speakers or amplifiers or anything", or maybe they might say "do you like to listen to jazz then?" In general, I find that most people are mostly not remotely interested.

 

As a final point from myself. Can we try to keep things at least vaguely on topic?

[fas42]

5 hours ago, Confused said:

 

In my experience, when I tell most people that I am an Audiophile, they mostly tend to respond with something along the lines of "oh, I could never tell the difference between speakers or amplifiers or anything", or maybe they might say "do you like to listen to jazz then?" In general, I find that most people are mostly not remotely interested.

 

As a final point from myself. Can we try to keep things at least vaguely on topic?

 

To keep it on topic, why you can't trust measurements is that they don't tell you when a combination of gear gets into the zone when people who haven't the slightest interest in the game of audiophilia are turned on by what they're hearing. Things like, that the women come in and really, really listen - or start bouncing around the room, from the energy of the music. That means, that all the technical nonsense is no longer relevant - because the message of the recording is finally properly communicating ..

[kumakuma]

1 hour ago, fas42 said:

 

To keep it on topic, why you can't trust measurements is that they don't tell you when a combination of gear gets into the zone when people who haven't the slightest interest in the game of audiophilia are turned on by what they're hearing. Things like, that the women come in and really, really listen - or start bouncing around the room, from the energy of the music. That means, that all the technical nonsense is no longer relevant - because the message of the recording is finally properly communicating ..

 

You've picked something which is subjective ("in the zone") and argued that we should reject measurements as a whole because they can't measure this unmeasurable and highly personal characteristic of music.

 

By this logic, we should remove the nutritional information from food packages because they don't tell us how the food inside the package tastes.

[fas42]

9 hours ago, kumakuma said:

 

You've picked something which is subjective ("in the zone") and argued that we should reject measurements as a whole because they can't measure this unmeasurable and highly personal characteristic of music.

 

By this logic, we should remove the nutritional information from food packages because they don't tell us how the food inside the package tastes.

 

"In the zone" may be subjective to you, but not to me. IME, this occurs when the signature of the playback rig becomes effectively inaudible; and you are only consciously aware of the characteristics of the recording. And many high end rigs fail badly in this regard. To put it into context, a CD I bought over 30 years ago will present identically in all the key areas, on my current setup, as compared to what I used back then, which was radically different in its make up ... if either have any pretensions to accuracy, how can it be any other result?

 

Of course measurements are critical when designing and assembling components to sell; consistency, and weeding out the faulty, are essential here. But until more revealing metrics are readily available to the consumer, then the whole thing is a lottery; as an example, if you were given what was typically measured for a modern DAC, and no identifying information, would you have any idea whether it was a winner, or something most audiophiles would get rid of as fast as possible?

[botrytis]

1 minute ago, fas42 said:

 

"In the zone" may be subjective to you, but not to me. IME, this occurs when the signature of the playback rig becomes effectively inaudible; and you are only consciously aware of the characteristics of the recording. And many high end rigs fail badly in this regard. To put it into context, a CD I bought over 30 years ago will present identically in all the key areas, on my current setup, as compared to what I used back then, which was radically different in its make up ... if either have any pretensions to accuracy, how can it be any other result?

 

Of course measurements are critical when designing and assembling components to sell; consistency, and weeding out the faulty, are essential here. But until more revealing metrics are readily available to the consumer, then the whole thing is a lottery; as an example, if you were given what was typically measured for a modern DAC, and no identifying information, would you have any idea whether it was a winner, or something most audiophiles would get rid of as fast as possible?

 

Stop playing schematics here, Frank. That is all you are doing. 

[fas42]

Just now, botrytis said:

 

Stop playing schematics here, Frank. That is all you are doing. 

 

I have no idea what you're trying to say ...

[botrytis]

You can't something is subjective when most everyone else say it is objective. 

[kumakuma]

2 hours ago, fas42 said:

 

"In the zone" may be subjective to you, but not to me. IME, this occurs when the signature of the playback rig becomes effectively inaudible; and you are only consciously aware of the characteristics of the recording. And many high end rigs fail badly in this regard.

 

 

This is a highly subjective measure of SQ as proven by the fact that few agree with the (subjective) conclusion you have reached (sentence in bold).

[fas42]

1 hour ago, kumakuma said:

 

This is a highly subjective measure of SQ as proven by the fact that few agree with the (subjective) conclusion you have reached (sentence in bold).

 

Okay, if I listen to a couple of high end rigs, playing a specific recording, that I know well, and:

 

1) Their presentation of that is very different from each other, and very much from what I know of it

 

2) Each makes various aspects of that recording unpleasant, or sound downright distorted

 

3) Both miss making quite a few sound elements audible, or if they can be detected, they are very muffled and indistinct; lack being able to separate those parts of the mix from the whole

 

4) Completely fail to be able to throw up a convincing soundstage, with the sound elements clearly delineated within the whole

 

What conclusion should I come to?

[kumakuma]

1 hour ago, fas42 said:

 

Okay, if I listen to a couple of high end rigs, playing a specific recording, that I know well, and:

 

1) Their presentation of that is very different from each other, and very much from what I know of it

 

2) Each makes various aspects of that recording unpleasant, or sound downright distorted

 

3) Both miss making quite a few sound elements audible, or if they can be detected, they are very muffled and indistinct; lack being able to separate those parts of the mix from the whole

 

4) Completely fail to be able to throw up a convincing soundstage, with the sound elements clearly delineated within the whole

 

What conclusion should I come to?

 

You can continue to play "schematics" all you want but articulating the reasons for an opinion doesn't make it any less subjective...