Jump to content
IGNORED

Misleading Measurements


Recommended Posts

13 minutes ago, fas42 said:

 

How I detect noise is by listening ... it's trivially easy for me to make a tiny adjustment to the electrical environment of the home in which one is listening, and hear the variation in SQ. Whether one wishes to call this noise is up to the individual, but what it really says is that the playback chain is not sufficiently robust to reject this input - why you should want to measure such I don't quite see; but if you want to make it really obvious, in some numbers, just hook up, say, a working arc welder into a nearby socket - that will give you plenty of juicy data to work with, 😁.

Since my hearing has never worked beyond perhaps 22kHz, and currently isn't even that good, I find that using only my hearing to detect noise problems - (such problems not always being in the audible frequency range), isn't adequate in a lot of cases.   Being able to use an objective measurement of some kind -- often the direct measurement being further processed so that the details are more clear seems to be more effective when available.  Properly presented details about the 'noise' can be helpful to pinpoint even unforseen problems.   Sure, there are cases where a direct measurement with technology might be too difficult, but having objective/technologically aided measurements available as a primary means can eliminate missing lots of potentially non-audible impairments.   Out of band impairments can indirectly cause in-band audible problems.

 

I am definitely not against listening for problems, but even primarily depending on listening might miss some unforseen issues.

 

John

 

Link to comment
29 minutes ago, John Dyson said:

Being able to use an objective measurement of some kind -- often the direct measurement being further processed so that the details are more clear seems to be more effective when available.  Properly presented details about the 'noise' can be helpful to pinpoint even unforseen problems.  

 During the development  of my DIY Class A Preamp I used a CRO at maximum sensitivity with an inline very low noise battery powered 10 x Preamp to highlight potential noise problems, which in this case appeared to be related mainly to nearby SMPS powered devices. Attending to these very low residual noise levels did result in an apparent Subjective improvement.

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment
18 minutes ago, John Dyson said:

Since my hearing has never worked beyond perhaps 22kHz, and currently isn't even that good, I find that using only my hearing to detect noise problems - (such problems not always being in the audible frequency range), isn't adequate in a lot of cases.   Being able to use an objective measurement of some kind -- often the direct measurement being further processed so that the details are more clear seems to be more effective when available.  Properly presented details about the 'noise' can be helpful to pinpoint even unforseen problems.   Sure, there are cases where a direct measurement with technology might be too difficult, but having objective/technologically aided measurements available as a primary means can eliminate missing lots of potentially non-audible impairments.   Out of band impairments can indirectly cause in-band audible problems.

 

If the perceived sound changes, then there is an issue - the mechanism that allows this to happen may be hard to track down, and if the primary goal is to improve the SQ then IMO resolving that should be the first focus. In the long run a full understanding will of course be highly desirable - but a lot of energy may be wasted prior to that dealing with red herrings, simply because the latter may be easier to get numbers for.

 

Ultimately, what's going on when people complain that the SQ is not good enough will totally measurable ... we just haven't got there, yet 🙂.

Link to comment
6 hours ago, John Dyson said:

A few tweaks are okay, but tweak tweak tweak isn't instructive.  Studying a bit of technical background and learning why the attempt at 'design' requires so many tweaks, is MUCH MUCH more important.

 

Tweaking doesn't create learning -- ask Mr Edison about that.  I doubt that 100yrs of Mr Edison tweaking would have created Tesla's new ideas.   Tweaking is an intuititve physical activity -- it is only an adjunct to the more important learning.   Or, most ideally -- a finishing touch.

 

John

 

 

I think you may be being a bit bold about this John ! 🤣🙄 ^^^

Sound Minds Mind Sound

 

 

Link to comment
6 hours ago, sandyk said:
6 hours ago, pkane2001 said:

Because I tested one [Lush cable].

Other than a Subjective check  , are you able to post some of your measurements, including it's measured impedance, which is possibly the most critical for a high quality USB cable , which could illustrate why so many members favour this cable over most generic USB cables ?

 

Being somewhat OT, @pkane2001could these measurements be misleading?

Sound Minds Mind Sound

 

 

Link to comment
4 hours ago, Audiophile Neuroscience said:

 

I think you may be being a bit bold about this John ! 🤣🙄 ^^^

The reason why I do that is that I tend to blather, and try to bring out the more important notes.  Perhaps not needed in short messages.  I worry about being too wordy, but my language skills are VERY VERY VERY poor...   Just desperately trying to communicate information, not so much being pretty.   I have a lot of respect for peoples time.  (The only thing that EVER really held me back in my career was my ability to communicate, not so much technical and practical competency -- which was usually the strongest of my peers of ALL education levels.  Perhaps 1/3 of my co-workers had been PhDs or very advanced experts, and I was very often the 'answer person on advanced issues.  But, until perhaps 15yrs ago, I could JUST BARELY write a coherent sentence, and still stuck at the level.

 

John

 

Link to comment
10 hours ago, sandyk said:

 During the development  of my DIY Class A Preamp I used a CRO at maximum sensitivity with an inline very low noise battery powered 10 x Preamp to highlight potential noise problems, which in this case appeared to be related mainly to nearby SMPS powered devices. Attending to these very low residual noise levels did result in an apparent Subjective improvement.

I don't disagree with your abilities/skill, but here is my position:

 

The problem is that just using 'scopes in the traditional ways are not very selective in providing noise information.   They are okay at general information, but it takes spectral and other presentations to study what is going on.  I am NOT claiming that *aided*  measurements are the only way to find problems, but nowdays we have so many easy-to-use information sorting aids, there is no reason not to use them.  Here is one of my long, blathering anecdotes (off topic, but an exemplar):

 

Do you know how to make an amazingly good AM/MW/SW receiver, very simply, if you know what you are doing?   A very simple stable oscillator at approx 4X receving frequency, a fancy analog switch circuit with a few specially chosen *almost commodity* analog switches, a very simple, carefully laid out circuit, and a good, wideband (at least 96k, but 192k is better) 24bit stereo audio interface, and connect to your computer.  Maybe a slight amount of input selectivity is a little helpful and minimal input gain -- any analog RF amp is tricky design with low enough distortion not to make the recevier worse than the raw RF switching converter.  Perhaps use the RF amp as a buffer for radation, and impedance match from a short wire antenna.   (The switching device likes to work in the 50-150ohm range.)  (Direct conversion SW receiver, using audio interface, full demod capabilies AM/FM/SSB/digital/etc.)   The only real limiation is the base band bandwith of the composite signal limited to 1/2 sample rate.

 

This VERY VERY simple design will very often out-perform a very highly engineered, very fancy analog SW receiver.  With a little more work, can easily blow it away.   There is a whole series of very new (pretty much in the last 20yrs), innovative designs, both for the switching RF converter designs and even traditional RF mixer designs (A guy named Trask did some good papers on his super-innovative improvements to RF mixer designs at lower frequencies, and I forget the name of the person who designed this crazy-good, but simple SW level receiver concept.)   I think that Trask did both a brilliant switching mixer & lossless feedback derivative of the traditional MC1496 type scheme.  Both methods used real scientific and innovative thinking.

 

With the anecodote above -- which is wisest? -- to do a lot of hard-core, grating engineering/design to develop a retrogressive SW receiver, perhaps the biggest advantage is that it 'looks and feels' like a traditional SW receiver, or a relatively 'smart' design using an ingenious RF receiver that takes advantage of current technology, and doesn't have any of the IMD effects of a superhet design?   The question is rhetorical, because there is no real answer -- but a direct conversion approach tends to be very common nowadays -- eliminating huge chains of complex, hard earned circuitry.

 

I am not claiming that the new way is the only way to do it, but USING new technology opens up opportunities and gives more information for understanding what is going on.   Clear away the weeds, rather than deal with all of the weeds left over from the past.   Sometimes the old way is okay, but an 'oscilloscope', still being a super useful tool -- just measuring gross noise levels (unless some kind of information processing, like spectrum analysis) is doing it the hard way.   There be dragons, otherwise.

 

Using my current project as an example...  I am pretty sure that if R Dolby had the technology commonly available today, and designing the DolbyA, he would have made significant improvements over the eventual current design...  He was a genius, and did a wonderful job much better than I would have -- but he would have also caught some audibly noticeable (because of the demands of current recordings) flaws in the design.   What he DID do was amazingly good -- for an easily understandable example, I made the mistake of criticizing is FET/transistor combination gain blocks that he used -- until I analyzed them.  They work very well - but he was a special genius -- some of his design parameters are very counter intuitive relative to what a hobby person would probably do today.   I was super surpised with the performance of his little gain blocks.

 

I doubt anyone corresponding here today are at the innovative level of Mr Dolby, but the avialble technology aids tend to level the playing field for us, more average innovative pseudo-geniuses :-).

 

John

 

Link to comment
1 hour ago, John Dyson said:

I don't disagree with your abilities/skill, but here is my position:

 

The problem is that just using 'scopes in the traditional ways are not very selective in providing noise information.   They are okay at general information, but it takes spectral and other presentations to study what is going on.  I am NOT claiming that *aided*  measurements are the only way to find problems, but nowdays we have so many easy-to-use information sorting aids, there is no reason not to use them.  Here is one of my long, blathering anecdotes (off topic, but an exemplar):

John

 You are overstating the problems when  using established design techniques such as Analogue Preamplifiers and Power Amplifiers based on the research of highly qualified designers such as Nelson Pass, Bob Cordell and Douglas Self .etc. 

 Neither are most E.Es across most areas of Electronics like you appear to be suggesting.

Most EEs these days appear to specialise in a selected area. Very few E.E's these days are even capable of marrying together both the Digital and Analogue areas of a DAC in order to  create an exceptional performing DAC other than one  having perhaps good measurements that sounds O.K.

 Very few complete consumer type products, other than from a small company are likely to be the result of just one person..

I doubt that Mansr could achieve that either, despite all the claims he  used to make, as well as the vast majority of the members of  that other forum that several members here keep quoting measurements from as being definitive..

 

I do however have a couple of modest  E.E. friends who do know their own limitations outside the areas they mainly specialise in,  and would .not be capable of  the  know how to make an amazingly good AM/MW/SW receiver without a great deal of research and assistance.

 

BTW, you are putting Ray Dolby on a pedestal that very few designers other can aspire to..

Alex

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment
2 minutes ago, sandyk said:

John

 You are overstating the problems when  using established design techniques such as Analogue Preamplifiers and Power Amplifiers based on the research of highly qualified designers such as Nelson Pass, Bob Cordell and Douglas Self .etc. 

 Neither are most E.Es across most areas of Electronics like you appear to be suggesting.

Most EEs these days appear to specialise in a selected area. Very few E.E's these days are even capable of marrying together both the Digital and Analogue areas of a DAC in order to  create an exceptional performing DAC other than one  having perhaps 

good measurements that sounds O.K.

 Very few complete consumer type products, other than from a small company are likely to be the result of just one person..

I doubt that Mansr could achieve that either, despite all the claims he  used to make, as well as the vast majority of the members of  

that other forum that several members here keep quoting measurements from as being definitive..

 

I do however have a couple of modest  E.E. friends who do know their own limitations outside the areas they mainly specialise in,

 and would .not be capable of  the  know how to make an amazingly good AM/MW/SW receiver without a great deal of research and assistance.

 

BTW, you are putting Ray Dolby on a pedestal that very few designers other can aspire to..

Alex

I think that my diversions as examples might diffuse the importance of what I am saying.   Let me try to translate:  we have so many great tools nowadays, IF we use them, and not try to be biased like:  'my hearing is so good that I don't need to measure', or 'the old tools are good enough', maybe we can do things better, quicker, more optimally, etc.  Maybe even have more fun doing other things also.  More time to do new things because of less time wasted.

 

Maybe each of these wonderful claims of sufficiency are true, but sometimes also we over-estimate our own capabilities.  Sometimes, even we totally miss the mark.

 

More succinctly:   I don't believe in an objective vs subjective view of things.  However, relying too much on the subjective, and not taking FULL advantage of the available objective measurement tools is a bit anachronistic, and often a waste of time.  This is a corollary of my anti-'Tweak-tweak-tweak' stance.   I don't mean to say: NEVER 'Tweak tweak tweak', but instead why not take advantage of the WONDERFUL tools that we already have.   Aggressively avoid 'tweak tweak tweak' instead.

 

It isn't either or -- the reason for the Dolby diversion is simply because he did SUPER well considering the limited tools available.  He could have done better with the WONDERFUL stuff that we have today.*   I doubt that given the time frame of his early work, that he would have had the chance to do a computer simulation for a first cut optimization.  I could do a wonderful low noise pre-amp without computer optimization and without careful spectral distortion analysis for the results -- but why not take *full* advantage of current tools?   It isn't difficult to do so.

 

I try to be more self critical instead of being totally self sufficent, knowing that my hearing is 'good enough'.   I know that EVERYONE HERE has human hearing and human intelligence -- and I have known some of the most brilliant people that there are -- but sometimes being biased towards one or the other technique might make someone a little less productive and innovative than they could have been.  Some of the brightest people that I have known (and certainly one degree of separation from some of the VERY brightest) have also been stunted by needlessly set-in-stone opinions.

 

  * Today, R Dolby wouldn't have even needed to do his 'DolbyA NR', but just using him and his situation as an example.

 

John

 

Link to comment
4 hours ago, semente said:

 

There's an interesting discussion going on at ASR where Amir has against his and others' expectations found that a speaker which was favoured in a Harman study and produces beautiful Spinoramas sounds bad. Many are calling bias, so entrenched are they in their beliefs regarding audibility that they refuse to accept the obvious. They've been Tooled to accept that nothing other than tonal balance matters...

 

 

Good observation is good science. Ignoring observation is dogma.


That’s only a controversy if one expects one’s person subjective opinion to be used to judge how something will sound to anyone else.
 

Amir has previously, and even in that same thread, admitted to making many obvious mistakes when judging something subjectively, sighted. The only controversy in my mind is why anyone objectively minded would take that one sighted test by Amir as nothing more than his personal opinion that has little relevance to anyone else.
 

His measurements are testable and repeatable, his one sighted speaker test, not so much. At least not without investing a few million $$$ into a research study, similar to what Harman did. There’s a good paper on the extent they went to construct as an unbiased testing facility as possible to test with as many people as possible. Do you think that if Toole believed that one person can be accurate 100% of the time that he’d have constructed this facility and ran multiple experiments with 40+ people involved?

 

This is just a simple case of argument by authority when people start believing that Amir has some super-human powers. He doesn’t.

Link to comment
6 minutes ago, pkane2001 said:

That’s only a controversy if one expects one’s person subjective opinion to be used to judge how something will sound to anyone else.

 

I disagree. It's not a matter of how it sounds (taste/subjective) but of identifying shortcomings (observation/objective).

 

9 minutes ago, pkane2001 said:

His measurements are testable and repeatable, his one sighted speaker test, not so much. At least not without investing a few million $$$ into a research study, similar to what Harman did.

 

Like I said, I am ever more inclined to believe that corners were cut in Harman's research. Methodology, small samples, untrained listeners...

 

7 minutes ago, pkane2001 said:

This is just a simple case of argument by authority when people start believing that Amir has some super-human powers. He doesn’t.

 

I am not convinced that Amir is a particularly experienced listener but I find it perfectly possible for the Infinity to produce audible distortion(s).

"Science draws the wave, poetry fills it with water" Teixeira de Pascoaes

 

HQPlayer Desktop / Mac mini → Intona 7054 → RME ADI-2 DAC FS (DSD256)

Link to comment
8 minutes ago, semente said:

 

I disagree. It's not a matter of how it sounds (taste/subjective) but of identifying shortcomings (observation/objective).

 

 

Like I said, I am ever more inclined to believe that corners were cut in Harman's research. Methodology, small samples, untrained listeners...

 

 

I am not convinced that Amir is a particularly experienced listener but I find it perfectly possible for the Infinity to produce audible distortion(s).

 

You seem to be missing the point. One person's sighted opinion doesn't invalidate detailed, repeated, published and peer-reviewed scientific studies.  Even if that one person is Amir, or even if it was Toole himself. The fact that some people on ASR are calling Amir to task on this is the right thing to do. There's no controversy here, and Amir knows he can't defend his personal preference. Here's what he said:

 

Quote

The measurement data is provided for all. And developed and advanced by others. Listening results are mine because none of you can listen to it. I have offered to fix that by loaning out the speaker.

 

If you get anything out of my BIAS thread, you'll realize how many things can go wrong when biases are not properly controlled. Even if you're a trained listener with years of experience. There were no controls used in Amir's listening session, so why would anyone expect the result to be objective and rise to the level of useful evidence?

Link to comment
13 minutes ago, semente said:

 

Amir expected the speaker, which performed well in a Harman study, to perform but it didn't. Yes the test was sighted and Amir saw the measurements prior to listening but they're not better nor worse than other speakers' which he liked in the past.

 

Is it so difficult to conceive that the could be a problem?

Are you that biased against listening assessments that are not of the ABX type?

You put a lot of trust in the detailed, repeated, published and peer-reviewed scientific studies; I see scope for flaws and corner cutting and even commercially driven bias and personal taste.

 

Meanwhile a pier-reviewed paper about Covid 19 (hydroxichloroquine) published in none other than the Lancet and the New England Journal of Medicine magazines has just been found void and retracted...

Trust science, yes, but not blindly.

 

I'm not going to read into Amir's mind why he preferred one speaker over another. There are just too many possibilities, you seem to think that expectation bias is a simple process. It's not. Which is why, by simple Occam's razor, his one listening test is not enough to invalidate anything that was done in properly conducted, bias-controlled studies: there are just too many obvious and simple explanations for why he heard what he heard, totally unrelated to the actual differences.


Let me ask you, if the president of the USA claimed that hydroxchlorquine is the best treatment for Covid-19, would that mean that all the scientific studies that show the opposite are now suspect? Why would you trust Trump? Or why would you trust Amir?

 

The only reason I trust Amir on his measurements is because they've been replicated by many others, some by me. I know he's not making these up or fudging the results. He might make mistakes, but these are easy to validate and fix. His listening test cannot be verified or validated, and cannot be fixed. It is not objective, by definition.

Link to comment
2 hours ago, John Dyson said:

This is a corollary of my anti-'Tweak-tweak-tweak' stance.   I don't mean to say: NEVER 'Tweak tweak tweak', but instead why not take advantage of the WONDERFUL tools that we already have.  

You seem to understand yourself well, John. So you own up to and try to make the best of your difficulty with communication skills (even though you seem to communicate fine to me).  So you should understand how it is that many audiophiles believe they have no technical skills and are bewildered or even frightened by technology and the tools we find helpful and even fun to use.

 

It’s as hard for them to learn how, when and why to use these tools as it was for you to learn efficient, effective verbal & written communication.  I have a similar problem with foreign languages.  Sadly, despite a bachelors degree in chemistry, 2 doctoral degrees and an MBA, decades of traveling around the world and trying hard to learn (plus a wife with graduate degrees and a prior career teaching French, Spanish & Italian at the college level), I cannot get beyond basic greetings and questions to which I often don’t understand the responses.  Just as you are by communication skills and I am by other than my native language, many audiophiles have trouble with and are really put off by even a simple FFT.  Nevertheless, they deserve love too :) 
 

Maybe your scientific side could be happier with something like this for those who can’t or don’t want to go beyond playing around:  observe-document baseline-tweak-observe-document change-tweak-observe-document change-stop and review efforts & results-repeat until happy with SQ.

 

Thanks so much for your contributions! I‘Ve learned a lot from you and look forward to your posts.  Have a great weekend and stay safe!

 

David

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...