Misleading Measurements

[opus101]

I'd not point to 'qualifications' myself as those have the same status as numbers in my estimation😄

[pkane2001]

55 minutes ago, opus101 said:

I'd not point to 'qualifications' myself as those have the same status as numbers in my estimation😄

 

More like tongue on the battery terminals, in my opinion ;)

 

[John Dyson]

7 hours ago, sandyk said:

I agree, and John may not be aware that you are also well qualified technically, not just anotherSubjective type.

Sometimes, there are people who are technically oriented and enjoy tweaking -- I don't.

In order to move technology forward (rather than copying schematics), there isn't much room for wasting time -- therefore use objective as the driver, and subjective as the cross check at best.  (Subjective does need the stats wrapper.)

[John Dyson]

9 hours ago, sandyk said:

 It hasn't always worked out that way in your PM group though, has it ? 😉

I will leave it at that, and not reply further, as we are so far off topic already. with virtually no poster  giving OBJECTIVE illustrations , whether by later refuting measurements or the results of DBT sessions that showed the measurements in the example given were misleading instead of the philosophical discussions  currently  .

 

 

  

My focus is on informing about the benefits of avoiding short term laziness, and spend some time doing what is needed for the longer term benefit of proper stats procedure and objective measurement.   In the longer term, the investment in more structured approaches will most often give a great payback.

 

Answering comments about my very necessary usage of the subjective (but I imposed controls that might not have been obvious):

 

My opinion about the weakness of subjective input is re-enforced from the PM group.   Sometimes subjective input is the only way to do things as I had noted before, but not very efficient and VERY noisy.   I counsel people on the best approach, but have been greatly frustrated by the statistical quality of subjective opinions.   It isn't to say that the subjective is to be ignored, but the data signal is weak and the noise is too strong to depend solely on unprocessed subjective inputs.   After some simple experimental controls, my opinion of objective data has recently been further diminished.  Unfortunately, I have necessarily spent lots of time with chasing rabbits -- and I suggest NOT to do that.   Relying too much on the subjective, esp without proper experimental controls, is tantamount to technological masochism.   I have sometimes needed to use the subjective on my project that had a similar effect as an act of self-flagellation and gotten the same results -- misleading.   All I can say -- only use inferior data sources like the subjective, if you really must.

 

If I wasn't pushing the bounds of what had been done before, then less subjective input would be needed.   We have learned a lot through the years about what needs to be measured on simple amplification and transducer equipment (well -- transducers aren't always simple, but we understand them very well.)   Dynamics processing is trickier to understand WRT impairments.*   Things can be very different and  subjective, noisy sources might be needed for verification, esp when chasing something WITH NO SPECS AT ALL.

Even  a very complex control system -- there are often parameters where the measurement of typical complexity needed for a linear system would not produce adequate input.  A well defined complex system could be measured (certainly avoiding the dependency on the subjective) by proper data processing and reduction.  I am afraid that some people might choose the subjective for a very near-sighted shortcut, but long-term costly decision.

 

* the linear processing is such a small part of the project, but still the HW variant can benefit from the well known techniques just the same.   If working in the HW domain, would definitely use the more reliable objective than just being a tweakabilly designer.

 

For the dynamics processing, unlike most compressor/expanders, the attack/release is multiple continuous nonlinear functions, and combinations of complex dynamics controls.  The dimensionality of any quality measurement on my project is huge.   Typical linear systems like preamps, amplifiers, even tuners have much less complexity than my project just wrapping up.   One thing for sure, I trust the quality of any attainable objective and controlled measurement much more than any subjective results that I have measured or have  been provided.   There is NO-WAY that the subjective could have given accurate response curves as created from my objective test materials.

 

For most anything done that IS NOT pushing the boundariess of technology, then take the very wise short cut, and use the objective technological means and data reduction methods that you have available, then go to the subjective as a cross check (make sure there isn't something obvious that you have over-looked.)   Subjective review is great at finding obvious errors, but I have even found that to have real troubles with errors and bias.

 

John

[manueljenkin]

Again, our friend John, is trying to push something as end all be all, when it is just him speaking for himself. I'm sorry John, if the test results felt conclusive enough for us we would have actually embraced it. I make a significant portion of my electronics purchases - say a display, or a camera, or a lens with a lot of weight to measurements. And when I say measurement, I refer to a rigorous analysis as done at anandtech, or notebookcheck, and for cameras I would also put some weight on video/camera samples from early adopters and users at Vimeo/flickr. Unfortunately, the measurement at ASR is not at all rigorous.

 

I do have a few other places which I do trust a little bit for measurements. Goldenears used to be a great resource for headphone measurements, measuring true impulse responses and CSD. Today I find, this page to be a valuable source of information : https://reference-audio-analyzer.pro/en/report/hp/shure-srh-1840.php . It is not conclusive, but it sure is not misleading. All you see is graphs, not one person's opinion or interpretation at the end. They also measure amplifiers, but by a very simple metric yet measure certain things that ASR doesn't measure. They test it across various loads, and actually index it into equivalent class A, Class B, Class D performance (though I don't like this tag, since implementation >> buzzword, it gets the job done). Again it's just numbers and is good enough to know if the amp is suitable for my use case or not (and not a gauge for absolute performance relating to audibility). High oi, ok it's fine if my headphone is a flat impedance profile or very high impedance, provided the amp has other parameters straight. Now this is not a conclusive test either but there is no personal agenda like at ASR where they just correlate SNR to be an end all be all index when in reality that claim is unfounded. The amps that they tend to show in bad light at ASR are still well above the known audibility levels in those indexes. And personal opinions like "bad engineering" or whatever, when it really is not, it's just not overoptimized for the numbers ASR cares about. They are optimized for few other things that have been experienced to be of better sq perceived by humans, but not yet conclusively correlated, that's all.

 

And a lot of have experiences where what did not show a change in ASR still showed meaningful change in sound quality.

 

[manueljenkin]

On 11/18/2020 at 6:41 PM, pkane2001 said:

 

I can use my tongue on the battery terminals to get a subjective observation of the remaining charge, but a voltmeter is going to produce a much more useful and accurate "observation". As long as the measurement device is properly calibrated and the error margins are known, there's no reason to keep relying on the difficult to perform and hard to validate subjective results.

What a wonderful straw-man hat argument as usual from @pkane2001 . A battery is used to power up a device. It's the devices requirements and correlation indexes that matter. (Unless you're a bot running on battery power 😅). And we have fairly established results correlating to the static/dynamic and voltage/current requirements of most loads, or atleast to the accuracy of what we want the load to perform at.

 

An audio reproduction system is used to cater to a human hearing system (ears+brain). So the correlation indexes that matter should come from humans. And it's not well defined yet! So you can't conclude using tests from ASR as conclusive means. End of story. 

[pkane2001]

7 minutes ago, manueljenkin said:

What a wonderful straw-man hat argument as usual from @pkane2001 . A battery is used to power up a device. It's the devices requirements and correlation indexes that matter. And we have fairly established results correlating to the static/dynamic and voltage/current requirements of most loads.

 

An audio reproduction system is used to cater to a human hearing system (ears+brain). So the correlation indexes that matter should come from humans. And it's not well defined yet! So you can't conclude using tests from ASR as conclusive means. End of story. 

 

I think you missed the point of that analogy. But kudos on using the "straw-man hat" attack, though. 

[manueljenkin]

There's a fine line between a useful analogy and a straw-hat argument. Unfortunately yours was on the other end.

[pkane2001]

5 minutes ago, manueljenkin said:

There's a fine line between a useful analogy and a straw-hat argument. Unfortunately yours was on the other end.

 

Seems you totally missed the context. Oh well. But funny how everything winds up being about ASR.

 

Look, produce something interesting and objective for us to discuss, something other than your opinion or I'm simply going to stop responding to you.

[fas42]

And here's a thread on ASR, from yesterday, that absolutely nails the ugliness of the people who post there ... https:// www.audio “science” review /forum/index.php?threads/paul-says-you-must-lift-speaker-cables-of-the-floor-for-better-sound-quality.17681/

 

Paul from PS Audio says very matter of fact things about the cable lifting question - but that doesn't stop the crowd from lamming in, with full force - all doing a Trump, hurling abuse at the spawn of the devil icon who has been evoked ... and some here wonder why there is a 'thing' about the ASR crowd ... 😁.

 

[pkane2001]

25 minutes ago, fas42 said:

And here's a thread on ASR, from yesterday, that absolutely nails the ugliness of the people who post there ... https:// www.audio “science” review /forum/index.php?threads/paul-says-you-must-lift-speaker-cables-of-the-floor-for-better-sound-quality.17681/

 

Paul from PS Audio says very matter of fact things about the cable lifting question - but that doesn't stop the crowd from lamming in, with full force - all doing a Trump, hurling abuse at the spawn of the devil icon who has been evoked ... and some here wonder why there is a 'thing' about the ASR crowd ... 😁.

 

 

Yawn. Boring, Frank! Got anything interesting to share? Like some objective evidence that cable lifters produce audible differences, perhaps?

 

[sandyk]

32 minutes ago, pkane2001 said:

 

Yawn. Boring, Frank! Got anything interesting to share? Like some objective evidence that cable lifters produce audible differences, perhaps?

 

 

Where does that  fall into the topic of Misleading Measurements when there hasn't been any posted or linked to ? 

[pkane2001]

7 minutes ago, sandyk said:

 

Where does that  fall into the topic of Misleading Measurements when there hasn't been any posted or linked to ? 

Keep up with the topic, Alex! It’s evolving rapidly ;)

[fas42]

33 minutes ago, pkane2001 said:

 

Yawn. Boring, Frank! Got anything interesting to share? Like some objective evidence that cable lifters produce audible differences, perhaps?

 

 

Nicely edged out to slips 🙂 ... okay, I've got something better - do you reckon it would be possible to get objective evidence that having a working, heavy duty arc welder plugged into a socket next to that feeding an audio system could produce audible differences?

[pkane2001]

1 minute ago, fas42 said:

 

Nicely edged out to slips 🙂 ... okay, I've got something better - do you reckon it would be possible to get objective evidence that having a working, heavy duty arc welder plugged into a socket next to that feeding an audio system could produce audible differences?

I got one, eh, two. They do make a very nice audible difference. Will arc weld all day long into a short. Made by Nelson Pass.

[fas42]

7 minutes ago, pkane2001 said:

I got one, eh, two. They do make a very nice audible difference. Will arc weld all day long into a short. Made by Nelson Pass.

 

Nicely edged away again ... always easier, if one doesn't answer the question ... 🙃

[pkane2001]

Just now, fas42 said:

 

Nicely edged away again ... always easier, if one doesn't answer the question ... 🙃

That was a direct answer. Maybe not the one you were looking for?

 

[opus101]

2 minutes ago, fas42 said:

always easier, if one doesn't answer the question ... 🙃

 

You ain't seen nothin' yet - just wait 'til the ducking stool comes out.....😄

[sandyk]

18 minutes ago, pkane2001 said:

I got one, eh, two. They do make a very nice audible difference. Will arc weld all day long into a short. Made by Nelson Pass.

 

 Please stick to the topic of the thread as you expect others to do.

I am genuinely interested to read about any measurements that were found to be misleading, and were confirmed by more recent measurements or DBT sessions to do so.

[fas42]

6 minutes ago, pkane2001 said:

That was a direct answer. Maybe not the one you were looking for?

 

 

The reference was to, how do we know that an audio system is 100% impervious to all types of electrical interference that can manifest in the area - just say, "The engineers who designed this should know what they're doing!", perhaps ... ?

[pkane2001]

4 minutes ago, fas42 said:

 

The reference was to, how do we know that an audio system is 100% impervious to all types of electrical interference that can manifest in the area - just say, "The engineers who designed this should know what they're doing!", perhaps ... ?

Umm. Measurements, perhaps? Just a guess on how any engineer would approach it.

[fas42]

Well, I've just discovered something quite juicy, https://en.wikipedia.org/wiki/Cucking_stool - got to read the bit about the origin of the name, 😉.

 

Now, where can I order one, over the Net ...

[fas42]

Just now, pkane2001 said:

Umm. Measurements, perhaps? Just a guess on how any engineer would approach it.

 

And where on the spec sheet for some consumer audio gear would I find that?

[pkane2001]

1 minute ago, fas42 said:

And where on the spec sheet for some consumer audio gear would I find that?

Why ask me, I don’t make audio equipment. Ask the manufacturers.

[fas42]

Just now, pkane2001 said:

Why ask me, I don’t make audio equipment. Ask the manufacturers.

 

So, if I want to confirm that there is no possibility of electrical noise causing any audible differences to my playback, I should ring the manufacturers, and ask to talk to the engineers who designed my kit - to ask them how they tested it?

 

Sounds like a very advanced, first world country, solution, to me ...