Misleading Measurements

[pkane2001]

4 hours ago, manueljenkin said:

I have a friend who does DBT on pretty much every gear he buys. He doesn't like to be around forums, but you can reach out to him if you're interested.

 

He runs a rme adi2 (which is touted by asr measurements to be completely impervious to upstream changes wrt usb, system scheduling) and found enough improvements on it using sonore switch. He has done enough DBT on this aspect.

 

He finds asr measurements to be almost irrelevant to the results he has got via DBT on many devices.

 

Just one of many examples.

 

Maybe you can reach out to him and have him post his results.

[vmartell22]

6 hours ago, pkane2001 said:

Maybe you can reach out to him and have him post his results.

 

And his methodology!   

[manueljenkin]

Well his profession involves rigorous statistics, and he often persuades me to do ABX so I doubt he has any problem with the methodology. Shoot me all your questions precisely (what you expect, what numbers etc). and I'll ask him when he's free.

[vmartell22]

14 minutes ago, manueljenkin said:

Well his profession involves rigorous statistics, and he often persuades me to do ABX so I doubt he has any problem with the methodology. Shoot me all your questions precisely (what you expect, what numbers etc). and I'll ask him when he's free.

 

Has the experiment been peer reviewed as well as the results?

 

v

[fas42]

23 hours ago, sandyk said:

Yes, and incidentally, Teresa should also be able to check these supplied files out for herself directly from the DropBox player WITHOUT needing to download them and use up her communal bandwidth, as can our K1W1 member.   

 

https://www.dropbox.com/s/6ak9tyqrpglq1w9/Tyros 3 - The Power Of Love (Instrumental)HF.mp4?dl=0

https://www.dropbox.com/s/a8koifix231idw3/Tyros 3 - The Power Of Love (Instrumental)LF.mp4?dl=0

 

Sorta "telling", Alex, that not a single other person has given you feedback on this ... whichever way they hear it, 😉 ...

[sandyk]

1 hour ago, fas42 said:

 

Sorta "telling", Alex, that not a single other person has given you feedback on this ... whichever way they hear it, 😉 ...

 Yes. I wonder how many had an anonymous listen  and weren't game to report back what they heard on ideological grounds

The majority of the Objective mob simply don't want to know, because it would throw into doubt many of the other things they claim , based solely on measurements. It would open up a big can of worms.

As an I.T. person, you had every right to be sceptical, just as 2 of my Sydney E.E. friends did originally too,  before hearing the differences for themselves from my supplied USB memory stick without me being present to influence them. (Sunday, January 19, 2014)

 The differences aren't subtle , are they ?  

[kumakuma]

14 minutes ago, sandyk said:

 Yes. I wonder how many had an anonymous listen  and weren't game to report back what they heard on ideological grounds

The majority of the Objective mob simply don't want to know, because it would throw into doubt many of the other things they claim , based solely on measurements. It would open up a big can of worms.

As an I.T. person, you had every right to be sceptical, just as 2 of my Sydney E.E. friends did originally too, before hearing the differences for themselves from my supplied USB memory stick without me being present to influence them.

 The differences aren't subtle , are they ?  

 

I listened to them and didn't hear any differences between them.

[danadam]

3 minutes ago, kumakuma said:

 

I listened to them and didn't hear any differences between them.

Same here.

[sandyk]

The funny thing here is that Frank can easily hear the differences with his laptop. What are you guys using, laptops, perhaps a stock standard Mac Mini  ?

 Suck on this one replacing LF version.

If you can't hear the difference with this one compared with HF version, then you have MAJOR problems.

Perhaps your brain won't let you ?

https://www.dropbox.com/s/gb1l2s9i2bcnpl3/Tyros 3 - The Power Of Love (Instrumental)LF2.mp4?dl=0

P.S.

The LF end on this one sounds greatly exaggerated, even when using the mediocre Dropbox player..

[sandyk]

1 hour ago, kumakuma said:

 

I listened to them and didn't hear any differences between them.

As expected . You probably can't even see the difference in brightness and sharpness between them right from the start either. 

[manueljenkin]

5 hours ago, vmartell22 said:

 

Has the experiment been peer reviewed as well as the results?

 

v

I don't see any certifiable peer reviews on the "conclusions" asr and the echo chamber makes either. I don't see any IEEE certifications or similar at ASR. It's just one person's opinion, and his personal interpretations of what he sees in a chart (extreme possibility of bias). Just because the site name has the word "science" in it doesn't mean it automatically gets credibility as a reliable source of information. Fyi there's a subreddit called r/sciencememes.

 

If that doesn't have a peer review to be accepted by you and the objective club, no reason this has to be. So here's a task, get the asr place and analysis articles certified and let's talk 😊 about us refuting it. Not much time to refute a non reliable source of information with massive holes. It is non reliable/inconclusive, end of story, things can swing either way. For myself, I personally experience the changes described with usb cables, software swaps, etc despite sine squiggle measurements of an analyzer approving zero change. No one has to "prove" it to me. I accept certain things that are real don't have a clear explanation yet.

[fas42]

A thought on whether differences, in general, are audible ... first of all, do it at a decent volume! I have found it laughable at times, the softness that some audio people listen to payback at - wind it up so you get the intensity that live music presents, and just about anything you do will alter the qualities of the sound ... I was thinking of this during a round of elevated volume of the active speakers we had going here, just before - it made it so obvious that conditioning of the mains still wasn't good enough yet; requiring disconnection of all other devices on the circuit to get rid of an edge, an 'offness' in the sound.

[pkane2001]

6 hours ago, manueljenkin said:

I don't see any certifiable peer reviews on the "conclusions" asr and the echo chamber makes either. I don't see any IEEE certifications or similar at ASR. It's just one person's opinion, and his personal interpretations of what he sees in a chart (extreme possibility of bias). Just because the site name has the word "science" in it doesn't mean it automatically gets credibility as a reliable source of information. Fyi there's a subreddit called r/sciencememes.

 

If that doesn't have a peer review to be accepted by you and the objective club, no reason this has to be. So here's a task, get the asr place and analysis articles certified and let's talk 😊 about us refuting it. Not much time to refute a non reliable source of information with massive holes. It is non reliable/inconclusive, end of story, things can swing either way. For myself, I personally experience the changes described with usb cables, software swaps, etc despite sine squiggle measurements of an analyzer approving zero change. No one has to "prove" it to me. I accept certain things that are real don't have a clear explanation yet.

 

Let me summarize the 43 pages of this thread for you, so you don't have to keep repeating it:

 

ASR sucks, we don't know everything, measurements are wrong because we hear things, and noise hidden in two, exactly the same digital files is different because one was recorded with an LPS and the other with a switching supply. Got it. Carry on. 

[manueljenkin]

2 hours ago, pkane2001 said:

 

Let me summarize the 43 pages of this thread for you, so you don't have to keep repeating it:

 

ASR sucks, we don't know everything, measurements are wrong because we hear things, and noise hidden in two, exactly the same digital files is different because one was recorded with an LPS and the other with a switching supply. Got it. Carry on. 

You're misquoting me. Measurements need not be "wrong". Just that the notion that all parameters of sound quality can be conclusively determined by the limited measurements done at asr is wrong. I haven't tried the digital file differences yet, so can't say much but "I can believe it" since I've had similar experience on another operation.

[pkane2001]

7 minutes ago, manueljenkin said:

You're misquoting me. Measurements need not be "wrong". Just that the notion that all parameters of sound quality can be conclusively determined by the limited measurements done at asr is wrong. I haven't tried the digital file differences yet, so can't say much but "I can believe it" since I've had similar experience on another operation.

 

Don't take this personally: I was summarizing the whole thread, not just your posts.

 

Belief is fine, but objective, repeatable, reproducible evidence is something else. Science is not based on belief. This seems to be completely missing here, despite the seemingly objective sub-forum charter. 

[The Computer Audiophile]

Just now, pkane2001 said:

 

Don't take this personally: I was summarizing the whole thread, not just your posts.

 

Belief is fine, but objective, repeatable, reproducible evidence is something else. Science is not based on belief. This seems to be completely missing here, despite the seemingly objective sub-forum charter. 

Come on Paul, only 42.5 of the pages are about that :~)

 

I saw a great quote yesterday. I'm sure many have seen if before, but it was new to me. I'm also sure many in the Objective-Fi form will like and know it well. 

 

“You can't convince a believer of anything; for their belief is not based on evidence, it's based on a deep seated need to believe.”

― Carl Sagan

[manueljenkin]

22 minutes ago, The Computer Audiophile said:

Come on Paul, only 42.5 of the pages are about that :~)

 

I saw a great quote yesterday. I'm sure many have seen if before, but it was new to me. I'm also sure many in the Objective-Fi form will like and know it well. 

 

“You can't convince a believer of anything; for their belief is not based on evidence, it's based on a deep seated need to believe.”

― Carl Sagan

Well we have an outlier here I guess 😅. I used to "believe" usb cables should make no difference to an ASR certified dac 😛 and once in a while used to troll the believers, now I'm on the other side of the club 😁. Maybe we could apply uncertainty/probability based estimates to this quote!!

[The Computer Audiophile]

11 minutes ago, manueljenkin said:

Well we have an outlier here I guess 😅. I used to "believe" usb cables should make no difference to an ASR certified dac 😛 and once in a while used to troll the believers, now I'm on the other side of the club 😁. Maybe we could apply uncertainty/probability based estimates to this quote!!

Hi Manuel, I have one foot solidly in each camp (objective and subjective) :~)

[Confused]

Reading the recent posts here, and mindful that this is an "objective" thread, it occurs to me that the bit perfect files sound different idea could be subject of a controlled blind test, similar to the ones performed recently by @Archimago (although I suspect he might not be interested in this one) and Mark Waldrep.

 

Maybe something along the lines of a number of sets of four music files, all bit identical, one pair saved by method A, another pair by method B.  The test would be to match the pairs.

 

Just one of my idle thoughts.  I might be wrong, but I suspect that nobody would be motivated enough to set this up.

 

It would be fun if somebody did though, and who knows, it might just provide some interesting results.

[John Dyson]

1 hour ago, Confused said:

Reading the recent posts here, and mindful that this is an "objective" thread, it occurs to me that the bit perfect files sound different idea could be subject of a controlled blind test, similar to the ones performed recently by @Archimago (although I suspect he might not be interested in this one) and Mark Waldrep.

 

Maybe something along the lines of a number of sets of four music files, all bit identical, one pair saved by method A, another pair by method B.  The test would be to match the pairs.

 

Just one of my idle thoughts.  I might be wrong, but I suspect that nobody would be motivated enough to set this up.

 

It would be fun if somebody did though, and who knows, it might just provide some interesting results.

The results might be interesting, but the laws of physics&math&computing limit my interest.   I have done too many things in my engineering life where bit perfect means bit perfect, with all of the associated ramifications.  Stuff really works that way.

 

There is an 'out' for the metaphysics advocates about real-time play of a flac vs. .wav file with exactly the same content, but that is all about analog noise sources and not-so-good analog layouts.   Those kinds of comparisons have nothing to do with the payload content though.

 

I don't think that many people who are competent enough to do such an experiment would be interested in (wasting time) doing it.  If the stats would be done correctly, physics and math show that the results will be the same.   Any 'signal' that shows a difference will be deep into the statistical noise.  Perhaps, though, maybe someone competent enough to do it, and  had time to totally waste, or they were paid to do it.....?

 

John

 

 

 

[vmartell22]

13 hours ago, manueljenkin said:

I don't see any certifiable peer reviews on the "conclusions" asr and the echo chamber makes either. I don't see any IEEE certifications or similar at ASR. It's just one person's opinion, and his personal interpretations of what he sees in a chart (extreme possibility of bias). Just because the site name has the word "science" in it doesn't mean it automatically gets credibility as a reliable source of information. Fyi there's a subreddit called r/sciencememes.

 

If that doesn't have a peer review to be accepted by you and the objective club, no reason this has to be. So here's a task, get the asr place and analysis articles certified and let's talk 😊 about us refuting it. Not much time to refute a non reliable source of information with massive holes. It is non reliable/inconclusive, end of story, things can swing either way. For myself, I personally experience the changes described with usb cables, software swaps, etc despite sine squiggle measurements of an analyzer approving zero change. No one has to "prove" it to me. I accept certain things that are real don't have a clear explanation yet.

 

The thing is whatever you find wrong with ASR's approach, at least it is supported by formally accepted principles - that's a good start even if the individual members are not leading lights of the IEEE or ACM or AES or..

 

https://en.wikipedia.org/wiki/Straw_man

 

The thing is that all we have here is argument by unknown/unverified authority...  and I am not saying that one should not compare, experiment, etc - all I am saying is that without reviewed and accepted rigor, you statician friend's results are no different from just another anecdote/opinion... no proof of anything...

 

Really have no desire to continue the argument - You do you!  Be Happy! - I will get out of our hair... not sure why I even replied in the first place - I will confess, prbly the reason is that "argument by authority" always makes me reply...  ah well 

 

we will never convince each other... no point I guess... such is life!

 

peace

 

v

[danadam]

16 hours ago, sandyk said:

What are you guys using, laptops, perhaps a stock standard Mac Mini  ?

rme adi-2 pro fs r and sennheiser hd650

16 hours ago, sandyk said:

Suck on this one replacing LF version.

Still the same.

16 hours ago, sandyk said:

If you can't hear the difference with this one compared with HF version, then you have MAJOR problems.

Yeah, sure 🙄

[sandyk]

8 hours ago, pkane2001 said:

Let me summarize the 43 pages of this thread for you, so you don't have to keep repeating it:

 

ASR sucks, we don't know everything, measurements are wrong because we hear things, and noise hidden in two, exactly the same digital files is different because one was recorded with an LPS and the other with a switching supply. Got it. Carry on. 

 As usual, you got it very wrong, especially the last part as there were no changes to the types of PSUs used .

 You got it right about ASR though, as you obviously are more of a fanatic regarding measurements than the enjoyment of actually listening to the music, or further improving your equipment to increase the enjoyment you get from well recorded music.  After all, this IS an Audiophile forum .

[pkane2001]

5 minutes ago, sandyk said:

 As usual, you got it very wrong, especially the last part as there were no changes to the types of PSUs used.

 You got it right about ASR though, as you obviously are more of a fanatic regarding measurements than the enjoyment of 

actually listening to the music, or further improving your equipment to increase the enjoyment you get from well recorded music.  After all, this IS an Audiophile forum .

 

Alex, stop the personal attacks. You don't know what I'm fanatic about, and it's certainly not measurements.

 

[sandyk]

11 minutes ago, pkane2001 said:

 

Alex, stop the personal attacks. You don't know what I'm fanatic about, and it's certainly not measurements.

 

It's not a personal attack. It's an observation. Otherwise , why have you gone to the trouble of designing  the S/W linked to in your signature ?

 BTW, I wasn't even going to post any further replies in this thread unless specifically directed at me., which you did in your  provocative post summing up the thread. 

 Surely you must have known that either Manuel or myself would reply in the manner that we did ?