Blue or red pill?

[Audiophile Neuroscience]

coming in late so apologies if already covered but:

What exactly is the null hypothesis?

What variables will/won't be controlled for?

When/ how will the null hypothesis be rejected or accepted, and at what level of statistical significance?

 

 I get if neither participant/s wants to make this a scholarly exercise and just keep it a bit of fun.

 

• <quote> Playing the same exact file using two different players or the same player with different settings can easily invalidate the test. You’re then comparing two players and their internal processing and not two storage devices. That’s why capturing the digital output to the DAC is critical to ensure that the same digital samples are being sent to it in both cases, as Mansr already said he’ll do.</quote>

 

My question is does the same digital samples captured really control for the variables of different players? I mean, say if they were played at slightly different volumes, could they still have the same "digital output"?

 

Regarding expectation or confirmation bias, this should be controlled by the blind testing and the agreed upon test methodology.

 

I agree with the poster about ideally not over discussing the outcomes prior to the experiment (like that MQA test a while back) and even just knowing its an 'experiment'/test *some* would say influences behavior (Hawthorne effect). However, in this case both participants likely have an agenda and so it gets back to controlling for biases. However, it is very possible IMO that the more it is discussed, the more Mani will feel potentially stressed about performing (....the blue pills may help this??). I believe it is important that Mani, who has set the challenge, feels comfortable that somehow the ground rules have not been changed from his original intention or design.

 

 

 

[Audiophile Neuroscience]

6 minutes ago, Spacehound said:

It doesn't really matter. Similar things have  been tried often before. Those who believe in myth and magic will continue to do so  whatever the test result because the methods, including any listening tests, blind or not,  are "faulty" as they don't coincide with their beliefs.

 

there is a certain futility to be sure.

 

I don't think its "faulty" to question beliefs if they don't match your particular experience. I also don't think it is "faulty" to question whether that experience  is somehow an illusion.

[Audiophile Neuroscience]

1 hour ago, Spacehound said:

I agree with all that, but as I said, but the 'myth and magic' crowd will say the methods are faulty.

 

They may be right

 

 

1 hour ago, Spacehound said:

 

They always do.

 

always?

 

 

1 hour ago, Spacehound said:

 

Not, of course, that they would have taken the trouble to understand the methods anyway.

 

enlighten us

 

 

1 hour ago, Spacehound said:

 

And though they are somewhat 'opposed', both Mani and Mansr know what they are doing.

 

would you prefer they didn't?

[Audiophile Neuroscience]

On 2/13/2018 at 1:57 AM, Spacehound said:

They are unlikely to spend much time  studying  something they 100% misunderstand from  the moment it's switched on and which they do not wish to understand

 

On 2/12/2018 at 7:03 PM, Spacehound said:

the 'myth and magic' crowd will say the methods are faulty. They always do. Not, of course, that they would have taken the trouble to understand the methods anyway.

 

On 2/13/2018 at 1:08 PM, Spacehound said:

 Much like Doctors, Airline pilots,  Quantum mechanics,  Vets,  etc are in their fields. All are  totally useless of course. 

 

9 hours ago, Spacehound said:

I know it won't prove anything to them, and said so earlier. 

 

It's interesting how these kind of threads bring out some people's biased assumptions.
 

17 hours ago, Spacehound said:

There ya go. You've decided already,  Before it's been done.

....and the kettle calling the pot black

 

 

 

7 hours ago, manisandher said:

 

No, there's too much evidence to the contrary - numerous people all over the world describing the differences they hear in exactly the way I hear them. Should it happen though, I'll concede that I've failed to prove I can hear any difference, and let people conclude what they want; either that I'm deluding myself, or that an A/B/X isn't the right way of testing.

Mani.

 

I believe this is the only scientifically valid conclusion. Trying to put aside agendas one way or another, if Mani is unsuccessful to a certain statistical level of significance it still allows for error. Similarly, unless the null hypothesis is formulated in words such as "never shows a difference" or"impossible to show a difference" even if Mani is successful, again to a certain statistical level of significance, it still allows for error. The error is risk that confirming or dis-confirming the null hypothesis is wrong. Aka Type I and II errors and the whole gamut of true/false positives and true/false negatives.

 

Noting that confirm and dis-confirm are not *typically* held to be proofs and indeed doctors are sometimes cautioned by statisticians that accepting a null hypothesis, an intervention making no difference (placebo), is not the same as the treatment intervention being of no real  benefit (not just placebo) in all circumstances. As much as as a cautionary medical approach prefers "false alarms" (Type I)  to a "miss" (Type II), the fact is both errors can cause harm to the patient. At the end of the day we are dealing in probabilities not proofs. At the end of the day clinical (or real-world) and statistical significance are not one and the same thing. It is all relative to the situation and a matter of weighing the probabilities, Type I and Type II errors.This remains true whether testing for verifiability or falsifiability. It is just easier to test for falsifiability where one counter example dis-confirms (rejects) the null hypothesis.

 

And yes. like it or not, much of the above hinges on the validity, reliability, sensitivity, specificity of the test determining outcomes. Blind testing is NOT the issue.It is a necessary controlling factor under which the *test* is carried out.That is not the same as saying audio testing ABX methodology is bullet proof for complex musical passages. It just seems to be assumed so on audio fora in a kind of hand me down fashion or 'its all we have right now so go with it'. I wish there was scientific data to verify it's credentials.

 

 

 

7 hours ago, pkane2001 said:

 

Nobody is pressuring Mani. He chose the format, he chose the venue and he chose the music. He can do whatever listening evaluations he wants, as long as he can't tell which source is playing.  I've done tests like that many times and found it no more stressful than doing them sighted. If not knowing which component is playing induces so much stress that the 'not subtle' differences become too subtle to distinguish, then perhaps one should pick another hobby.

 

What may or may not be stressful to you may be quite different for others.

 

 

 

[Audiophile Neuroscience]

1 hour ago, Spacehound said:

Not at all.

Unlike some I know how 'digital' actually works. We made it from scratch in about 1943 and it hasn't changed  significantly since.  (We didn't pick it off a tree and try to  figure it out.)

And I am aware of the exact point in the 'chain' where 'differences',  if  the inputs are identical,  might  begin to occur. And said so.

 

Yes I agree, you are full of yourself.

 

1 hour ago, Spacehound said:

 

They don't argue with their heart surgeon, or airline pilot, Why argue with us? We are equally qualified in our field.

 

Argue No, question always. Go to two medical specialists and they not uncommonly offer two different opinions.

 

...and who is "us"? Are you an engineer?

 

1 hour ago, Spacehound said:

 

Where a discipline overlaps  the lay public's 'experience' this often occurs. In fact their opinions  are  just 'noise'. They are like many 'do it yourselfers' who do not have the knowledge to realise how poor their efforts actually are.

 

Your condescending and ill informed attitude aside, the only "noise" I hear is from you.

 

 

 

 

[Audiophile Neuroscience]

12 minutes ago, Spacehound said:

I am an engineer (of sorts) My qualifications are  in physics though  after qualification I merely implemented it. My work is in high level computing. 

 

Your original post, though interesting, is  irrelevant. Medicine is neither science (though in some ways science dependent)    nor engineering.

And you didn't design  or create what you are working on.  So  to an extent you are 'working in the dark' .

We did both and aren't..

 

I shall ignore your insults as they impede communication.

 

So you are "sort of" an engineer and yet you claim to "design and create" what exactly?.. other than your self inflated ego? In fact you claim a hell of a lot but offer nothing but your self professed superiority.

Unlike you I readily acknowledge I don't have all the answers, just a respect for scientific inquiry. I do however recognize BS when I see it and perhaps if you stop piling it on, the "darkness" which you project on others might be lifted from your eyes.

 

[Audiophile Neuroscience]

 

1 minute ago, beerandmusic said:

 

If it is not noise, then why do "YOU" suspect there is a difference in sound...there really isn't anything besides bits and noise.

 

Good question Beery. I recall when it was "bits" and "timing", as in jitter, and then noise and some of the solutions as offered by John Swenson et al.

[Audiophile Neuroscience]

3 minutes ago, beerandmusic said:

 

I never said there is timing in a file...if you don't understand what i said, then i don't care to elaborate. 

 

perhaps understanding isnt his 'thing". he is too busy creating the universe, or recreating in his image.

 

Btw I think there would be a few that would disagree with your lumping together of timing/ jitter and noise. I must say I was interested with this new idea of time "smearing" alleged by MQA but couldnt see how they explained why it was different to jitter

[Audiophile Neuroscience]

1 minute ago, Spacehound said:

He woudn't know what it  is.

 

Surprise, another incorrect assumption.I also studied physics. In fact I was chatting about it to a Professor of physics the other day, well she was chatting and I was learning.

 

 

1 minute ago, Spacehound said:

 

My part is very small, 

 

Yes we know but you compensate for it in other ways

 

1 minute ago, Spacehound said:

 

testing parts of an incomplete language, somtimes on an actual device remotely.

 

and yet you still remain evasive. It was Mansr who asked the question. Do you also assume he wouldn't know what quantum computing is?

[Audiophile Neuroscience]

27 minutes ago, mansr said:

If you're talking about quantum computing, why don't you just say so?

8 minutes ago, PeterSt said:

  3 minutes ago, Audiophile Neuroscience said:

It was Mansr who asked the question.

8 minutes ago, PeterSt said:

 

No, it was me. And I know nothing much, Ask Mansr.

 

Mansr asked the question.

 

Re you: It appears mansr didnt liked how you stated he would not possibly be able to explain the results of the test. I doubt that anyone would imply you know nothing much

[Audiophile Neuroscience]

 

3 minutes ago, PeterSt said:

If I may ask - what is the subject ?

2 minutes ago, Spacehound said:

Try reading it again. I said you wouldn't  know what it is - not your fault, I've never seen a 'straightforward' explanation and some of the words used in these explanations  are  gibberish.

But I do know what it is, just not claiming any expertise. Mansr nominated the subject.

[Audiophile Neuroscience]

4 minutes ago, PeterSt said:

Maybe you guys can read back and see that I too wouldn't know how to interpret results.

Thats how I interpreted it.

[Audiophile Neuroscience]

2 minutes ago, Spacehound said:

Mansr simply said Why don't you say  it is quantum computing? He made no other comment.

Not sure why Mansr would need to make a comment after the question but never mind.

[Audiophile Neuroscience]

4 minutes ago, pkane2001 said:

 

Take the little blue pill if you're experiencing performance anxiety (how about that, back on-topic for this thread! )

 

 

Agreed, performance anxiety is on topic and will affect us all differently. That is why I asserted that just because someone does not find a test stressful doesn't mean others will not. I do agree Mani offered the 'challenge' so one would think he is not stressed by the scenario. However, as said, the caveat as to not feeling stressed is that it should stay close to how Mani offered it.That's not to say others' suggestions are not valid.

[Audiophile Neuroscience]

7 hours ago, manisandher said:

 

One thing that I'd like to demonstrate to Mans is streaming vs. local (which was actually the stimulus for the invitation). We'd use Roon/HQPlayer for this.

 

Mani.

 

This really is at the heart of it IMO. IF either Mansr agrees and Mani "passes", then there are grounds for further enquiry.

 

 

7 hours ago, beerandmusic said:

 

In regards to this subject, we will know when engineering has properly dealt with noise, when there is a consensus that swapping out the digital source no longer causes a difference in SQ.

 

 

Disagree.Thats at the heart of the very problem IMO, consensus in perceiving SQ, and what I think mani's challenge is about.

 

6 hours ago, semente said:

 

Can I suggest that after @mansr has performed his listening test you point to aspects of sound or instrument or vocals that you think better express the differences you're able to identify, to see if he agrees?

 

Great idea IMO. Perception is after all subject to conditioning and learning.Paying attention to things previously not noticed.This is NOT a failure if something is only noticed when someone draws your attention to it.It is in the spirit of cooperation and open mindedness.

 

4 hours ago, Spacehound said:

Not to any part before where the tiny DAC chip (most use a chip)  converts its binary input to its analogue  voltage output. There might be some noise on that, but it won't be because the input data has become 'inaccurate'  as it won't or 'furry round the edges', it's ok with that.

 

 

Are you familiar with John Swenson's work and if so, any comment?

 

3 hours ago, Spacehound said:

 

"Paradoxically, no matter how weak the transmitter is, how weak the receiver is, what the distance is, and how much noise there is, it's always possible to transmit information with  (any chosen) arbitrary precision"

Bartosz Milewski

Physicist (Ph. D. in quantum field theory), Mathematician, Programmer

 

 This sounds very plausible but that's precisely why theories are tested. Perhaps the signal has changed or perhaps there is another explanation hereto not known.Some variable that has been overlooked. I dont think anyone however is questioning bits is bits.

 

3 hours ago, Spacehound said:

 

Physics does not take your personal beliefs into consideration.

 

Precisely. However our understanding of it is malleable and subject to change dependent on new evidence.

 

3 hours ago, manisandher said:

There are a lot of guys here who've studied Physics at degree level or above. Well count me in as one too (a long time ago, but hopefully that's allowed). I can't tell you how pissed off I am that the things we've discussing in this thread exist at all. I would absolutely love it to be the case that I could use whichever source, whichever software player, whichever cable, whichever interface, and for there to be no change in the sound. I would rejoice like you couldn't imagine.

snip

We're not all crazy subjectivists. (Not suggesting anyone has called anyone else this in this thread.)

 

Mani.

 

+1

2 hours ago, mansr said:

That's because is running on a filthy SMPS.

 

40 minutes ago, Ralf11 said:

 

 

just lift the veil

 

and reduce the noise

 

who cares about darkness?

 

 

3 minutes ago, pkane2001 said:

 

I suspect this is all expectation bias-driven. Having gone through multiple tests of Wi-Fi -> USB -> DAC, Ethernet -> USB -> DAC, USB directly, and USB over Ethernet -> DAC, I can't tell the difference between them unless there's not enough bandwidth.

 

 

So this is what this test with Mani and Mansr is all about IMO.

[Audiophile Neuroscience]

6 hours ago, pkane2001 said:

 

I suspect this is all expectation bias-driven. Having gone through multiple tests of Wi-Fi -> USB -> DAC, Ethernet -> USB -> DAC, USB directly, and USB over Ethernet -> DAC, I can't tell the difference between them unless there's not enough bandwidth.

I said:

 

 

6 hours ago, pkane2001 said:

6 hours ago, Audiophile Neuroscience said:

7 hours ago, pkane2001 said:

I suspect this is all expectation bias-driven. 

 

So this is what this test with Mani and Mansr is all about IMO.

 

Yes!

 

To clarify; my comment "So this is what this test with Mani and Mansr is all about IMO. " was aimed at you not being able to hear differences.

 

Expectation biases may or may not come in to it but either way should be controlled for.

[Audiophile Neuroscience]

1 hour ago, Spacehound said:

Best to use a measuring device then.

 

Which takes us back to......

 

...are you measuring what you think you are measuring....and how accurate is your test.

[Audiophile Neuroscience]

1 hour ago, Spacehound said:

I measure everything I can think of.

 

..and what of the things you can't think of

 

1 hour ago, Spacehound said:

So I miss something. But I won't do that as if I could hear it I would measure it.

 

So, provided you can perceive it and provided your test is up to the task.Two possible failures right there.

1 hour ago, Spacehound said:

And all this stuff about humans being able to hear   'impulses' better than some other things is is nonsense.

 

If you say so we should believe you, because you say so

1 hour ago, Spacehound said:

Anyway, proper impulses  don't exist in 'reality'

 

I have an impulse. it says you are wrong.

 

1 hour ago, Spacehound said:

proper impulses  don't exist in 'reality'

 

perhaps Einstein said it best.

<quote> Reality is merely an illusion, albeit a very persistent one </quote>

[Audiophile Neuroscience]

3 minutes ago, Shadders said:

I had a phase of doing that. I remember, it was 7 inches and a wrinkle, but given my age, it's now 7 wrinkles and an inch.

 

You should be beaten within an inch..I mean wrinkle, of your life! What is the SI unit of wrinkle anyway?

[Audiophile Neuroscience]

18 minutes ago, Spacehound said:

Einstein was wrong.

 

between Einstein and yourself, let me see, yep I'd rather believe Einstein.

Quote

Reality will remain even if the human race (or any life)  never existed.

 

How would you know it existed if you never existed?

Esse est percipi !

 

off topic- the end

[Audiophile Neuroscience]

7 hours ago, pkane2001 said:

 

There are two participants in this test, and their aims are not the same.

 

Who said they were?

 

7 hours ago, pkane2001 said:

And by the way, I've been in Mani's shoes for many years. Spent a ton of money on digital clean-up equipment, cables, etc. Until I decided to seriously test what I was hearing. I've done the proposed blind test many times over, and no, personally I can't hear the difference.

 

You are not alone in your inability to hear differences but that is not everyone's experience. Again, IMO, this is the essence of Mani's 'offer'.I think 'offer' possibly better describes the scenario rather than test or challenge. It certainly may progress beyond that but lets wait and see.

 

 

7 hours ago, pkane2001 said:

I'm fully open to Mani proving his point. And if he does, just like Mans, I'd like to try to understand what caused the audible differences even if PeterSt thinks that's impossible to figure out. An objective outcome, positive or negative is something I would welcome, unlike all the subjective reports that dominate this topic.

 

I am also open to Mani or Mans finding whatever they find ie I really am interested in the result either way. I am less convinced about "proving his point" as, previously stated, IMO we are likely talking probabilities still at this stage not proof. But lets wait and see what happens.I too would like to understand more about the causes of hearing differences.

 

7 hours ago, pkane2001 said:

 

Cogito ergo sum. I think therefore I am. No amount of argument or even a blind test will prove to me that I don't exist  You, on the other hand, I'm not so sure about... perhaps you're just a figment of my imagination.

 

 

 

I have no quibble with Descartes Cogito ergo sum as I do for some of his other ideas. I would just say that the dictum does not seek to prove existence of anything other than self. Descartes indeed , to my understanding, noted the seemingly illusory nature of existence but comforted himself that at least he must exist if he could ponder such things. The illusory nature of things, as in the senses, is very much on topic for this thread, thus my reply here. The 'conversation' with Spacehound was more a tongue in cheek dig at Ontological materialism/realism vs Ontological idealism. But paraphrasing yourself...if you have to explain it,...... I would say this, if I'm "just a figment of your imagination" that you have been discussing this with, would that mean you are having an illusion or delusion?

[Audiophile Neuroscience]

35 minutes ago, mansr said:

How can you be sure you exist? See also Cotard delusion.

 

Well because @pkane2001 "grand" delusion applies to others not self.He therefore doesn't qualify for Cotard's delusion.

 

I have never seen a case of Cotard but have seen many times where people do not perceive/recognize body parts in various post stroke neglect/extinction/inattention phenomena. It extends to say neglecting everything on one side of a meal plate and only eating from the other side. I know you're not as interested as much to what occurs to the signal from tympanic membrane to brain but different sensory processing pathways maybe explains why some people hear things that others do not. It could be pathological or physiological but I would ague not an 'illusion' in the sense we talk about in audio fora whereby the assumption by some is it is all psychological or "nuts". "Nuts" is also a possibility.

 

Have fun with the 'testing'

[Audiophile Neuroscience]

1 hour ago, mansr said:

Measurement equipment is far more sensitive than our ears, so if it can detect no difference, then neither can the ears. If someone still hears something differently, the cause must be outside the audio gear.

 

Putting aside test accuracy for the moment, for tests to always reveal/correlate /discern differences in hearing, everything that is audible must be measurable, covering the sum of all (known) possible measurement parameters. Some measurements protocols may only come about only after, and in response to, an observation. A genuine question: Is there an audio signal measurement correlate for sound stage depth and height?

[Audiophile Neuroscience]

39 minutes ago, mansr said:

Those are mostly products of room reflections. A normal stereo recording doesn't convey depth or height information.

 Thanks for the reply

I am assuming that the room will remain constant. So, If Mani uses such soundstage cues to successfully hear differences, would this not be an example of a heard difference not appearing or measurable in the audio signal?

[Audiophile Neuroscience]

2 hours ago, beerandmusic said:

7 hours ago, Audiophile Neuroscience said:

 

 everything that is audible must be measurable,

 

is that a known fact?

 

perhaps a misunderstanding. I am not saying everything that is audible IS measurable. "Everything that is audible must be measurable" is a prerequisite in order to be confident that any and all audible differences heard by Mani will be detected by Mans, or anybody else. If the test gear is not able to detect everything the ear can it is of little value in the context of this thread......IF measurements are included in the setup as some have suggested. I am of the belief that 'not everything that is audible is measurable and not everything that is measurable is audible'. Kinda related to "Not everything that can be counted counts and not everything that counts can be counted."

-William Bruce Cameron (sometimes attributed to Einstein)