Jump to content
IGNORED

AB Testing


Recommended Posts

I would think you would realize that there is no one explanation for audio phenomena like this. Saying the only operator involved with the audible signature of a MP3 is masking is just plain wrong Dennis.

 

I agree which is why I never said that.

 

Perhaps you know of a study where MP3s have been implemented with perfect reproduction. If so, I'll reconsider my thinking. But you are making an extraordinary claim here, and it will require extraordinary proof. Especially with a preponderance of evidence against it.

 

Perhaps you could read the very passage of mine you quoted here where it says, and I quote myself,"nor is anyone claiming it works perfectly."

 

One could say you were building an extraordinary straw man. But it might be more of the Calico Cat and the Gingham Dog variety. The old Dutch clock is certainly easy to believe in.

 

CatStuff: The Gingham Dog and the Calico Cat

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
I do undertand that Dennis's persistence can feel like he's paying special attention to raining on your parade. :-)

 

Hi Jud,

 

If by "parade" you mean my tweaking parade: no, not really. I'm not a tweaker. I believe amps/DACs/cables sound different and that's about all I'm interested in. Oh, I'd consider component isolation also.

But from his point of view, I'm guessing it looks more like he's trying to fight a lot of misunderstanding and misinformation.

 

That's fine if someone specifically asks for his information rather than aggressively forcing views and self-professed facts on others.

Link to comment
They do admit that DBT cannot be used to prove no difference exists, though. The problem is they are too busy being subjective to see why DBT is flawed.

 

Of course. They realize the issue though - if even one person can hear a difference then their argument is kaput. That's why guys like me are attacked relentlessly when we post a blind test detected this or that difference - it must be a DBT, it must be volume matched by oscilloscope, the person involved has commercial interests, - the list goes on. Its in fact for the very heart and soul of Hi Fi - they know it - which is why arguments about it go on and on but get nowhere with each side retreating to their impregnable position.

 

Thanks

Bill

Link to comment
Actually saying "what you are saying is (probably) wrong because" is NOT badgering... The badgering comes when the same claim is repeated continually without any additional information as if some how stating it 5/10/20 times makes the information stronger each time...

 

I don't understand what you're saying here. Not that I'm that interested either, so forget it.

 

Somewhat after [Peter Aczel over at the Audio Critic]'s views were introduced and the response was [i don't regard people like Peter Aczel or the Hydrogen Audio forum as fellow audiophiles.]

 

This is not that clear either, but if you're providing an example of a subjectivist forcing his/her views, attacking, or ridiculing an opposing point of view then I'd say that's quite a stretch.

 

the attacks actually came from those who suggested that fuses made a difference…

 

You can't be serious.

 

I seam to remember one of the negative comments being...

 

Originally Posted by chg

If these hi-fi tuning fuses provide such a performance boost, wouldn't it make sense for all manufacturers to include them in their products? Since most (all?) do not, doesn't that say something?

 

Or was that comment not meant as a negative towards fuses...?

 

Is that a negative comment, or a legitimate, thoughtful question that lead to constructive conversation? If you look at the replies following that comment you'll have your answer. Reading other member replies I learned that a few manufacturers do use such fuses and some members, whom I respect, report hearing improvements; those opinions have led me to keep an open mind to their potential benefits.

Link to comment

Now for those who thought the Calico Cat and Gingham Dog poem was just random reference, I include the final stanza. The one with the point of mentioning this wonderful child's poem. For those who don't bother to spend two minutes or less reading it:

 

Next morning, where the two had sat

They found no trace of dog or cat;

And some folks think unto this day

That burglars stole that pair away!

But the truth about that cat and pup

Is this: they ate each other up!

Now what do you really think of that!

(The old Dutch clock it told me so,

And that is how I came to know.)

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
chg,

 

Very early in this very thread, my first post to this thread, my first comment was blind testing was about the least enjoyable least fun thing you could do with your audio system. My second comment in that first post was that blind testing was about the least convenient thing you could do with your system. And further for those reasons I considered such testing not really fit for people to carry out on their own.

 

Now if you have gotten the idea I think all these folks reading here at CA should make decisions on the various aspects of their system by an on going series of careful blind tests conducted by themselves, then yes I can see how you and others wonder about the enjoyment I get. Or how you could think what I 'prescribe' for others would be like torture.

 

Further I also like measurements of what is going on in some of these matters. I do happen to like that myself though not everyone does. On the other hand it isn't like I spend the majority of my time measuring what is going on. I do at times spend some time when I am trying to understand something about this audiophile stuff. It is useful in some cases, and clear cut, and sometimes doesn't and sometimes does agree with plain old listening.

 

Sometimes less pleasant or objectionable activity is overall worth doing to makes gains and improve more fun things down the road. It is I think beneficial if one bothers to go 'through the torture' to share it with others so they don't each and every one need to go through that to learn something. That is also why I think making use of available psycho-acoustic knowledge, and well founded electrical engineering knowledge makes good sense. Instead, when it contradicts plain old fun listening, it is the plain old listening that takes precedence in many people's minds. I say that as someone who once approached things just in the plain old listening way, and since taking advantage of the hard won knowledge of others have found it quite rewarding, useful and resulting in improved fidelity and musical enjoyment.

 

CA is supposed to be a place where computer audio hobbyists come for enjoyment.

 

I am sure it is an on going evolving thing, but Chris can say what his idea of what CA is supposed to be. I am sure it is for CA hobbyists to come for fun and share ideas. Does that mean that we should only relate the simplest most fun things and not have talk of more serious matters regardless of whether they are true or not? I wouldn't think that is the case, but Chris can have his say on that.

 

I once only posted if I could make a direct and helpful input to someone. I have been in a position to do that at least a few times. More than I have done that others have done the same for me. But a great many posts aren't about concrete equipment or software issues. It is just discussing ideas. If we eliminated those, then posts like this one, by Chris, about AB testing wouldn't be allowed. Since Chris himself started this one (though 4 years ago) I take it he doesn't wish to eliminate such idea exchange.

 

Maybe I read it wrong, but your reply, in my view, comes across as arrogant, condescending, and demeaning: You are the self appointed expert and since we "plain old listeners" don't have the "knowledge" and/or proper listening/testing capabilities we should all just reference your conclusions and follow what you say. Sorry, not trying to give you grief, but that's how I read it. I don't know if you do it on purpose, but that seems to be your style of communication.

 

BTW, my mention of "torture" is more along the lines of the mental torture of constantly questioning/fearing your own perceptions. For example, you perform your own listening test, actually report hearing a difference, but still wonder if it was a delusion or not. Constantly questioning ones own perceptions can't be healthy.

 

...I am sure it is for CA hobbyists to come for fun and share ideas. Does that mean that we should only relate the simplest most fun things and not have talk of more serious matters regardless of whether they are true or not?

 

...But a great many posts aren't about concrete equipment or software issues. It is just discussing ideas. If we eliminated those, then posts like this one, by Chris, about AB testing wouldn't be allowed. Since Chris himself started this one (though 4 years ago) I take it he doesn't wish to eliminate such idea exchange.

 

Pretty condescending. Of course no one wants to eliminate idea exchange or "talk of more serious matters" (You might as well have said big boy/grown up talk). It's the manner in which some exchange their ideas that is the problem.

Link to comment
(The old Dutch clock it told me so,

And that is how I came to know.)

Hey Dennis

You haven't been playing Dutch Ovens again , have you ? (grin)

Kind Regards

Alex

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment

Maybe I read it wrong, but your reply, in my view, comes across as arrogant, condescending, and demeaning: You are the self appointed expert and since we "plain old listeners" don't have the "knowledge" and/or proper listening/testing capabilities we should all just reference your conclusions and follow what you say. Sorry, not trying to give you grief, but that's how I read it.

 

Read more carefully. I said it was inconvenient and not fun. Also I consider it rather time and resource intensive. Regular old listeners could do it, but why not let others handle the heavy, boring, and non-fun work. I don't think I said they couldn't just that they likely wouldn't and probably shouldn't.

 

BTW, my mention of "torture" is more along the lines of the mental torture of constantly questioning/fearing your own perceptions. For example, you perform your own listening test, actually report hearing a difference, but still wonder if it was a delusion or not. Constantly questioning ones own perceptions can't be healthy.

 

The questioning yes, I never used fearing, that is something you put in there. The torture was in reference to someone else who used the word. I thought it was you, but may have been someone else. I consider reflection, pondering and questioning quite healthy. Unhealthy is not to do that because your sense of self is too fragile to do so. My perceptions, like any other human perception, is not perfect or unerring. Seems unwise to not note that and consider it in your experiences.

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
Hey Dennis

You haven't been playing Dutch Ovens again , have you ? (grin)

Kind Regards

Alex

 

No Alex, I hear you, but no Dutch Ovens for me thanks.

 

Dennis

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
If these hi-fi tuning fuses provide such a performance boost, wouldn't it make sense for all manufacturers to include them in their products? Since most (all?) do not, doesn't that say something?

 

Not really. It's a fallacy to believe most manufacturers primary concern is sound quality and they actually conduct listening tests to determine it and exactly what influences sound quality. If they did IMHO the products we have would be a lot different. The few manufacturers I know that actually do this (usually small boutique concerns) produce products that pretty much leave the other stuff for dead.

 

An example is a DAC well known out our way called the Killer DAC that can be had for $5.5K and is built almost entirely that way using parts selected purely for sound quality - even special and difficult to obtain wire is used - plus tons of other stuff. I know of no commercial DAC from the big boys at any price point that can show that DAC a clean pair of heels on PCM material - and that includes stuff that is much more expensive. And yes tests have been done to prove it. If you want to discuss it that DAC has its own forum:

All about the KillerDAC - Index

 

Thanks

Bill

Link to comment

bhobba,

 

Hey Bill,

 

Seriously, can you provide the name of the vendor? Wouldn't be bad for business, and I would like to know who you are referring to in case I get the chance to audition them sometime.

And always keep in mind: Cognitive biases, like seeing optical illusions are a sign of a normally functioning brain. We all have them, it’s nothing to be ashamed about, but it is something that affects our objective evaluation of reality. 

Link to comment
bhobba,

 

Hey Bill,

 

Seriously, can you provide the name of the vendor? Wouldn't be bad for business, and I would like to know who you are referring to in case I get the chance to audition them sometime.

 

There's that condescending, demeaning, objectivist esldude sarcasm again.

Link to comment
Seriously, can you provide the name of the vendor? Wouldn't be bad for business, and I would like to know who you are referring to in case I get the chance to audition them sometime.

 

They are hand built to order by a guy called Steve Garland here in Australia. You can contact him via the forum I linked to - he posts as Stevenvalve. He is generally considered to have one of the best if not the best systems here in Australia.

 

Recently building them became too much and now is done by a guy up the road from where I live called Craig Conner who posts as Rawal99. I have one on order myself to complement my cadre of DAC's. I have heard it and its hype is justified. I doubt you will ever hear one in the US. Like I said it is the antithesis of mass produced stuff, designed and built entirely differently - basically all done by ear using stuff 'objectivists' will tell you makes no difference such as special copper wire.

 

I was at a DAC shootout about 2 years ago and it blew away all comers.

 

If you build or sell DAC's I suspect it would be very bad for your business to have one around - it really is that good.

 

Thanks

Bill

Link to comment
Maybe I read it wrong, but your reply, in my view, comes across as arrogant, condescending, and demeaning: You are the self appointed expert and since we "plain old listeners" don't have the "knowledge" and/or proper listening/testing capabilities we should all just reference your conclusions and follow what you say. Sorry, not trying to give you grief, but that's how I read it.

 

Read more carefully. I said it was inconvenient and not fun. Also I consider it rather time and resource intensive. Regular old listeners could do it, but why not let others handle the heavy, boring, and non-fun work. I don't think I said they couldn't just that they likely wouldn't and probably shouldn't.

 

BTW, my mention of "torture" is more along the lines of the mental torture of constantly questioning/fearing your own perceptions. For example, you perform your own listening test, actually report hearing a difference, but still wonder if it was a delusion or not. Constantly questioning ones own perceptions can't be healthy.

 

The questioning yes, I never used fearing, that is something you put in there. The torture was in reference to someone else who used the word. I thought it was you, but may have been someone else. I consider reflection, pondering and questioning quite healthy. Unhealthy is not to do that because your sense of self is too fragile to do so. My perceptions, like any other human perception, is not perfect or unerring. Seems unwise to not note that and consider it in your experiences.

 

Yes, looks like I did read it entirely correctly. It's inconvenient and not fun, time and resource intensive, heavy, boring, and non-fun work. So, esldude will do all that tough work for you and he will also tell you all how to think and if you think differently then he will tell you you are wrong and dillusional.

 

I don't think I said they couldn't just that they likely wouldn't and probably shouldn't.

 

If they "wouldn't and probably shouldn't" do you think they should listen to you (or some other biased objectivists) for all the answers?

 

My perceptions, like any other human perception, is not perfect or unerring. Seems unwise to not note that and consider it in your experiences.

 

"Note that" and constantly obsess, question, and doubt are two very different things.

Link to comment
I can't speak for all 'subjectivists' but I, and most guys I know (who are also subjectivists), do not ignore the claims of 'objectivists'. I, and people I know, do blind tests on stuff the objectivists claim there is no difference all the time such as amps. cables etc etc and readily and easily hear differences. [...removed DAC designer reference ...] However DBT's are difficult to organise and analyse so hobbyists like me do SBT's. [... removed wine tasking analogy ...]

 

[...removed invitation...]

 

The issue with so called 'objectivists' is they will ignore what does not suit them and say all reputable tests show there is no difference - where they decide what is reputable. Blind tests like I do are invalid because they are not DBT's - and of course since DBT's are so hard to organise and do - guess what - they get away with it since no one really has the time or inclination to take them up on it. DBT's done by manufacturers are tainted because they are commercial. It goes on and on - basically they are not interested in the truth - rather they are simply interested in promulgating a pre-decided agenda.

Bill - with respect I believe your reply totally missed my point, yet also made it.

 

A test is set up (and I'm talking about sighted tests here) to compare an aspect of an audio system which when looked at from an objective point of view. Everyone agrees that the test should show a difference, the person X says the difference is "night and day" while person Y argues that there should be no difference. The test is carried out on equipment which Person Y chooses. I think you will agree this is a typical situation.

 

Okay, so they do the test, swapping back and forward to compare the two situations. Each time doing a simple A-B; B-A, A-A or B-B comparison and the question is asked "is there a difference?" Person Y listens and each time says if there is a difference or not. The time is spent quite amicably and relaxed.

 

At the end of the test, Person Y feels they haven't heard a difference.

 

There are several possible results to this.

1) There is a difference but Y doesn't notice it because they are deaf or untrained.

2) There is a difference but Y doesn't notice it because the equipment wasn't resolving enough to notice the differences.

3) There is no difference even though X feels he has heard it.

 

In my experience there are very few people with a subjective outlook who will accept that (3) is a possibility. That's all I'm asking for. In every occurrence of sighted testing, you have to accept that it is possible that any difference noticed. Just as the subjectives call for the objectives to accept that they don't have all the answers or can't measure all the differences.

 

Eloise

Eloise

---

...in my opinion / experience...

While I agree "Everything may matter" working out what actually affects the sound is a trickier thing.

And I agree "Trust your ears" but equally don't allow them to fool you - trust them with a bit of skepticism.

keep your mind open... But mind your brain doesn't fall out.

Link to comment
They do admit that DBT cannot be used to prove no difference exists, though. The problem is they are too busy being subjective to see why DBT is flawed.
Of course. They realize the issue though - if even one person can hear a difference then their argument is kaput. That's why guys like me are attacked relentlessly when we post a blind test detected this or that difference - it must be a DBT, it must be volume matched by oscilloscope, the person involved has commercial interests, - the list goes on. Its in fact for the very heart and soul of Hi Fi - they know it - which is why arguments about it go on and on but get nowhere with each side retreating to their impregnable position.

It constantly amazes and frustrates me this attitude...

 

The subjective extremist: I changed this aspect and I hear a difference therefore it MUST exist even though it can't be measured. That you can't hear what is so obvious to me must make you deaf or unable to hear it due to another reason.

 

The objective extremist*: You made a change and because you knew you expected there to be a change. I've put your change under measurement and there is no difference. It must all be a result of the flawed human hearing.

 

The reality: You changed something in the signal path, that may or may not cause a change in the output at the speakers. Currently science cannot measure the output, but equally the human sensory system is flawed so it is possible your perception is tainted by your knowledge of the change.

 

Eloise

 

Note *: Though in actual face the true objective accepts that there are something that can't be measured and balances listening testing with scientific theory and measurement - that's what being objective means; weighing up ALL the evidence before coming to a conclusion about what is likely.

Eloise

---

...in my opinion / experience...

While I agree "Everything may matter" working out what actually affects the sound is a trickier thing.

And I agree "Trust your ears" but equally don't allow them to fool you - trust them with a bit of skepticism.

keep your mind open... But mind your brain doesn't fall out.

Link to comment
That's fine if someone specifically asks for his information rather than aggressively forcing views and self-professed facts on others.

I'm sorry - but this is the nature of Internet forums...

 

Everyone is entitled to post their views, and repeat them if they feel people are ignoring them or missing the point.

 

As for "self-professed" facts, when he is talking about (for example) the way MP3 is supposed to work - that isn't a self-professed fact, it's a fact of how the Fraunhofer Society intended MP3 to work when they created it.

 

Eloise

Eloise

---

...in my opinion / experience...

While I agree "Everything may matter" working out what actually affects the sound is a trickier thing.

And I agree "Trust your ears" but equally don't allow them to fool you - trust them with a bit of skepticism.

keep your mind open... But mind your brain doesn't fall out.

Link to comment

In my experience there are very few people with an objective outlook who will accept that (2) is a possibility.

 

QUOTE]2) There is a difference but Y doesn't notice it because the equipment wasn't resolving enough to notice the differences.[

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment
In my experience there are very few people with a subjective outlook who will accept that (3) is a possibility.

 

With respect from my experience you are dead wrong. I am a subjectivist, all my acquaintances are subjectivists and we all agree 3 is possible. In fact some blind tests I have been part of have resulted in 3. Manufacturers I know who use blind tests to ensure a difference exists have experienced 3. However many more blind tests I have seen and/or been part of the exact opposite results - they confirmed what we heard sighted. My first exposure to this was when I purchased my speaker cables a few years ago. I had never previously heard differences in cables. The difference sighted was huge. My sister (not an audiophile) who was with me at the time was astonished (as was I) - how could a wire make such a difference - but it did. To be sure I had a blind test done. I even had the trick played on me of saying the cables were changed when it wasn't - I didn't hear a difference and said so. It was easy to pick blind. My sister picked it.

 

The issue I have with objectivists is when I give examples like that rather than accept the obvious conclusion they say it had to be a DBT etc etc. Basically its what I said - they claim all reputable tests show there is no difference - but they decide on what are reputable tests.

 

Thanks

Bill

Link to comment
In my experience there are very few people with an objective outlook who will accept that (2) is a possibility.

 

Yea - and not only that they won't come around and hear systems like mine where it is easily heard. Instead they would rather keep tapping away with posts saying when differences are heard the methodology must be flawed.

 

Thanks

Bill

Link to comment
Though in actual face the true objective accepts that there are something that can't be measured and balances listening testing with scientific theory and measurement - that's what being objective means; weighing up ALL the evidence before coming to a conclusion about what is likely.

 

Not in my experience - not by a long shot. If true I would be inundated with people taking me up on the offer to hear my system to see if what I say is true. Instead they refer me to the DBT's that support their position that no difference exists and anyone that hears them is deluded.

 

If you are an objectivist why not go down to your local Hi Fi store and ask to see if their staff can pick the stuff they sell blind. Expose any who can't do it. It's a much more practical exercise than arguing about it internet forums. I have mentioned this many times - to the best of my knowledge not a single person has ever done it.

 

Thanks

Bill

Link to comment
Not in my experience - not by a long shot. If true I would be inundated with people taking me up on the offer to hear my system to see if what I say is true. Instead they refer me to the DBT's that support their position that no difference exists and anyone that hears them is deluded.

 

Bill

They would rather have their teeth extracted without anaesthetic than be proved wrong ! (grin)

Alex

 

How a Digital Audio file sounds, or a Digital Video file looks, is governed to a large extent by the Power Supply area. All that Identical Checksums gives is the possibility of REGENERATING the file to close to that of the original file.

PROFILE UPDATED 13-11-2020

Link to comment

One guy I know well (he posts under the name Kdoot) was and is very skeptical of this stuff saying for example all USB cables sound the same - and they must sound the same. What sets him apart though and why I count him as a fellow subjectivist is when he hears a clear and obvious difference he accepts it. He heard clear difference is USB cables, and while flabbergasted, does not go into denial like objectivists do. And yes the difference was heard blind - he was so shocked he checked it blind to be sure.

 

Thanks

Bill

Link to comment
With respect from my experience you are dead wrong. I am a subjectivist, all my acquaintances are subjectivists and we all agree 3 is possible.

That's why I said "in my experience"...

 

In fact some blind tests I have been part of have resulted in 3. Manufacturers I know who use blind tests to ensure a difference exists have experienced 3. However many more blind tests I have seen and/or been part of the exact opposite results - they confirmed what we heard sighted. My first exposure to this was when I purchased my speaker cables a few years ago. I had never previously heard differences in cables. The difference sighted was huge. My sister (not an audiophile) who was with me at the time was astonished (as was I) - how could a wire make such a difference - but it did. To be sure I had a blind test done. I even had the trick played on me of saying the cables were changed when it wasn't - I didn't hear a difference and said so. It was easy to pick blind. My sister picked it.

More power to you... Though to be of relevant proof I would ask you (a) what the equipment used was and (b) what cables were being tested and © what the set up was (i.e. how were you blinded)?

 

The issue I have with objectivists is when I give examples like that rather than accept the obvious conclusion they say it had to be a DBT etc etc. Basically its what I said - they claim all reputable tests show there is no difference - but they decide on what are reputable tests.

I think you are talking about the extremes (as perhaps I was).

 

(Again in my experience) the objectivists was to know how the test was done. As I say above, if you are presenting it as evidence, then at a minimum I would ask what equipment was used and what cables were being tested before stamping a "Proven" stamp on the "cables make a difference". Very few people would argue about bell wire vs. Nordost Valhalla (for example); but when it comes to a 5m run of Kimber 8vs at £266 vs Kimber Crystal 24 at £1,030 that's where the arguments get more heated.

 

Eloise

Eloise

---

...in my opinion / experience...

While I agree "Everything may matter" working out what actually affects the sound is a trickier thing.

And I agree "Trust your ears" but equally don't allow them to fool you - trust them with a bit of skepticism.

keep your mind open... But mind your brain doesn't fall out.

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...