Poll: Where are you along the cable divide?

[bluesman]

That may provide a reason for being skeptical and questioning the evidence but, IMO, it falls far short as a basis for rejecting it.

I don't reject social sciences as valid disciplines. I reject conclusions based on weak (or no) evidence.

[Jud]

@esldude (Dennis) - Very good summary with respect to ABX testing.

 

15 seconds is very much on the higher side of the references I've seen, most of which say we should expect echoic memory to last in the neighborhood of 4-6 seconds. The one reliable exception seems to be loudness, which as you correctly point out is the reason it's essential to match levels.

[mansr]

But I find it strange that instead of dismissing listening as a valuable assessment tool you have not tried to perfect it, to make it less unreliable or as easily mislead.

 

One other thought about this. It helps to use non-music comparison signals in some ways.

 

Many audible differences are frequency response effects. Using pink noise will usually show that up quite well if you can switch quickly at matched levels. You won't find yourself contemplating the emotional impact thru unit A was better than thru unit B on the pink noise signal. Nor will you describe the extra ease, depth, and space of one vs the other using pink noise. So pink noise is a good quick test signal to use even sighted.

 

Also, the differences are often much more obvious this way. With music the difference might only show up on a few fleeting notes making it rather easy to miss.

 

Before someone points out that frequency response isn't everything, let me say that if there are clear differences in FR, you're wasting your time trying to listen for more subtle effects.

[Jud]

Also, the differences are often much more obvious this way. With music the difference might only show up on a few fleeting notes making it rather easy to miss.

 

Before someone points out that frequency response isn't everything, let me say that if there are clear differences in FR, you're wasting your time trying to listen for more subtle effects.

 

Very much agree with the "few fleeting notes" statement, and would add that as long as you're using pink noise, may as well measure rather than or in addition to ABX.

 

As far as whether everything else is a waste of time if FR isn't right, I figure it doesn't hurt to fix what you can when you can.

[semente]

Yes, I think people ignore when I say I have gotten positives. One example from a few months back. Resampling software. There is a older version I can listen to and don't really think it is different. Yet dotting I's and crossing T's I tried it in Foobar ABX. Felt completely uncertain, and expected random results. Instead I hit 19 of 20 followed by 17 of 20. Looking at what is left from that resampler I would not have expected that. Since sigma delta DACs are sometimes said to have noise floors sensitive to such residuals I tried it on two more DACs all with different chips. 18 of 20 and 19 of 20. So I don't think it was a sigma delta effect. Just a quick note, the resampler in Reaper seems like it slightly roughens sound quality and I can ABX it as well. Not usually 19 of 20, but 15 or 16 of 20 often enough it corroborates that I do hear it.

 

I do think taking enough such tests you are comfortable with them is important. I have run across a couple or three other things I can hear blind while thinking it sounds the same. Doing that prevents you from throwing in the towel when it seems hard and to really start guessing. Then are things I heard different as am doing it blind and do get positive results. The flip side is the much larger number of things I am pretty sure I can hear a difference between and cannot do it better than random guessing. So experience is helpful. Relaxed awareness is helpful.

 

So one of your statements:

But I find it strange that instead of dismissing listening as a valuable assessment tool you have not tried to perfect it, to make it less unreliable or as easily mislead.

 

I have done some messing about with that. NUMBER ONE is you must match levels. Must do it. Being able to switch quickly is the next part to doing good comparisons. If you wish to do sighted comparisons those are still vitally important. You will often find even if you think you hear differences the magnitude of differences is greatly diminished if you match levels and can switch quickly. If you have spent very little time and don't have an opinion yet, you quite often will find yourself flummoxed at things sounding so close to the same even sighted.

 

Beyond that things get so murky that normal listening as an assessment tool is so degraded it would only be useful with truly large differences. Yet I have done it, and had the normal practice common among audiophiles for years that trying something a couple weeks and taking it out of the signal chain is how I knew what I thought of it. And doing so gave me great confidence in my appraisal. That appraisal stood the test of time solidly. (as long as I knew the gear in use)

 

My current opinion is the longer you do that, the more opinion you express even privately, the greater the likelihood of biasing or corrupting your experience. It takes so surprisingly little to plant a germ of an opinion in your head that sighted it will grow and dominate your ability to hear in your head what your ears are actually hearing it becomes impossible to get meaningful results. It can reinforce itself without your knowledge so you never believe it is happening. Even months later without conscious awareness that old template of a pattern that grew inside your mind will spring forth to alter your subjective experience from what is real. A big part of our brain activity is pattern matching. If you give it a shortcut with sighted knowledge, it will in the interest of efficiency pull out the old pattern at a moment's notice to use with what you are hearing.

 

There are a number of accidental experiences that reinforce that. Here is one. Mixing some recorded music I had an idea how I might make an instrument appear on one side, slide to the other, shimmer and then coalesce into a central tight image of that instrument over the first few seconds after it appears. I worked on it for several hours. Finally making changes and doing things I thought I had it. Not quite perfect like I imagined, but pretty darn good. As I was comparing plain to my imaging tricks the phone rang. I had a short conversation, and then hit the play button I had paused. It was the plain version rather than the gimmicked one. So I went to solo the gimmicked version, only I was on the gimmicked version. I played it again, and sure enough it shimmered and then focused. But that stuck in my mind and bothered me. Next day I dropped both versions into Foobar ABX. They sounded the same. I let two other people listen and they didn't know what I was talking about as it sounded the same either way. I had convinced myself and superimposed my desired result onto what I was hearing. For weeks afterward if I knew what it was I heard the imagined effect which I knew for sure was not real. Yet it was there matching a pattern in my mind that couldn't be wiped out.

 

So any way, back to perfecting listening as a valuable tool. Match levels and be able to switch quickly. Moving to long term listening just increases the chance of biasing yourself and starting to build a pattern in your mind. Level matching is essential, and how do you do that between today and tomorrow? How do you match levels if one session was a Saturday afternoon with noisy neighbors doing yardwork, and the other is a quiet Sunday evening? Even if the signal is the same, the noise is very different and effectively like hearing it at a different volume. Your memory beyond about 15 seconds is like an MP3 of the actual aural perception. 90% of the raw aural info is thrown away. Continuing to reinforce whatever pattern of highlighting this vs diminishing that in your memory pattern intensifies itself each time you repeat it that way. And each repeat makes you more confident, comfortable and easy in believing in your memory. It is quite the trap in the right circumstances.

 

Thanks for the thorough description of your methodology and for providing some examples.

I am impressed with your perseverance, you wouldn't catch me listening to the same tune 5 times in a row let alone 20.

I don't think that I am fit to participate in such trials.

 

I agree that the memory is easily 'confused', in AB comparisons, at least that's been my experience.

That's why I am asking for you to experiment is something different.

 

Ignore level matching and direct comparison for a moment.

Listen to one single Redbook track that Mario provided intensively to familiarize yourself with it's sound (tone, decay, transients, etc.) as much as you possibly can.

After a week listen to the 24/96 and see if you notice any improvements.

Don't expect anything, just listen.

 

Then, if you wish, take one of your own ABX.

 

R

[semente]

But I find it strange that instead of dismissing listening as a valuable assessment tool you have not tried to perfect it, to make it less unreliable or as easily mislead.

 

One other thought about this. It helps to use non-music comparison signals in some ways.

 

Many audible differences are frequency response effects. Using pink noise will usually show that up quite well if you can switch quickly at matched levels. You won't find yourself contemplating the emotional impact thru unit A was better than thru unit B on the pink noise signal. Nor will you describe the extra ease, depth, and space of one vs the other using pink noise. So pink noise is a good quick test signal to use even sighted.

 

This I agree with.

It's taken me a while to be able to focus on sound / ignore the music when performing sound assessments but I am getting more confident.

But you will need music if you're to evaluate anything other than tonal balance.

 

R

[mansr]

Very much agree with the "few fleeting notes" statement, and would add that as long as you're using pink noise, may as well measure rather than or in addition to ABX.

 

As far as whether everything else is a waste of time if FR isn't right, I figure it doesn't hurt to fix what you can when you can.

If your FR is way off even identifying other problems can be challenging.

[esldude]

Thanks for the thorough description of your methodology and for providing some examples.

I am impressed with your perseverance, you wouldn't catch me listening to the same tune 5 times in a row let alone 20.

I don't think that I am fit to participate in such trials.

 

I agree that the memory is easily 'confused', in AB comparisons, at least that's been my experience.

That's why I am asking for you to experiment is something different.

 

Ignore level matching and direct comparison for a moment.

Listen to one single Redbook track that Mario provided intensively to familiarize yourself with it's sound (tone, decay, transients, etc.) as much as you possibly can.

After a week listen to the 24/96 and see if you notice any improvements.

Don't expect anything, just listen.

 

Then, if you wish, take one of your own ABX.

 

R

 

Well that was once my standard methodology. And unless I am forgetting a few instances, I always heard a difference. That is how you become someone who says, "everything matters and no two pieces of gear sound the same". If I do this with Mario's excellent tracks I am pretty sure they will sound different.

 

People complain listening in blind testing is different than normal listening. And it is. Your described method however has the same problem. I listen for a week, and then play a different version. While you say not to expect anything just listen when I switch, that has already been poisoned when you said familiarize myself with tone, decay, transients. When I listen to the new piece I will be unable to not key on decay, transients etc. Or perhaps size of soundstage. That too isn't the same as listening over the previous week.

 

I am not being obstinate if I don't try your method. It simply is one I have done many times and I am pretty sure of the results. It will sound different.

 

Now this is conjecture on my part, I don't know it to be what happens. What I think happens is you listen and build up a firm long term memory pattern over a week or two weeks whatever. Then you try a different version or different wire whatever you are trying out. Firstly you can't avoid listening for differences you simply can't make the change and listen with no expectations or at least I can't. So now you are comparing the direct echoic memory which is 15 seconds or less (probably less as Jud pointed out) with long term memory. Maybe you key on sound stage or transients whatever is your preference. And it sounds different. Because it is different. Long term memory has both thrown out some of the direct info, and been shaped by prior experience like, "this cable always excelled in a deep sound stage", and is prone to being wrong due to other uncontrolled factors like some people's practice of inching up the volume just a bit to listen closely when they do this. Sometimes you hear a difference you are willing to spend money on, sometimes you think it too slight to be worth a change, sometimes you think it is a difference that has pluses and minuses. In my experience it is always different if you do it this way. I believe this is because you have opened the experience up to the maximum possible number of factors to muddy the results.

 

So long term listening is fine for large enough differences. It is fine for the gestalt of differences that are real. Like listening to two different recordings of a symphony. Or gear that is very different like a triode amp vs SS amps. Or recognizing your Mom's voice even over a low fi phone connection rather than someone else. When it comes to very small or non-existent differences there are better methods due to our propensity to find a difference.

[Tony Lauck]

That may provide a reason for being skeptical and questioning the evidence but, IMO, it falls far short as a basis for rejecting it.

 

Anything done at the 5% level is suspect, regardless of the field. It leaves too much room for luck and "file drawer" studies. Even with wonderful statistical significance there is still the opportunity for fraud. This doesn't even get into questions of "group think" facilitated by peer review and centrally controlled sources of funding.

 

https://www.newscientist.com/article/dn7915-most-scientific-papers-are-probably-wrong/

[esldude]

Anything done at the 5% level is suspect, regardless of the field. It leaves too much room for luck and "file drawer" studies. Even with wonderful statistical significance there is still the opportunity for fraud. This doesn't even get into questions of "group think" facilitated by peer review and centrally controlled sources of funding.

 

https://www.newscientist.com/article/dn7915-most-scientific-papers-are-probably-wrong/

 

Yes, that is why I think you need 3 sigma results instead of 2 sigma. Or better yet always require replication if you stop at 5% (2 sigma). Which it just so happens is statistically about what 3 sigma is.

 

I don't know if any journals would find it practical. With all the problems in replication maybe they should not publish unless two separate groups have replicated something with similar results that reject the null hypothesis. But not my area of work and outsiders often don't get all the issues involved.

[wgscott]

https://www.newscientist.com/article/dn7915-most-scientific-papers-are-probably-wrong/

 

The probabilistic form of the Liar's Paradox.

[sandyk]

If I do this with Mario's excellent tracks I am pretty sure they will sound different.

Then why don't you try them and see for yourself as suggested ? Mario's files in the different formats are free to DL if you ask him.

[gmgraves]

Very much agree with the "few fleeting notes" statement, and would add that as long as you're using pink noise, may as well measure rather than or in addition to ABX.

 

As far as whether everything else is a waste of time if FR isn't right, I figure it doesn't hurt to fix what you can when you can.

 

 

Allow me to point out that the "inventor" of the ABX comparator, Arnie Kruger, believes that everything except speakers and other transducers sounds the same these days (amps, preamps, DACs, CD players, media players, etc.). So that should tell us something about the results obtained from ABX!

[esldude]

Then why don't you try them and see for yourself as suggested ? Mario's files in the different formats are free to DL if you ask him.

 

I already have them. Circumstances presently prevent me from having time to audition them this way.

 

I would have hoped my explanation was clear however as to why I think they will sound different. And why I think that isn't big news. So maybe doing an audition that way would be a surprise and I would hear no difference. Which tells me what exactly?

[Allan F]

Allow me to point out that the "inventor" of the ABX comparator, Arnie Kruger, believes that everything except speakers and other transducers sounds the same these days (amps, preamps, DACs, CD players, media players, etc.). So that should tell us something about the results obtained from ABX!

Some might even suggest that it tells you almost everything about the validity of those results or, more importantly, the lack thereof.

[Allan F]

Anything done at the 5% level is suspect, regardless of the field.

Which, of course, begs the question of where to draw the line for the real significance of anything done above the 6% level.

[gmgraves]

Some might even suggest that it tells you almost everything about the validity of those results or, more importantly, the lack thereof.

 

OK, I was being conservative in my criticism. Yes, it tells us almost everything about ABX results! Remember, Meyer and Moran used ABX to prove that DSD can be decoded and re-coded as 44.1/16 bit PCM over and over again in a daisy chain arrangement, and nobody (statistically) will be able to tell multi generation copies from the original DSD!

[semente]

Well that was once my standard methodology. And unless I am forgetting a few instances, I always heard a difference. That is how you become someone who says, "everything matters and no two pieces of gear sound the same". If I do this with Mario's excellent tracks I am pretty sure they will sound different.

 

People complain listening in blind testing is different than normal listening. And it is. Your described method however has the same problem. I listen for a week, and then play a different version. While you say not to expect anything just listen when I switch, that has already been poisoned when you said familiarize myself with tone, decay, transients. When I listen to the new piece I will be unable to not key on decay, transients etc. Or perhaps size of soundstage. That too isn't the same as listening over the previous week.

 

I am not being obstinate if I don't try your method. It simply is one I have done many times and I am pretty sure of the results. It will sound different.

 

Now this is conjecture on my part, I don't know it to be what happens. What I think happens is you listen and build up a firm long term memory pattern over a week or two weeks whatever. Then you try a different version or different wire whatever you are trying out. Firstly you can't avoid listening for differences you simply can't make the change and listen with no expectations or at least I can't. So now you are comparing the direct echoic memory which is 15 seconds or less (probably less as Jud pointed out) with long term memory. Maybe you key on sound stage or transients whatever is your preference. And it sounds different. Because it is different. Long term memory has both thrown out some of the direct info, and been shaped by prior experience like, "this cable always excelled in a deep sound stage", and is prone to being wrong due to other uncontrolled factors like some people's practice of inching up the volume just a bit to listen closely when they do this. Sometimes you hear a difference you are willing to spend money on, sometimes you think it too slight to be worth a change, sometimes you think it is a difference that has pluses and minuses. In my experience it is always different if you do it this way. I believe this is because you have opened the experience up to the maximum possible number of factors to muddy the results.

 

So long term listening is fine for large enough differences. It is fine for the gestalt of differences that are real. Like listening to two different recordings of a symphony. Or gear that is very different like a triode amp vs SS amps. Or recognizing your Mom's voice even over a low fi phone connection rather than someone else. When it comes to very small or non-existent differences there are better methods due to our propensity to find a difference.

 

Perhaps we are indeed different...

I don't care for "soundstage" and never pay any attention to it.

Besides, I find comparisons a very tedious affair and don't like spending money.

I avoid replacing anything in my system unless I have identified shortcomings and hopefully know the causes.

 

Tonality or frequency variations are reasonably easy to determine with recordings I'm familiar with; and yet I have used pink noise and a mic to position my speakers.

It's other types of distortion that are a bit more difficult to pin down.

 

What I do is listen attentively to different attributes of sound, just as I would measure different parameters in order to better characterize performance; I try to be as objective as possible and avoid being influenced or emotionally disturbed by the music - I listen to sound not the music.

 

There is some form of comparison involved but it's not the same kind as the one we do when A-B'ing; it's very important that we set performance benchmarks, through listening to high performance equipment and that we used quality recordings of acoustic unamplified music.

And it doesn't matter if the test is blind or not; just as you've learned to distrust sighted tests I have learnt that brand, price and looks have no relation with performance.

 

I have no doubt that measuring instruments are more sensitive, unbiased, and that through measurements we are able to quantify whatever is being measured.

Some measurements even have a reasonably direct correlation with listening, others not as much.

For me, listening and measurements are equally important tools that complement each other.

 

R

[PopPop]

When you get that palpable sense of your carpet getting wet from the condensation dripping out of Al Molina's trumpet; is there a tool to measure that?

[semente]

When you get that palpable sense of your carpet getting wet from the condensation dripping out of Al Molina's trumpet; is there a tool to measure that?

 

I doubt you'd "get that palpable sense" in a live performance...I bet he was close-mic'ed in that recording.

 

Either that or you were spilling the beer...

 

R

[PopPop]

It's happened before. [emoji15]

[semente]

You're not alone there:)

[Jud]

Allow me to point out that the "inventor" of the ABX comparator, Arnie Kruger, believes that everything except speakers and other transducers sounds the same these days (amps, preamps, DACs, CD players, media players, etc.). So that should tell us something about the results obtained from ABX!

 

Too general. There's some very good science to tell us when we can expect ABX to be of value and when not.

[sandyk]

So maybe doing an audition that way would be a surprise and I would hear no difference. Which tells me what exactly?

 

That would simply confirm what many people who are able to hear these differences have long suspected. (grin) Either you are unable to hear the differences, whether due to your hearing (highly unlikely) , your inability to overcome your heavily ingrained negative Expectation Bias, or something in your equipment chain is letting you down. Additional DSP processing perhaps, or the use of a Class D amplifier which is spewing RF /EMI back into the A.C. mains sewer, despite otherwise measuring impeccably?

[mansr]

Allow me to point out that the "inventor" of the ABX comparator, Arnie Kruger, believes that everything except speakers and other transducers sounds the same these days (amps, preamps, DACs, CD players, media players, etc.). So that should tell us something about the results obtained from ABX!

In a modern amp, any deviation from a flat, linear response is intentional.