Who's afraid of DBTs

[Daudio]

I am not afraid of DBT's. I don't see how they pose any threat to me. I just couldn't care less about them.

 

With apologies, IMO this thread has become akin to one more boring debate about how many angels can dance on the head of a pin. The neo-Luddites of the world are proudly at peace.

 

Allan,

 

I agree, not afraid of DBT's, and don't give a damn about them either.

 

re. this thread - It reminds me of a statement I read concerning a political troll: "he is just scoring points in a game no one else is playing".

 

Unfortunately, some here are playing along, allowing the OP to score points in his juvenile game. Through, to be fair, some interesting information and points have been brought out about DBT's: FN/FP's and such. That may be of some value to a few CA members, but I don't see any significant improvement, change, or education, in the OP.

 

As usual, playing a trolls game tangles one into a rigged 'logical' system that is unwinable. It only amuses the troll, and wastes the others time. IMO, the only way to approach these situations is to jump out of their tightly constrained systems, to a highly level, where their errors, inconsistencies, blind spots, and perhaps even motivations, can be seen.

 

Thus my earlier response to the thread question (implied along with the insult) was to talk about the inconsequential impact of DBT on the audio industry in three and a half decades or more.

 

OK, I'm done... continue discussing angels, dancing, pins, and DBT's

[Superdad]

Wonderful stuff, Peter - motivates me to BS right back.

 

OK, first of all, to repeat something: No one here is doing Science. Sorry, but I don't see anyone here with the facilities of a large academic institution including millions of dollars worth of test apparatus and employees to construct whatever test apparatus you can think of but don't have, going through Institutional Review Boards since your experiments will have human subjects, having your work extensively peer reviewed by experts in the field and then published by recognized academic journals. People like Bill Scott know what this entails, and he knows that no one here and almost certainly no one at HA is doing anything on this level. So no one here can claim to be on the side of Science. That's why my finicky insistence on things like false negative rates and consistency of proposed measurement tools, if we are using the "S word."

 

Now for just comparative listening that consumers are trying to make as reliable as possible for themselves, heck, DBT away. Does anyone here have an example of doing DBT in the course of making an audio purchase decision? The material about echoic memory makes me curious about how this would work in practicality. In fact I have some thoughts about a fairly easy blind test I might want to try out on some members here if I have the time.

 

In turn, I have no personal claims about the effectiveness of my own methods. I know I've wound up with a system I like a lot, that is worth (in terms of cost to replace) a great deal more than I paid for the components. I get tremendous pleasure from my music nearly every day, and feel quite lucky to be able to do so. But could I have had the same enjoyment for less? How could I honestly say that wasn't possible? And even if I did wind up with an excellent system relatively inexpensively, that could have been dumb luck - I may simply have selected from among many substantially equivalent choices.

 

But there is a level between doing Science and consumers running around doing comparisons that may or may not help them in their purchase decisions. (Whatever "side" you're on, it's the other guys I'm talking about doing comparisons that won't help! ) This level I'll call Good Engineering. It's where people are doing what I bolded above in Peter's post. They've got the technical knowledge to know what *should* occur, and the long experience to be able to identify it when it *does* occur. These are folks like Peter, Miska, Gordon Rankin, and John Swenson. It doesn't mean they're always right, or that they may not from time to time chase ideas that turn out not to make any difference. It means they're more likely to be able to make progress than those of us who lack that same degree of technical knowledge and experience.

 

The technical knowledge is invaluable. (Peter, in English that paradoxically means "extremely valuable.") As soon as you begin to substitute dogma for technical knowledge, you're at a tremendous disadvantage, because no piece of equipment or music file exists in isolation. Taking Dennis's experience with the Soundlab speakers and the amp as an example once again, if Dennis knew nothing about impedance he could easily think the amp itself was just crap, and might advise a friend against them, when the friend owned different speakers that were a much easier load throughout the frequency range. Or he could have thought DBT would tell him everything he needed to know, and the frequency range at which the amp strained with the Soundlabs might not have been in his test material. For some people, MIT cables may prevent amp oscillation. People with such systems might switch in other cables, and as long as the system doesn't send the amp into oscillation, they'll think cables can't make a difference to the sound. In other systems, the particular properties of MIT cables and their boxes might have an audible impact. When these people switch in other cables, they'll believe cables do make a difference, and anyone who doesn't believe it is a cloth-eared idiot. Or like me, you can compare a fancy USB cable to a "printer cable" with one DAC and think "Wow, how can a simple digital cable make such a difference?" And then you can compare them with Peter's DAC and think "Hmm, digital cables don't make such a big difference." (That is, I heard a difference between the "printer cable" Peter supplied and the cable I normally use, but the difference I heard was much smaller than it was with my own DAC. I imagine Peter could tell us some of the reasons for that.)

 

I notice this in law. Non-lawyers tend to over-generalize each specific example to far too many situations where it doesn't apply. Lawyers have the technical knowledge and experience to understand which of many different legal principles is appropriate for a given set of facts. Similarly, the discussions among people in this forum who aren't audio engineers have a tendency to be more general and dogmatic, while the remarks of professionals like tailspn, John Swenson, Peter, etc., tend to concentrate on the specifics. As Peter says above, "I should be able to know from the technical characteristics...where differences should be."

 

So go ye and perform whatever sorts of comparisons make sense to you and are practical for you. But perhaps also try to gain technical knowledge if and when you can, and be duly humbled to realize how little of it you really have, compared to someone who's gone through the requisite education and experience.

 

 

Great post Jud! You have the patience of a politician (I mean that in a good way, as politicians are some of the only folks willing to keep explaining things to people they will never win over).

[trithio]

One other thing I should say that my long screed was about - probably the main thing, in fact: No matter what our leanings, toward DBT or otherwise, as much technical know-how as we can get is a good thing. Of course it's good to be cautious about the sources we choose to learn from.

 

I am not sure what exactly are you still looking for. Enough DBT samples were posted already, enough related literature and science to demonstrate that those things are as well thought as they could be. Or need to be. Also enough was said about the related FN/FPs too. Also enough DBT samples to be found in the wild. The good the bad and the ugly kind. Of course Some are better than others but the important point is that they all paint a pretty consistent picture.

So, what exactly is still missing for you?

[trithio]

Another big one. I did not read much but seems a lot more inclined towards objectivist positions

https://groups.google.com/forum/m/#!msg/rec.audio.high-end/jC8gZJt0KEk/EldGj2NNeOsJ

[mmerrill99]

Another big one. I did not read much but seems a lot more inclined towards objectivist positions

https://groups.google.com/forum/m/#!msg/rec.audio.high-end/jC8gZJt0KEk/EldGj2NNeOsJ

 

Who are you trying to convince, yourself?

 

This thread is dead. All the logical arguments for why DBTs are judged to be of unknown reliability (lack of false negative controls) have been given & no counter argument made as to the reliability of DBT results.

 

You have two ways to resolve this:

- produce FN statistics for DBTs to overcome this issue

- continue using unreliable DBTs while admitting that they are no more reliable than anecdotal reports.

 

I can predict that neither of these options will be chosen - just more hand-waving, pseudo-science posturing, rehashing of hackneyed sound-bites from audio forums, anything to avoid the self-examination necessary

[Jud]

Who are you trying to convince, yourself?

 

This thread is dead. All the logical arguments for why DBTs are judged to be of unknown reliability (lack of false negative controls) have been given & no counter argument made as to the reliability of DBT results.

 

You have two ways to resolve this:

- produce FN statistics for DBTs to overcome this issue

- continue using unreliable DBTs while admitting that they are no more reliable than anecdotal reports.

 

I can predict that neither of these options will be chosen - just more hand-waving, pseudo-science posturing, rehashing of hackneyed sound-bites from audio forums, anything to avoid the self-examination necessary

 

I don't know that we can conclude DBTs in audio are "no more reliable than anecdotal reports." I did say there isn't a sufficient body of work for me to think of them as capital-S "Science," but that doesn't mean they can't be useful to those choosing to employ them, nor that *positive* results, for instance, couldn't be more reliable than positive results from a sighted test.

 

Edit: BTW, I'm still thinking of and hoping to find time this weekend (somewhere in the middle of a long "honey do" list) for a simple blind test that we could have some fun with.

[Allan F]

I don't know that we can conclude DBTs in audio are "no more reliable than anecdotal reports." I did say there isn't a sufficient body of work for me to think of them as capital-S "Science," but that doesn't mean they can't be useful to those choosing to employ them, nor that *positive* results, for instance, couldn't be more reliable than positive results from a sighted test.

 

But, hopefully, we can conclude that auditory experiences do not have to be confirmed by DBTs in order to be real.

[Jud]

But, hopefully, we can conclude that auditory experiences do not have to be confirmed by DBTs in order to be real.

 

I'd agree with this. In fact, in another thread, I've just asked whether folks want to hear about sighted but independent listening tests by different people coming to identical conclusions.

[mmerrill99]

I don't know that we can conclude DBTs in audio are "no more reliable than anecdotal reports." I did say there isn't a sufficient body of work for me to think of them as capital-S "Science," but that doesn't mean they can't be useful to those choosing to employ them, nor that *positive* results, for instance, couldn't be more reliable than positive results from a sighted test.

In the absence of any statistics that show the error rate for false negatives errors, we can't really conclude much of anything about the results. It's not like the issue of false negatives is something that was plucked out of the air - there's very good reasons to be concerned about this issue.

 

But I don't see the value of running a test that is suspected of an unknown skew towards a negative result. If we get a positive result in such a test what does it tell us - that the difference is so gross & noticeable that it overcame this neg skew - that the test subject has great powers of concentration & training to overcome the negative skew? I'm not sure what it tells us about the actual devices or audio under test?

Edit: BTW, I'm still thinking of and hoping to find time this weekend (somewhere in the middle of a long "honey do" list) for a simple blind test that we could have some fun with.

Cool

[Jud]

In the absence of any statistics that show the error rate for false negatives errors, we can't really conclude much of anything about the results. It's not like the issue of false negatives is something that was plucked out of the air - there's very good reasons to be concerned about this issue.

 

But I don't see the value of running a test that is suspected of an unknown skew towards a negative result. If we get a positive result in such a test what does it tell us - that the difference is so gross & noticeable that it overcame this neg skew - that the test subject has great powers of concentration & training to overcome the negative skew? I'm not sure what it tells us about the actual devices or audio under test?

 

What if the positive result concerns one of those hoary old questions like whether there are audible differences among cables? Or remember the Mike Moffat quote I ran earlier in the thread where he said a blind test with short segments of music was useless, but he passed multiple times when he could listen to 30 minutes of music? What if there was a positive result on a test like that?

[esldude]

But, hopefully, we can conclude that auditory experiences do not have to be confirmed by DBTs in order to be real.

 

It has been confirmed many times, that humans have a real experience of auditory difference when there was no difference to experience.

 

It can be confirmed that humans have a real experience of an auditory difference when there is a real difference to experience.

 

That means there is an area in between where humans cannot fully trust a heard difference is a real difference. You need some outside experience confirmation to be sure once past certain thresholds. That can be measurements, understanding of the physical phenomena of sound, or in some cases DBT results. There is no getting around the fact you get into that area of perceived differences being nevertheless uncertain. Comforting idea that if I hear it then it is so. Usually it is so. But not always.

[mmerrill99]

But, hopefully, we can conclude that auditory experiences do not have to be confirmed by DBTs in order to be real.

Absolutely! In fact the type of audio DBTs being discussed here - A/Bing of small snippets of audio - are focussed on a particular subset of our auditory perception & I've seen it suggested that this ignores some of what is known about auditory perception - auditory scene analysis (ASA). ASA is how we make sense of the auditory signals that we perceive - how we decide on which groups of signals are to be bound togetehr as coming from the same object (auditory grouping) - how we follow auditory objects over time (auditory streaming). Without consideration of this aspect of hearing in a DBT test, we are just looking at a subset of audio perception.

[mmerrill99]

It has been confirmed many times, that humans have a real experience of auditory difference when there was no difference to experience.

 

It can be confirmed that humans have a real experience of an auditory difference when there is a real difference to experience.

 

That means there is an area in between where humans cannot fully trust a heard difference is a real difference. You need some outside experience confirmation to be sure once past certain thresholds. That can be measurements, understanding of the physical phenomena of sound, or in some cases DBT results. There is no getting around the fact you get into that area of perceived differences being nevertheless uncertain. Comforting idea that if I hear it then it is so. Usually it is so. But not always.

The problem here, esldude, is that measurements are very unlikely to tell us if we will hear an audible difference (look at all the amps with perfect specs sounding different), DBTs, with dubious, unreliable results are unlikely to tell us if we can hear an audible difference, understanding of physical phenomena of sound needs a lot more research before we understand auditory perception & what we perceive as sounding different

[mmerrill99]

What if the positive result concerns one of those hoary old questions like whether there are audible differences among cables?

Yes, I would be more convinced by two separate sighted listening describing the same impressions of the sound between two different cables.

 

But what often happens next in these cases is that someone then tries to "confirm" what they heard sighted by using a blind test of the type described here, which 99% of the time will fail & because of the belief system built up around DBTs (the pseudo-science BS) they begin to doubt themselves & guess what they will then create an expectation bias that there is no difference to be heard & the next time they test the cables sighted they will hear no difference. QED. It's an insidious & clever trick of creating negative expectation biases in listeners

Or remember the Mike Moffat quote I ran earlier in the thread where he said a blind test with short segments of music was useless, but he passed multiple times when he could listen to 30 minutes of music? What if there was a positive result on a test like that? That's why controls for false negatives are needed - so that people aren't being duped any more.

Yes, longer term listening, is a different type of listening - it's relaxed & unforced & picks up differences that forced, focussed listening won't. It might be that the differences are not nailed down to specifics but that they are nebulous - felt "more involved", "more of the emotions of the piece came through", etc These holistic impressions don't sit well with the types of DBTs being spoken of here - the A/B short snippet test

[Paul R]

It has been confirmed many times, that humans have a real experience of auditory difference when there was no difference to experience.

 

It can be confirmed that humans have a real experience of an auditory difference when there is a real difference to experience.

 

That means there is an area in between where humans cannot fully trust a heard difference is a real difference. You need some outside experience confirmation to be sure once past certain thresholds. That can be measurements, understanding of the physical phenomena of sound, or in some cases DBT results. There is no getting around the fact you get into that area of perceived differences being nevertheless uncertain. Comforting idea that if I hear it then it is so. Usually it is so. But not always.

 

This is really where we disagree a bit. Yes, it is certainly possible that someone heard a sound with no possible physical stimuli. This is actually pretty normal. You have to be able to do this to have any auditory memory at all. And no, it is not a direct memory of a sound, or it is *usually* not a direct memory of a sound. It is a synthesis of many elements combined into a memory that allows you to recognize a sound, and that sound originally had to be a physical stimuli.

 

But it is also just about as probably that someone will *not* hear a sound that is really there during blind testing. Yep, that is the most common occurrence of all. Listen hard, but still miss it, whatever "it" might be in this case.

 

That's why these ABX tests that the gasbag forum is so hot on are rather useless in absolute terms. And certainly not the standard to judge equipment upon. Now, you add in training for what to listen to, including guided listening if appropriate, and voila! Those test results *do* seem to change. Of course, then one hears nothing less than absolutely enraged squeals because the pure ABX test has been polluted and corrupted somehow to "give false readings."

 

I really do like your idea of a baseline test, but with training added in. But just any old DBT from out there? Good only for personal usage, not as a guideline or reference.

 

-Paul

[Superdad]

Yes, longer term listening, is a different type of listening - it's relaxed & unforced & picks up differences that forced, focussed listening won't. It might be that the differences are not nailed down to specifics but that they are nebulous - felt "more involved", "more of the emotions of the piece came through", etc These holistic impressions don't sit well with the types of DBTs being spoken of here - the A/B short snippet test

 

While I find that longer term listing gives me more appreciation and insight into what a choice has accomplished (not to mention more pleasure as I wander away from my test tracks!), I exclusively use sighted A/B/A testing (with the first minute of several very distinct and different acoustic tracks with difficult timbres and attacks) to make any better/worse decision. Instruments such as piano, (like Bill Evans' very raw piano and nuanced touch playing on "Peace, Piece" off the "Everybody Digs…" XRCD) can be ultimate litmus tests of tone and realism.

 

 

Just yesterday I spent 90 minutes comparing the SQ of cheap wall wart and tabletop low voltage power supplies (both linear and SMPS) to choose one to include with our forthcoming USB Regen device (which already has John Swenson's favorite ultra-low-noise regulators on it).

I had spent a lot of time researching and ordering candidates for consideration (based on limited available specs and schematics), but when I asked John if he wanted to measure some of them, he told me it would be faster and more relevant for me to just listen to them. (And yes, he has the gear and knowledge to measure power supplies well beyond just noise and ripple--to include broadband noise and crap kicked in both directions by them.)

Our big JS-2 was the clear and obvious benchmark, but it was not at all hard to rank each of the others in order below it. Guess, what? The $11 Mean Well GS25A tabletop and the $4 Fujia Appliance wall wart that came with the Geek Pulse--both SMPS--handily beat the Jameco regulated linear wart I had previously been using with the Regen. Was not the result I had wanted or expected, but that's the way it was.

[Daudio]

auditory scene analysis (ASA). ASA is how we make sense of the auditory signals that we perceive -

 

mm,

 

Ah ! Thanks for that, new to me, and very interesting to dig into !

 

The model name 'Auditory Scene Analysis' fits in nicely with that of 'Soundscape' - from Murray Schafer's "The Tuning of the World". Two sides of a coin, but different time scales.

 

Cheers,

[mmerrill99]

Let me play devil's advocate here & ask did you do your testing blind, sd?

[mmerrill99]

Indeed, ASA & other research into auditory perception are very relevant to our hobby & to any serious attempts at understanding audio & the limitations of audio tests.

I've seen it suggested that the hearing thresholds should be established with more complex signals thane the clicks/tones used up to now.

[Daudio]

Yes, it is certainly possible that someone heard a sound with no possible physical stimuli. This is actually pretty normal. You have to be able to do this to have any auditory memory at all. And no, it is not a direct memory of a sound, or it is *usually* not a direct memory of a sound. It is a synthesis of many elements combined into a memory that allows you to recognize a sound, and that sound originally had to be a physical stimuli.

Paul,

 

Thanks for that description, very thought provoking

 

Reminds me of my recent Pink Floyd - live version -versus- my album version memory experience. It was not an internal 're-hearing', but various elements sparking the perception of version difference, with only a faint hint of those original parts of the music in my head, like a shadow perhaps. Nothing like an echo. Another aspect is that it simple happened, unbidden, as I enjoyed listening, totally uncritically.

[Allan F]

That means there is an area in between where humans cannot fully trust a heard difference is a real difference. You need some outside experience confirmation to be sure once past certain thresholds. That can be measurements, understanding of the physical phenomena of sound, or in some cases DBT results. There is no getting around the fact you get into that area of perceived differences being nevertheless uncertain. Comforting idea that if I hear it then it is so. Usually it is so. But not always.

 

On a number of occasions you have posted that you used to hear differences and it was subsequently proven to your satisfaction that those differences did not exist. Since that time, do you still hear the same type of differences, but question their authenticity? Or do you no longer hear them?

[esldude]

mm,

 

Ah ! Thanks for that, new to me, and very interesting to dig into !

 

The model name 'Auditory Scene Analysis' fits in nicely with that of 'Soundscape' - from Murray Schafer's "The Tuning of the World". Two sides of a coin, but different time scales.

 

Cheers,

'

 

http://www.amazon.com/Auditory-Scene-Analysis-Perceptual-Organization/dp/0262521954/ref=sr_1_1?s=books&ie=UTF8&qid=1422654453&sr=1-1&keywords=auditory+scene+analysis

 

This has been suggested as an excellent treatment of the subject. I have not had hands on this book. And it is rather pricey, written as a textbook etc.

 

Spatial Hearing - Revised Edition: The Psychophysics of Human Sound Localization: 9780262024136: Medicine & Health Science Books @ Amazon.com

 

This is an older treatment of related material in a sense. I have read excerpts from it. It is excellent. It too is more of a textbook. It is dense and not for casual reading.

[mmerrill99]

Here's a good overview with music as the reference http://www.ems-network.org/ems09/papers/mountain.pdf

[esldude]

On a number of occasions you have posted that you used to hear differences and it was subsequently proven to your satisfaction that those differences did not exist. Since that time, do you still hear the same type of differences, but question their authenticity? Or do you no longer hear them?

 

Under certain conditions I do still hear them. You might think that strange. One is the case of interconnects. There just isn't anything in normal interconnect to sound different (barring very odd impedances of equipment involved). Plug 'em up sighted, look at them, listen to them they can seem to sound different. Do it again blind, they are all the same.

 

It gets me to thinking about things others don't seem to wish to be bothered with. For instance, if cabling upgrades make even 1% improvement, even subjective improvement, and we get new cables supposedly better than ever before every couple of years, by now versus plain cable we should be experiencing a large 30-40% difference. And some will claim that and more.....sighted. Such a difference should be childsplay to hear blind. Even easier to measure something different. Yet you can do neither.

 

All of this sort of thing plays out almost in identical fashion in other areas. Somehow our hobby is different, wine tasting is different, Italian violins are different, recipes for food are different, and on and on and on.

 

I understand how it is so attractive. Imagine if it could be proven beyond all doubt that the ACME amplifier is fully and completely transparent under all conditions, signals and loads. Ditto for everything except microphones and speakers. The only way for something to be different is non-transparncy or matching it at lower price or with better cosmetics or features. That would be pretty boring. Audiophiles would not identify with their system as an extension of their personality and it wouldn't be nearly so satisfying.

 

Now I think we are closer to the above scenario rather than farther. Which is why mixing up preference with fidelity is such a bad thing to do.

[mmerrill99]

redundant post