Golden ears vs super scopes

[SoundAndMotion]

1 hour ago, jabbr said:

Again just because you didn't happen to measure something doesn't mean that thing couldn't be measured.

You're playing a semantic game with the meaning of "can". Does it mean "possible" or "able" (now). With your definition, I'd be able to say:

I can speak Russian. I can practice law. I can go to the moon. 

...but I should hasten to add, that I never have, and cannot today. I can if: someone teaches me Russian; I go to law school and pass the bar; NASA is willing to prepare me and give me a ride.

With my definition, I'd have to wait until I've learned what I need to know.

On 3/1/2020 at 3:04 PM, jabbr said:

You are not, by any means, limited to using an AP analyzer alone to measure UpTone’s  equipment.

[snip]

Current popular measurement regimens like phase error plots? 1/f noise? 3D output impedance plots? EMI patterns? 

The question is what "thing" do I want to measure. Without a definition for "thing", I wouldn't know how to use an AP to measure it, and I wouldn't know if the popular measures help me with that. 

I would start with careful listening tests to confirm a predictable, verifiable, repeatable "thing". Then I'd try for an unambiguous, non-controversial name. If current hardware/software is up for it, then a measurement method can be created (possibly). Perhaps new hardware/software would need to be developed. Knowing what and how to measure is not a given, so I'd argue we can't measure everything.

 

On 3/1/2020 at 4:02 AM, jabbr said:

Do we believe that everything audible is measurable? Do we know the audibility limits? Which data sways subjectivists (if any)? Which data sways objectivists?

-With your definition, yes, if a measurement method exists or can be developed. With my definition, no.

-Perceptual limits is my field. Do we know some (specifically defined) audibility limits? Yes. Do we know all that are necessary to understand perception of reproduced music and the ability of current equipment to do "well"? I don't know, but my experience tells me to truly doubt it.

-For the last 2 questions, I'm reminded of a thread (since deleted?) on WBF. Someone asked "what would it take for you to change your mind about XYZ". I forgot what XYZ was... cables, DACs or something... A small number of people (incl. me) wrote specific answers. Most answers were of the form: "I know I'm not wrong, so the question doesn't make sense". For both groups, the answer of the hardcore will be: nothing!

[SoundAndMotion]

11 minutes ago, pkane2001 said:

Empirically, we can subtract the two waveforms and listen to the differences. I don't need to know what TIM or jitter or THD or IMD is to be able to tell if two waveforms measure different, by how much, and whether this difference is audible.

 

Hi Paul,

I agree with everything you say, except what I have quoted and bolded. From your work with deltaw, I assume you'll correct this assumption of mine: Always (or nearly so), when you subtract the two (digitized analog) waveforms, the result is never all zeroes. This leads to two problems:

You assume that if you listen to that difference and can't hear anything identifiable, that means you can't hear the difference between the two waveforms. Similarly, you assume that if you generate that delta-wave, and you "know the limits of audibility", you can just declare the difference is inaudible. For any pair of waveforms you've encountered, the assumptions may have been true. But I reject the notion that your assumptions describe the way differential and absolute thresholds actually work. So I reject the idea that you can tell whether this difference is audible with deltaw. I wouldn't be convinced without a listening test. I mentioned that is the starting place for me in my post.

 

So if 2 analog waveforms, distinguishable with a listening test, never null completely, and you can't declare the difference is inaudible, you have not measured the effect.

 

What effect? I'm dangerously close to the argument "if you haven't measured everything and you don't know everything, then you just can't be sure". But I'm quite comfortable walking this tightrope in this way: I do know about the nonlinearities in perception and that superposition does not always apply (not speculation). I do know how nearly all the threshold experiments you use to discuss "audibility" were done (do you?), or perhaps you also do and have a relevant reference. I do know sensory perception is fine-tuned for pattern recognition, not detection or discrimination of simple stimuli or noise(s). The ear is not a good microphone and the eye is not a good camera, but both do their tasks better than their mechanical counterparts. In your favor: most complex stimuli have higher absolute and differential thresholds than simple stimuli. In my favor, not all.

Cheers, SAM

 

3 minutes ago, jabbr said:

 

That is called religion.

 

 

No semantic games. Just because you personally can't measure something doesn't mean that it is unmeasurable (by someone else) ... ditto, so Russian is speakable, Law is practicable and travelling to the moon is possible, just like measuring things  

I agree the hardcore on both extremes have unshakable beliefs that can be called religion. 

 

Your list (derived from mine) of do-ables don't apply to me or anyone with the same level of training as me. For my list, there are people who are better trained. If an effect that can be verified by listening test has never been measured, then no one knows how or has had the training. It may be easy to measure, or not. But you have to know how/what to measure. The semantic game: I say they can't (without more info); you say they can. Some people will agree with me, some with you. There is no need to play with the words anymore.

[SoundAndMotion]

2 hours ago, SoundAndMotion said:

-Perceptual limits is my field. Do we know some (specifically defined) audibility limits? Yes. Do we know all that are necessary to understand perception of reproduced music and the ability of current equipment to do "well"? I don't know, but my experience tells me to truly doubt it.

 

1 hour ago, jabbr said:

Then I trust you understand that a microscope and telescope extend the perceptual limits of the eye, and do amplifiers extend the sensory limits of the ear. An oscilloscope extends the frequency response of the ear upward and similarly the amplitude sensitivity. Shall we continue?

 

Ah! You added something.

 

We should continue... until you make a point. Did i miss it? Extending perceptual limits along one conceptual dimension (e.g. seeing distant things or very small things) does not extend all limits. Even mentioning both telescope and microscope show that; also neither one extends my ability to read a paperback as well as reading glasses.

 

Yes, I don't see how what you write answers what I wrote... I must not have been clear. Probably not worth the effort for either of us. 

 

[SoundAndMotion]

Hi Paul,

I'm reminded of our 2 week long PM exchange in Sept. 2018, which morphed from audibility testing to a discussion of HRTFs. 

17 hours ago, pkane2001 said:

I accept that the audibility question is complex. But I don't accept that it is impossible to study. 

Of course I agree with you, otherwise accepting money for my work would be fraud. I'll add that the complexity does not come from difficult to understand concepts and principles (actually not difficult), but from the sheer number of variables that influence the result, thereby producing a set of results. Obtaining a "complete set" is both prohibitively tedious, but thankfully mostly unnecessary. It is important though to use an appropriate value for your specific goals.
 

17 hours ago, pkane2001 said:

maybe you can help to better formulate the issues with audibility testing and why this area is more complex than many of us think. A better understanding of the problems and issues gives me better ideas about how to go about solving them

I didn't reread our entire exchange, but I found a couple of early quotes that may relate here. I said: "My first question is: what is your goal?" Tailoring an answer to your specific goals and background is much more efficient. On the other hand, with about 15 min. prep, I could prepare 1 or 2 hour-long lectures for a group with comparable backgrounds (colleages, grad, undergrad, 4th grade, kindergarten - I've done all of these), but I won't transcribe such a thing now. We could have a very efficient back-and-forth by phone or FaceTime chat, but...
A couple quick tips:
Finding *the* value for *the* threshold of audibility is meaningless. There is no single value; it depends on the type of stimulus (e.g. sine, noise, music, etc.), the method used (e.g. rising, falling, adjustment, 2AFC, etc.), absolute vs. differential, and must account for significant intra- and inter-subject variability. You wrote: "I believe that 'average' results often reported for others don't apply to me, so I tend to want to try to measure things for myself, with me as a test subject." Trying to test all variables, results in a combinatorial explosion.

 

One important misconception is that a sensory-perceptual threshold is binary: you hear it or you don't. It is statistical, not single-valued, and follows a sigmoidal probability density function, called a psychometric function:

 

The cyan line is not correct. You can make it "look" correct if you choose a very large range for the stimulus level scale, but that would be deceptive. BTW, interesting things happen during the rising part of the curve. 

 

Let me know if I can help more...

 

EDIT: I misstated an opinion in a post yesterday that you address in your first paragraph above. There are things that can be considered inaudible, if you are careful with the assumptions made. It is safe to say that both those with normal hearing and those with freakishly, aberrantly good hearing cannot hear a tone at -16dBSPL (3µP).

[SoundAndMotion]

11 hours ago, pkane2001 said:

 

Hi SAM,

 

Appreciate the continued conversation!  While some of the details have changed since our conversation a few years ago, generally my goal remains the same.

 

Simply stated, I'm after the most realistic audio reproduction possible. The number of variables is large, which is why I've  been taking the long, scenic route to get there, exploring, but also trying to narrow down the things I need to look into. I learn better through this sort of random walk, where I discover things that might or might not be relevant, often going off on tangents, but I'm a firm believer these will give me a better understanding in the long run, or at least provide a different perspective. 

 

Testing for audibility is, of course, a large part of this journey. And perhaps that's where you can help. To simplify the discussion, let me just propose a list of questions that I've formed over the last few years. I've seen some studies with attempted answers, but nothing I can accept as definitive. I've done a large number of ABX tests, but I'm always looking for better ways to test. Sorry to dump all of these on you, and if a voice or video chat works better, we can do that, as well 

 

1. Are there any test protocols that you've feel work best for minor impairment detection/discrimination testing? ABX, Triangular, paired, MUSHRA, etc.? Any new ways that have been proposed recently? As an example, someone suggested playing the two DUTs in stereo (one left the other right ear). I didn't find this test extremely good at discrimination, but perhaps it might be good for detecting phase anomalies.

 

2. Are discrimination tests best conducted with very short snippets and fast switching, or long-term evaluation? (long term here meaning anything over 10 seconds. Sometimes weeks are recommended by our subjective brethren)

 

3. Is there an issue with fatigue in fast-switching tests that must be dealt with, and what is the 'threshold' when that sets in (# of repetitions, length of time)? 

 

4. Are qualitative tests better than the binary discrimination ones in detecting differences? 

 

5. Any other variables in such testing that we, audiophiles, either don't know about or ignore that can have a large influence on the outcome of the test? I know, there are a huge number of variables. Can you list say the top 5 that have the most influence on the outcome?

 

Looking forward to your answers!

Hi Paul,

For a handful of reasons, I'm not in the "right space" to thoroughly and thoughtfully answer your post in written form. I've sent a PM to see if a voice chat might work. Perhaps you or I can/will record it and create a nice post. I'm totally with you on taking the scenic path. That is the path that I and every colleague/friend of mine has taken. The shortcut is overrated: very often muddy and full of bugs, thorns, poop and poison ivy.

But just quickly: I asked "what is your goal?" and you say testing for audibility with the ultimate goal of the most realistic audio reproduction possible. I was unclear. What exactly, specifically is your goal? For example: I want to test my own absolute threshold for SINAD, that is created and varied using this specific method-..., is an entirely different goal from: I want to find the range of estimates of "normal" humans for the angular width of the soundstage using a specific musical sample. And my answers below would need to take that goal into account.

My answers to questions 1 and 2, will start with and branch from asking the specific goal. Although any one test can have more than 1 variable, I strongly recommend harshly minimizing the number. Every other possible variable (answer 5) should be held constant to the best of your ability. Most choices for those involve trade-offs.

Some quickies:
1 - It really, really depends... the idea @Jud used of separating samples to each ear is a trade-off (see @sandyk response above). I participated in his experiment, but found it to be 1 creative step forward, but two back.

2 - Depends on the variable you choose to measure. snippet time and switch times are not the same. Simple detection, like audibility of measured specs (THD, FR, etc.), probably requires echoic memory, so a short switch time. But it is easy to come up with variables/tests that would require longer listening times and greater immunity to long switch times.

3 - There is a fatigue issue for all tests: avoid it! There is no threshold that I know of; it is equivalent to the question of how long can people pay attention. If a test is boring, people must fight to not "drift away". But similarly, if a task is pleasurable, people must fight to focus on the task and not just sit happily listening. I always require the subject announce they are ready for the next trial (allowing zero to several seconds or a minute of self-controlled break, to gather themselves). I force a few minute (5+) break every 15 minutes. I stop for at least a half day, after about 1-1.5 hours testing. My experiments are "lots of fun" for the first 1 or 2 dozen trials, but I usually need 100-300 trials, so it gets boring and I must query and push for alertness and attention.

4 - "in detecting differences" No, because of your stated goal. If I can easily detect the difference, but my preference depends on the snippet of source material (stimulus), I may answer in an apparently random fashion. It is a confound. 

5 - My top 4, for those forgotten, misunderstood or taken for granted...
    Instructions (depending on goal; super-important for me).
    Subject selection. 
    Method, including what statistics will be used: know before start.
    Stimulus selection.   
    Others are also important, but are often thought of: bias control, calibration of all measures, subject-comfort...

Cheers, SAM

[SoundAndMotion]

4 hours ago, pkane2001 said:

SAM (@SoundAndMotion) and I spent a good hour chatting on Skype yesterday about testing, audio thresholds, and experiments. A number of my questions were answered, and I really enjoyed the conversation. In-person (even virtually) works so much better than exchanging forum posts!  Looking forward to future discussions. Thank you, SAM!

Thanks Paul,

I thought it was a great exchange and enjoyed it too. I love chatting about this stuff. Great idea - let me know if I can help with a summary. ...and I hope we’ll continue the conversation.

Cheers, SAM