When do measurements correlate with subjective impressions

[Bill Brown]

17 hours ago, kumakuma said:

 

Life's too short to waste it on poorly recorded music. 👺

 

Oh, I disagree strongly.  How then could I listen to Charlie Parker?  Louis' Hot Fives and Sevens?  Toscanini?

 

I would rather listen to those on an AM radio than, what did JGH call it?- well-recorded "Tibetan nose flute music" or something like that.

[fas42]

13 hours ago, Confused said:

My girlfriend sounds different in different rooms, depending on the room acoustics.  Does that mean that she is not properly sorted?

 

Well, if the acoustics get so bad that she sounds like an old flame, and you blurt out that name - then your girlfriend may decide to sort you out ... 😉.

 

Edit: Hadn't come across kumakuma's reply yet, when I posted this.

[Confused]

2 minutes ago, fas42 said:

 

Well, if the acoustics get so bad that she sounds like an old flame, and you blurt out that name - then your girlfriend may decide to sort you out ... 😉.

I shall rebuff all these humorous responses with a Pulp song......

 

 

[fas42]

7 hours ago, jabbr said:

I’d like to see if @pkane2001’s distort software can simulate this. I would want to first upsample to 24/384, then apply a phase noise plot, then send to a low jitter DAC. That’s how I would do it on my system. 

 

My experience with the Distort software, via Archimago's THD test, is that it does things that mimic the type of problems I hear in systems, that remind me of issues that require "sorting out" - the sense of unease when listening is certainly made worse by the deliberately introduced artifacts.

 

To repeat what I have said many times - I have never directly modded or tweaked something with the intention of "reducing jitter" - if the numbers changed by what I did, then it was a side effect, and wasn't what I was aiming to do. Whether the measured jitter being reduced, say, was indeed what caused the SQ to improve I can't say - all that mattered was that the robustness of the system had been improved, with commensurate improvement in what I was hearing.

[barrows]

next answer to the question of the thread:

 

When the component is the Mola Mola Tambaqui DAC.  See measurements here:

 

https://www.audiosciencereview.com/forum/index.php?threads/mola-mola-tambaqui-dac-and-streamer-review.10770/

[pkane2001]

3 hours ago, barrows said:

next answer to the question of the thread:

 

When the component is the Mola Mola Tambaqui DAC.  See measurements here:

 

https://www.audiosciencereview.com/forum/index.php?threads/mola-mola-tambaqui-dac-and-streamer-review.10770/

 

That -110dBFS peak at 50kHz needs to be worked on! I'm not going to pay that much money and still have distortion at that level!

Well, I probably wouldn't pay that much money for a DAC, period, but that really is one well-measuring DAC.

 

[pkane2001]

6 hours ago, jabbr said:

I got that far, then tried to load a music file.

 

I can load and process a file (click Save...) I can't play a file yet, because the audio library I'm using requires .NET 4.0, which isn't part of Wine 5 install. What happens when you try to generate a file? Just pick a WAV file you want to process, then pick the desired distortion, then click Save... button to generate the distorted file. I used Audacity to play it on Ubuntu.

 

I'll see if I can change the target .NET framework for the library, or install .NET 4.0 on Wine -- using it for the first time, so not remotely an expert 😄

 

[barrows]

48 minutes ago, pkane2001 said:

 

That -110dBFS peak at 50kHz needs to be worked on! I'm not going to pay that much money and still have distortion at that level!

Well, I probably wouldn't pay that much money for a DAC, period, but that really is one well-measuring DAC.

 

Come on man, you are smarter than that right?  That peak is due to the slightly relaxed roll off of the digital filter, and would not ever be present with a music signal. But if you are still concerned you could always over sample to a 2x rate in software with steep filter to eliminate it.

[pkane2001]

2 minutes ago, barrows said:

Come on man, you are smarter than that right?  That peak is due to the slightly relaxed roll off of the digital filter, and would not ever be present with a music signal. But if you are still concerned you could always over sample to a 2x rate in software with steep filter to eliminate it.

 

That was a joke, sorry. I thought the smirking smiley face would give it away, but apparently I need to find better emojis.

 

[barrows]

2 hours ago, pkane2001 said:

 

That was a joke, sorry. I thought the smirking smiley face would give it away, but apparently I need to find better emojis.

 

Sorry, my bad!  Reacted to quick...

[Audiophile Neuroscience]

I think the OP poses one of the great perennial questions in audio/music perception……when do measurements correlate with subjective impressions?

 

I think if we knew the answer to this it would solve a lot of disputes. More importantly, I'm just plain curious to find out the answers. I could be wrong but what I have so far taken from this thread and several others like it, is that there is no firm objective scientific rules that apply.

 

At the heart of the question of any correlation is whether there is an association or link between the level or magnitude of measurement/variable A and the magnitude of measurement/variable B moving in the same (or reverse) direction. If you chuck in concordance which is basically the extent of how the levels of the two measurements match i.e.  their agreement or sameness between levels and/or their reproducibility, you then have a  meaningful association. This of course does not mean a causal relationship necessarily.

 

So, where I am leading with this is, is there any measurement that correlates and is concordant with a particular sensory audio perception? We know for example (or at least I have read and have had some experience) that speaker directivity and room reflectiveness will affect imaging in certain predictable ways for most observers. There are measurements for these things but are there predictive  values that will hold true for most people?

 

There has been some talk of degrees of various distortion measurements, jitter, which may be perceptible and at different sensitivity levels for different people and different levels of training etc. Again, are there predictive values that reproducibly hold true for most people about what the subjective correlate sounds like? Is it just that it somehow  makes things sound irritating in ways that are hard to define? Somehow a little less natural, organic or less "real".

 

 On the other hand some distortions appear to make things sound subjectively better, at least to some. In the case of tube distortions at least there  seems to be some reproducible and shared subjective correlate, namely "warmth" or similar description.

 

To paraphrase the OP question, what do any of these measurements tell us about what the music sounds like? Do any of them have predictive value for perception and in what way?

[fas42]

29 minutes ago, Audiophile Neuroscience said:

There has been some talk of degrees of various distortion measurements, jitter, which may be perceptible and at different sensitivity levels for different people and different levels of training etc. Again, are there predictive values that reproducibly hold true for most people about what the subjective correlate sounds like? Is it just that it somehow  makes things sound irritating in ways that are hard to define? Somehow a little less natural, organic or less "real".

 

 

It may not be "jitter" per se, but my modus operandi for decades has been to just listen to a rig, and wait for "things (that) sound irritating in ways that are hard to define", which immediately translates to the SQ being "less natural, organic or less "real"".

 

My point would be, which is a better use of my time? To go to great effort to extract some number, by some means, which is an exact correlate of what I'm registering - or, merely correct the causal factor ... so far, the latter has won out ...

[barrows]

With digital playback, while the artifacts are usually very low in level, they (artifacts) are often non harmonically related to the music, and it seems to me that these types of non-harmonic (or is the proper word enharmonic?) artifacts are much more disturbing to the ear/brain perceptual system, perhaps because these types of artifacts do not generally exist in nature.  This is purely speculative on my part though, but if accurate it would seem to explain why analog playback often sounds more "natural" than digital playback, even though we know scientifically, that digital playback has far less distortions/noise than analog.

Furthermore, I think some of the best digital designers are aware of what, at least some of, these artifacts are, and I think these designers work to eliminate these non-harmonic artifacts.

Some digital products, which measure poorly on standard measurements, but audiophiles seem to like how they sound, may be "sounding good" by doing either: masking the artifacts via the allowance of "pleasant" distortions, such as 2nd harmonic as one might get with a tube based output stage, or perhaps they are actually reducing the artifacts, and in the process of doing so, increasing other types of distortions/noise, but because the really annoying artifacts are gone, the sound is still pleasing.

I suspect until designers and those who study psychoacoustics really get together and work on this (perhaps using brain imaging techniques) it is often just going to be speculative as to what artifacts are problematic, and what distortions are acceptable.  And then engineers have to figure out how to measure them.

[fas42]

4 minutes ago, barrows said:

With digital playback, while the artifacts are usually very low in level, they (artifacts) are often non harmonically related to the music, and it seems to me that these types of non-harmonic (or is the proper word enharmonic?) artifacts are much more disturbing to the ear/brain perceptual system, perhaps because these types of artifacts do not generally exist in nature.  This is purely speculative on my part though, but if accurate it would seem to explain why analog playback often sounds more "natural" than digital playback, even though we know scientifically, that digital playback has far less distortions/noise than analog.

Furthermore, I think some of the best digital designers are aware of what, at least some of, these artifacts are, and I think these designers work to eliminate these non-harmonic artifacts.

Some digital products, which measure poorly on standard measurements, but audiophiles seem to like how they sound, may be "sounding good" by doing either: masking the artifacts via the allowance of "pleasant" distortions, such as 2nd harmonic as one might get with a tube based output stage, or perhaps they are actually reducing the artifacts, and in the process of doing so, increasing other types of distortions/noise, but because the really annoying artifacts are gone, the sound is still pleasing.

I suspect until designers and those who study psychoacoustics really get together and work on this (perhaps using brain imaging techniques) it is often just going to be speculative as to what artifacts are problematic, and what distortions are acceptable.  And then engineers have to figure out how to measure them.

 

+100 ... !!

[Audiophile Neuroscience]

2 hours ago, fas42 said:

 

It may not be "jitter" per se, but my modus operandi for decades has been to just listen to a rig, and wait for "things (that) sound irritating in ways that are hard to define", which immediately translates to the SQ being "less natural, organic or less "real"".

 

My point would be, which is a better use of my time? To go to great effort to extract some number, by some means, which is an exact correlate of what I'm registering - or, merely correct the causal factor ... so far, the latter has won out ...

 

Hi Frank,

your "MO" is well known and i dare say not dissimilar to what most other audiophiles do.

 

I agree that if you can fix the cause of the problem then probably no need to measure it. I already do this by placing for example tube traps in corners of the room. I could measure the resonances but I already have a fair idea of what they will be and where they will be and a number will not likely change where I locate a tube trap.

 

None of this however  addresses the OP question.When (and what) measurements correlate with subjective listening impression? I would love to know the answer.

 

My crude impression is that measurements tell us that the item is operating to spec, whatever those parameters are.They tell us how they will interact and may be suitable to perform with other devices, like is there enough current to drive difficult speakers or will there be impedance mismatching etc etc. No doubt these things have impact on SQ. But still, what are the measurements that correlate with perceived sound characteristics?

[fas42]

43 minutes ago, Audiophile Neuroscience said:

 

Hi Frank,

your "MO" is well known and i dare say not dissimilar to what most other audiophiles do.

 

Cheers, welcome back!

 

Indeed I do no more than what others do - the difference is what that I have a very specific agenda when doing such; because I know from repeated successes what occurs when enough is done, 😉.

 

43 minutes ago, Audiophile Neuroscience said:

My crude impression is that measurements tell us that the item is operating to spec, whatever those parameters are.They tell us how they will interact and may be suitable to perform with other devices, like is there enough current to drive difficult speakers or will there be impedance mismatching etc etc. No doubt these things have impact on SQ. But still, what are the measurements that correlate with perceived sound characteristics?

 

I have no doubt it's the presence or absence of certain types of distortion - but they are of a style where it's hard "to pin the tail on the donkey". Not purely random noise, nor simple THD, IMD, etc - they correlate to some degree with the input signal, but are random enough in nature to make it hard to point to a picture on a screen, and utter, "There's the culprit!"

 

Unfortunately, the particularly nasty varmits are those which embed that distinctively grey, flat quality in replay from digital source - I don't know anyone who can point with conviction at numbers specifying this.

[Archimago]

On 2/21/2020 at 7:12 PM, pkane2001 said:

 

That -110dBFS peak at 50kHz needs to be worked on! I'm not going to pay that much money and still have distortion at that level!

Well, I probably wouldn't pay that much money for a DAC, period, but that really is one well-measuring DAC.

 

 

Yeah, for that price, I hope there's a way to play with the filter settings a little!

[Archimago]

5 hours ago, Audiophile Neuroscience said:

I think the OP poses one of the great perennial questions in audio/music perception……when do measurements correlate with subjective impressions?

 

I think if we knew the answer to this it would solve a lot of disputes. More importantly, I'm just plain curious to find out the answers. I could be wrong but what I have so far taken from this thread and several others like it, is that there is no firm objective scientific rules that apply.

 

Great discussion AN. I think it's worth remembering that to some extent, it's going to be very difficult to find firm rules beyond population "norms" due to the variability of auditory acuity. To be even more accurate, we would need norms for gender and the effect of age.

 

All of this would need to be done on large populations as a result... Very much like neuropsychological test result interpretation. By necessity, this will also require controlled blind testing.

 

Quote

At the heart of the question of any correlation is whether there is an association or link between the level or magnitude of measurement/variable A and the magnitude of measurement/variable B moving in the same (or reverse) direction. If you chuck in concordance which is basically the extent of how the levels of the two measurements match i.e.  their agreement or sameness between levels and/or their reproducibility, you then have a  meaningful association. This of course does not mean a causal relationship necessarily.

 

Yes, very reasonable. Again, who's going to do this? We do know from the literature over the years that there have been sophisticated systems developed to try to determine correlation between distortion and perception. For example, here's a presentation using a neural network model to calculate a "perceptual distortion index" or "PTHD" to model distortion audibility.

 

Toole and Olive's work on the speaker "spinorama" is another example of correlating blind listening with measurements. You should enjoy Toole's Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Rooms.

 

Quote

So, where I am leading with this is, is there any measurement that correlates and is concordant with a particular sensory audio perception? We know for example (or at least I have read and have had some experience) that speaker directivity and room reflectiveness will affect imaging in certain predictable ways for most observers. There are measurements for these things but are there predictive  values that will hold true for most people?

 

There has been some talk of degrees of various distortion measurements, jitter, which may be perceptible and at different sensitivity levels for different people and different levels of training etc. Again, are there predictive values that reproducibly hold true for most people about what the subjective correlate sounds like? Is it just that it somehow  makes things sound irritating in ways that are hard to define? Somehow a little less natural, organic or less "real".

 

 On the other hand some distortions appear to make things sound subjectively better, at least to some. In the case of tube distortions at least there  seems to be some reproducible and shared subjective correlate, namely "warmth" or similar description.

 

To paraphrase the OP question, what do any of these measurements tell us about what the music sounds like? Do any of them have predictive value for perception and in what way?

 

I'm totally with you in this question and the desire for understanding these correlations between objective measurements and subjective perception is very important. The only way this can be answered is with lots of blind testing.

 

Ultimately, I think this needs to happen in the academic space, not in the hobby arena especially with strongly "subjectivist" audiophiles. When so many are against even the importance of doing blind tests, and magazines produce articles about how they're "flawed", how can we "culturally" even begin any meaningful research?

 

[Audiophile Neuroscience]

3 hours ago, Archimago said:

 

Great discussion AN. I think it's worth remembering that to some extent, it's going to be very difficult to find firm rules beyond population "norms" due to the variability of auditory acuity. To be even more accurate, we would need norms for gender and the effect of age.

 

All of this would need to be done on large populations as a result... Very much like neuropsychological test result interpretation. By necessity, this will also require controlled blind testing.

 

 

Yes, very reasonable. Again, who's going to do this? We do know from the literature over the years that there have been sophisticated systems developed to try to determine correlation between distortion and perception. For example, here's a presentation using a neural network model to calculate a "perceptual distortion index" or "PTHD" to model distortion audibility.

 

Toole and Olive's work on the speaker "spinorama" is another example of correlating blind listening with measurements. You should enjoy Toole's Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Rooms.

 

 

I'm totally with you in this question and the desire for understanding these correlations between objective measurements and subjective perception is very important. The only way this can be answered is with lots of blind testing.

 

Ultimately, I think this needs to happen in the academic space, not in the hobby arena especially with strongly "subjectivist" audiophiles. When so many are against even the importance of doing blind tests, and magazines produce articles about how they're "flawed", how can we "culturally" even begin any meaningful research?

 

 

Thanks Archimago

 

I have the Floyd Toole publication. Thanks for the semantic scholar PDF, I will take a look.

 

I am likely going to disappoint you now by declaring I'm not a fan of the reliability of blind testing in the audio setting. I would never argue against the necessity of eliminating the obvious bias, just the problem of introducing interdependent variables in the process, thus making the test unreliable. I believe there is a propensity for false negatives and this has been borne out to some extent in the literature.

 

However, let's certainly not go into the pros and cons of blind testing.  At the end of the day I am basically reinforcing what you said that this kind of research, including properly conducted blind testing, is best done in the academic domain.  I think it is very difficult to do properly conducted blind testing. The problem here is that it is unlikely that academic institutions will be interested or have funding to do such research.

 

All that said it still surprises me that we haven't been able to come with more measurements that correlate to subjective listening impressions. Apart from some speaker examples I can only think of one other example off the top of my head. That would be the various measures of dynamic range and correlating that to descriptions of "compressed sound".

 

Cheers

David

[Iving]

imo the last 2 posts in this thread are in a different league compared with everything we have seen here lately in the subj. obj. debacle. Way, way ahead of all the trolling nonsense. Potentially a platform for a debate worth reading. tbh I think the point about the academic space is uncomfortably true for this forum - although as I see it - speaking only generally - it's a lack of intellectual rigour and humility on the part of objectivists posting here that determines that need rather than subjectivists' "cultural" resistance to DBX. 

[sandyk]

21 minutes ago, Iving said:

imho the last 2 posts in this thread are in a different league compared with everything we have seen here lately in the subj. obj. debacle. Way, way ahead of all the trolling nonsense.

 Iving

This is what happens when a well respected member decides to resume posting again when the reasons for him ceasing posting here are removed. He simply found the atmosphere here had become on the toxic side, and no longer wished to participate here.

If he had been able to at the time, David would have cancelled his membership.

 Yes, I do know David personally , and it was through this forum and our common interest in high quality audio that we met originally.

 

Alex

[Iving]

12 minutes ago, sandyk said:

If he had been able to at the time, David would have cancelled his membership.

 

Not at all surprised. 

Where there's life there's hope!

[Iving]

54 minutes ago, Iving said:

I think the point about the academic space is uncomfortably true for this forum

 

caveat: in "this forum" I allude to what I seen of late here. If I were speaking comparatively I would rank AS #1 for potential and ASR lowest. But I am no Forum whore/butterfly and say this from limited experience of other places.

[tapatrick]

1 hour ago, Iving said:

imo the last 2 posts in this thread are in a different league compared with everything we have seen here lately in the subj. obj. debacle.

Really enjoying the overall change in tone and level of discussion in the past week! Indeed there is hope - dare I say even enjoyment. We all knew there was more to this interface between subjective/objective than semantic butchery.

[semente]

12 hours ago, Audiophile Neuroscience said:

I think the OP poses one of the great perennial questions in audio/music perception……when do measurements correlate with subjective impressions?

 

I think if we knew the answer to this it would solve a lot of disputes. More importantly, I'm just plain curious to find out the answers. I could be wrong but what I have so far taken from this thread and several others like it, is that there is no firm objective scientific rules that apply.

 

I think the problem arises not because measurements do not correlate with listening but due to the fact that some people find it difficult to swallow that they like equipment which measures poorly.

 

There, I've solved all disputes.

 

Measurements quantify particular parameters of reproduction. They are deemed good or bad in relation to the accuracy with which the signal is being "handled". Some measurements are bad from a fidelity perspective but the offending distortions sound good (euphonic) to some people.

Whether good measurements sound good or not depends on the listener's preference regarding presentation: it's a matter of taste.