BLIND TEST INVITATION: Can you hear high THD?

[Archimago]

Alright guys and gals, a new test for you to try out for 2020!

You know that measurements of audio gear typically will include harmonic distortion tests. We see numbers like 1% THD, 0.1% THD+N, etc... all the time. Typically also included with graphs in DAC, amp, preamp, phono cartridge results.

But what does this mean when it comes to listening and with real music?

Here's an opportunity to try for yourself and let me know what you hear - preferably using the best sound system you have! As usual, the results will be collected, analyzed, and published once the data gather is done by the end of April 2020. Plenty of time to listen and let me know what you hear.

Remember that I have used rather high amounts of distortion in some of these samples... Theoretically, this should be nowhere as difficult as "can you hear 16-bit vs. 24-bit?" or "24/96 vs. 16/44.1" tests.

Enjoy 

Test: Is high Harmonic Distortion in music audible?

[Ralf11]

this should be good...

[AudioDoctor]

In before the bickering...

[fas42]

Just downloaded the file ... and sampled the vocal with light rock instrumentation track variations - trivially easy to pick the best version, and how bad it gets with the different added distortion, on casual laptop speakers playback - should be straightforward for most listeners ... 🙂.

[Confused]

I'm in too, no issue downloading the files.

 

I have to say that having made the effort to try previous blind test for music players, it was absolutely fascinating to later read through the results.  Quite enlightening.  This test should be well worth the effort I think.

[Confused]

On 1/25/2020 at 10:51 PM, fas42 said:

Just downloaded the file ... and sampled the vocal with light rock instrumentation track variations - trivially easy to pick the best version, and how bad it gets with the different added distortion, on casual laptop speakers playback - should be straightforward for most listeners ... 🙂.

Maybe not, with a well sorted rig that does not add any distortion in the chain, adding harmonic distortion might be euphonic.  It is a bit like the harmonic distortion inherent with vinyl playback, it adds a touch of warmth, some might say realism.

 

Indeed, it will be fascinating to see how results are distributed between various systems and demographics.

 

I think this test might be a little more interesting that simply picking which version is "best"?

[Archimago]

On 1/25/2020 at 2:51 PM, fas42 said:

Just downloaded the file ... and sampled the vocal with light rock instrumentation track variations - trivially easy to pick the best version, and how bad it gets with the different added distortion, on casual laptop speakers playback - should be straightforward for most listeners ... 🙂.

 

Awesome Frank. I haven't looked at most of the submissions in part to keep myself at arms length from the results. I hope you submitted your results already!

 

[Archimago]

6 hours ago, Confused said:

Maybe not, with a well sorted rig that does not add any distortion in the chain, adding harmonic distortion might be euphonic.  It is a bit like the harmonic distortion inherent with vinyl playback, it adds a touch of warmth, some might say realism.

 

Indeed, it will be fascinating to see how results are distributed between various systems and demographics.

 

I think this test might be a little more interesting that simply picking which version is "best"?

 

Exactly. And this why you see that there are 2 different but potentially related questions...

 

1. Tell me which sounds best --> worst.

2. Is the above different from what you believe to be least --> most distortion added?

 

It will be interesting to see if there are some listeners differentiate the two. Presumably out of previous experience, some listeners might have insight that what sounds "best" to them is when they hear a certain amount of distortion... Will see...

 

[Speedskater]

How does one go about adding harmonic distortion to a track?

Is it low pass filtered?

[pkane2001]

2 minutes ago, Speedskater said:

How does one go about adding harmonic distortion to a track?

Is it low pass filtered?

 

https://distortaudio.org/

[sandyk]

1 hour ago, Archimago said:

some listeners might have insight that what sounds "best" to them is when they hear a certain amount of distortion... Will see...

 Archimago

 As an aside, John Dyson recently had a Eustachian tube blockage requiring 2 courses of different Antibiotics. In that period John reported that he had a far greater ability to detect certain kinds of distortion that he hadn't previously noticed.

 

Alex

[Archimago]

2 minutes ago, sandyk said:

 Archimago

 As an aside, John Dyson recently had a Eustachian tube blockage requiring 2 courses of different Antibiotics. In that period John reported that he had a far greater ability to detect certain kinds of distortion that he hadn't previously noticed.

 

Alex

 

Interesting. Yeah, possible given change in perceived frequency response and changes in tympanic resonance.

[fas42]

2 hours ago, Archimago said:

 

Awesome Frank. I haven't looked at most of the submissions in part to keep myself at arms length from the results. I hope you submitted your results already!

 

 

No ... 🙂. The casual listen, to me, showed one version to be very clear - and the others were murky, to various degrees. The trick will be assigning a rating of murkiness, to get those in the "correct order" - so, I will probably go through all the different clips, because I suspect the style of the music will determine how much the varying degrees of distortion degrade, subjectively.

[Archimago]

3 hours ago, fas42 said:

 

No ... 🙂. The casual listen, to me, showed one version to be very clear - and the others were murky, to various degrees. The trick will be assigning a rating of murkiness, to get those in the "correct order" - so, I will probably go through all the different clips, because I suspect the style of the music will determine how much the varying degrees of distortion degrade, subjectively.

 

Hey, even if the general order looks right, that would be nice! In fact, just picking out the "very clear" one and telling me that this is DEFINITELY the least distorted would be useful to know.

[fas42]

4 hours ago, Archimago said:

 

Hey, even if the general order looks right, that would be nice! In fact, just picking out the "very clear" one and telling me that this is DEFINITELY the least distorted would be useful to know.

 

Will do a "sit down and do it seriously" effort in the next day or so ... 🙂.

[fas42]

The weather has been filthy the last few days - as in super hot. Wasn't in the mood to tackle more careful listening, but today outside has finally settled down ... but I did do an extra round which, to me, showed that on least the music sample I was listening to that it wasn't clear which was 'worst', between two versions. So, aim to look at this again today, examining all four music selections.

[fas42]

And, I think I'm happy with the ordering now - the middle two in quality were more subtle, to me, but the pattern of what's happening to the sound clarified when I listened to the other selections; my original pick for the ordering was confirmed.

 

The almost purely vocal track seemed to be the best 'tool' - the difference in those two middle versions was that the "better" had a more natural feel about the sound, I could relate to there being a person there more easily; with the lesser one I felt more of a strain to "make it work", the instinct was to move my head closer to the speakers, so that "I could hear it better".

 

Will give it a day or so, and revisit, to see if my current ranking still makes sense - if yes, will then post my choices on the test website.

[fas42]

Haven't forgotten about this ... 😁. Too much else going on - finally relooked at the samples ... and a problem arises! ... Life is never simple ... 😜.

 

I'm OK with the best and worst versions, it's the middle two that are hard to rank ... why? Well, it depends on the player I use ...🤣. Yes ...... with foobar I get one clear 'better', with Media Monkey it switches over ... bugger !!!

 

IOW, the complication of "everything matters" climbs on board - the idiosyncrasies of the laptop, player and samples combine, meaning how it comes across alters significantly, subjectively - the "preference" is a result of how I react to particular subtle alterations.

 

What to do? Sit on this a bit more, and then have another go when I'm refreshed - trying to take this into account.

[Archimago]

On 2/16/2020 at 6:19 PM, fas42 said:

Haven't forgotten about this ... 😁. Too much else going on - finally relooked at the samples ... and a problem arises! ... Life is never simple ... 😜.

 

I'm OK with the best and worst versions, it's the middle two that are hard to rank ... why? Well, it depends on the player I use ...🤣. Yes ...... with foobar I get one clear 'better', with Media Monkey it switches over ... bugger !!!

 

IOW, the complication of "everything matters" climbs on board - the idiosyncrasies of the laptop, player and samples combine, meaning how it comes across alters significantly, subjectively - the "preference" is a result of how I react to particular subtle alterations.

 

What to do? Sit on this a bit more, and then have another go when I'm refreshed - trying to take this into account.

 

Thanks for the note Frank... Hey, keep trying with the middle two. As with many listening tests (and most things in life), the shades of grey will be harder to discern but potentially worthwhile especially these days where distortion levels for devices are quite well controlled. It would be awesome to see if among audiophile testers, there is at least a "trend" to differentiate distortion levels "better than average" for modern devices like amplifiers.

 

[Confused]

On 2/17/2020 at 2:19 AM, fas42 said:

Haven't forgotten about this ... 😁. Too much else going on - finally relooked at the samples ... and a problem arises! ... Life is never simple ... 😜.

 

I'm OK with the best and worst versions, it's the middle two that are hard to rank ... why? Well, it depends on the player I use ...🤣. Yes ...... with foobar I get one clear 'better', with Media Monkey it switches over ... bugger !!!

 

IOW, the complication of "everything matters" climbs on board - the idiosyncrasies of the laptop, player and samples combine, meaning how it comes across alters significantly, subjectively - the "preference" is a result of how I react to particular subtle alterations.

 

What to do? Sit on this a bit more, and then have another go when I'm refreshed - trying to take this into account.

Here is the solution.  Pick some favorite tracks from your music collection, try on Foobar and then Media Monkey, see which of the two you prefer.  With this settled, try the blind test files again with your preferred music software.  Simple!

 

I always use Roon / HQPlayer for my "serious" listening, so that is what I used for the blind test.  No need to over complicate things.

 

Oh, and please can you stop posting too many specifics about your listening experiences, you are starting to fuel my expectation bias!🙂

[gmgraves]

On 1/25/2020 at 12:06 PM, Archimago said:

Alright guys and gals, a new test for you to try out for 2020!

You know that measurements of audio gear typically will include harmonic distortion tests. We see numbers like 1% THD, 0.1% THD+N, etc... all the time. Typically also included with graphs in DAC, amp, preamp, phono cartridge results.

But what does this mean when it comes to listening and with real music?

Here's an opportunity to try for yourself and let me know what you hear - preferably using the best sound system you have! As usual, the results will be collected, analyzed, and published once the data gather is done by the end of April 2020. Plenty of time to listen and let me know what you hear.

Remember that I have used rather high amounts of distortion in some of these samples... Theoretically, this should be nowhere as difficult as "can you hear 16-bit vs. 24-bit?" or "24/96 vs. 16/44.1" tests.

Enjoy 

Test: Is high Harmonic Distortion in music audible?

I don’t need to do the experiment. I have found that the human ability to hear harmonic distortion depends on where that distortion originates. Some tube amps have as much as 4% THD (like some of the French Jadis amps) and people don’t notice it (in fact, Jadis amps are highly regarded in the high-end community). But one percent in a speaker is very audible, as is one percent in an analog tape deck or an FM tuner.

 

[Archimago]

Hi guys & gals,

Just a little bump for this topic... Closing off the test at end of month.

 

2 weeks left to send in your results!

 

[danadam]

Results:

• part I - Procedures & Settings
• part II - Respondent Results

[Confused]

There is a lot to digest here.  Certainly there is some evidence that the levels of distortion used in the test makes a difference, but it is a very long way from a statically clear correlation.

Some interesting trends though, such as those that used headphones performed a lot better than the speaker only group.  The "speaker only" group failed to find even the 3% distortion distortion sample "worst", and looking at the of the order of the samples best to worst, it looks almost random to me!  (Note, I am not a statistician)

Another interesting result is that the only group that correctly identified the lowest distortion track as, was the group that believed subjective preference did not correlate to THD.  Remarkable, although this was a very small group, and it begs the question what their order of subjective preference was.  Did they really pick out sample D as having the most THD (which they did) but then stated a preference for this?  (this bit not clear) 

In addition, there are many cases where even the 3% sample was not found to be "worst".  I guess this hints at why some valve amps can sound great when they have seemingly appalling on paper measurements.

So interesting results that indicate some correlation with THD.  I suspect this will be one article that I will probably re-visit and re-read a couple of times, to try to fully adsorb and think through what is there, but I am starting to think that THD relates to sound quality rather less than I previously thought.  Which then begs the question, what does?  Maybe we need more tests like this to establish what does?

[firedog]

My take is that only some of us can tell the difference-even though we all tend to think we can. And that some of us like the sound of distortion.

It probably also means that HW that measures very well is a good thing, but also that it's probably not necessary to chase that last db of performance of electronics. "Good enough" does kick in a some point.