When do measurements correlate with subjective impressions

[Jud]

1 minute ago, pkane2001 said:

 

Cable I was testing was Lush^2. It made a small but measurable difference in noise level compared to a no-name USB cable. It actually slightly increased the level of noise at the output of the Emotiva DAC. I didn't find Lush^2 to make any difference with other DACs.

 

I assume the increase in noise had to do with shielding/grounding configuration of Lush^2 picking up EMI or introducing a ground loop. (I was using the stock Lush^2 configuration it was shipped with).

 

Of course this raises what's somewhat the flip side of good measurements correlating with subjective good impressions - what types and levels of distortion and noise are euphonic for at least some people? (Beyond the usual mention of tube electronics euphony, there are things like the Aphex Aural Exciter, literally a piece of electronics to produce noise and hash, used in the production of Born to Run, for example.) 

[pkane2001]

7 minutes ago, Miska said:

 

With TIM, the problem was that the measurements being used were operating on static sine waveforms like in THD and IMD. While problem was with transient signals. Thus TIM is measured with mixture of square and sine waves (15 kHz sine + 3.18 kHz square wave at 4 : 1 level ratio, -3 dB at 100 kHz).

 

In these cases it was due to the amplifier going to internal saturation, due to internal bandwidth issues, during transients only.

 

And this happened already with vinyl and open reel tape recordings, for example in crescendos. Problem was found because some engineers believed their ears.

 

 

Do you have any evidence that TIM is a concern in any modern DAC implementations? I thought about adding TIM measurement to DeltaWave (and actually had a test version of this working) but found no useful information in those results.

[Miska]

9 minutes ago, CG said:

Because of the averaging function, you could see distortion way into the noise.  That's what averaging does - it assumes that the fundamental tone and the distortion tones are constant in amplitude, which probably is an ok assumption (maybe) and then assumes that the rest are just random chaotic events that don't repeat either in amplitude or frequency over all those sweeps.  So, much of the noise voltage just gets averaged away.

 

This is reason why some of the funky ESS Sabre behavior is so hard to catch. Because it is relative low level, but time varying. Some of it is pretty fast, while some cycles seem to take about one minute. I almost accidentally noticed it because I happened to have suitable combination of analyzer settings.

 

[Bill Brown]

1 minute ago, pkane2001 said:

 

I'll point you to my own software, DISTORT, if you want to explore audibility of certain kinds of distortions. For example, you can try out only even harmonics or only odd. Or create a specific custom harmonic mix. Or, add a compressor-type effect. Or change noise floor, etc. I'm adding support for different filter types and phase controls to it, but that's still work in progress

 

 

Thank you for the kind offer, though I will probably have to pass.  This doesn't mean that your software wouldn't be VERY educational, and I fully believe in the underlying science, but I don't want to confuse myself- perhaps the main reason .  Also, I would probably have to wait for my golden-eared son to return for a visit(!)- he is the one who passed the filter blind test and described in similar terms what I perceived.

 

My current approach is to shoot for what I believe is transparent in my daily system and enjoy the music while thinking about other things that I am not as sure about- I have some tube amp ideas in my head and a design for a pistonic, uniform-directivity loudspeaker (I love diyaudio.com).

 

Thanks again,

 

Bill

[pkane2001]

1 minute ago, Jud said:

 

Of course this raises what's somewhat the flip side of good measurements correlating with subjective good impressions - what types and levels of distortion and noise are euphonic for at least some people? (Beyond the usual mention of tube electronics euphony, there are things like the Aphex Aural Exciter, literally a piece of electronics to produce noise and hash, used in the production of Born to Run, for example.) 

 

That's precisely why I wrote DISTORT. Measurements are great, but they are just an engineering result, showing how well a device is working. They don't tell the whole story of what's audible, or what distortions I may find pleasant.

 

DISTORT lets me apply controlled levels of various distortions (in any combination) to any music track so I can determine if it rises to audible level while listening on my "perfect" system. Anyone can do the same with this app, with their music, in their system.

 

@Archimago's latest internet blind test is an attempt to do this on a larger scale, gathering input on audibility of various levels of THD. 

[Miska]

2 minutes ago, pkane2001 said:

Do you have any evidence that TIM is a concern in any modern DAC implementations? I thought about adding TIM measurement to DeltaWave (and actually had a test version of this working) but found no useful information in those results.

 

Time will show when I have collected enough data around it. But there are certainly differences between DACs in this respect and the components are in the audio band.

 

Highest figures I've measured have components around -65 dB, and it is not unusual to have them around -100 dB. So within range I know people can hear.

 

[pkane2001]

1 minute ago, Miska said:

 

Time will show when I have collected enough data around it. But there are certainly differences between DACs in this respect and the components are in the audio band.

 

Highest figures I've measured have components around -65 dB, and it is not unusual to have them around -100 dB. So within range I know people can hear.

 

 

Would be curious to know which DACs show such a high level of TIM. Might be interesting to see how (or if) it affects other measurements and metrics.

[Bill Brown]

TIM distortion is now largely looked upon as discredited by many members on diyaudio (including the eminent Scott Wurcer), but we have to remember that the early SS amps were measurably very good with the measurements of the time but subjectively not pleasing- I hope that isn't too subjective for here....:)

 

Also, that many people pronounced RBCD perfect from the beginning (by measurement), but faults were found (jitter, etc.) that were not known, or at least not adequately addressed, by the engineers of the time.

 

What else is out there?  Perhaps not much.  But.....

[Miska]

5 minutes ago, pkane2001 said:

Would be curious to know which DACs show such a high level of TIM. Might be interesting to see how (or if) it affects other measurements and metrics.

 

You can also get idea with two IMD sweeps. If you first start with for example 19+20 kHz and then sweep the second tone up to 100 kHz for example. And then do another sweep where you sweep both together up 100 kHz.

 

With TIM test some extra hash appears on some DACs. And compared to normal 19+20 kHz IMD test, the ~1 kHz figure is usually quite similar, while highest peaks tend to be between 10 and 20 kHz. But overall, many of the results look generally quite different from each other in terms of distortion patterns when you look at everything.

 

But I mean this is just one of the many things. Combine everything together and the devices are very different...

 

[Arpiben]

Otala/Leinonen/Curl TIM papers are from the seventies. Anything else significant and new to measure at DAC's output? Any new Input stimulation signal rather than sinusoïdal tones?

[Bill Brown]

And I sure hope Miska doesn't find something anomalous with my RME ADI-2!

 

I would have to fall back on "don't worry, Bill, you enjoyed it before this so it shouldn't change anything."

 

I am using the same technique to resist the upgraded version recently released.

[CG]

45 minutes ago, pkane2001 said:

 

Turn off averaging, and use peak-hold. Problem solved? 

Maybe.  I think it would help in this one example, but that's just my opinion.  Doing this would not only obscure distortion measurements in many cases, but it would also freak a lot of people out.

 

Have you ever seen the results of that published anywhere?

 

And, just imagine the ensuing arguments!

[jabbr]

1 hour ago, Jud said:

 

Measured differences? In what measures? (Curious as to what if any measurable differences a cable might make as part of a system.) Are there any differences between cables that meet spec?

Sure there are differences. Different shielding is one example. The spec is a range. The rise times might be affected slightly. The cable couples the sending and receiving components - the D+\D- pair reduces differential mode noise but not common mode noise. Different shields can couple the components together and alter the common mode noise transmission. 
 

TL:DR - measure common mode noise transmission/EMI

[pkane2001]

3 minutes ago, CG said:

Maybe.  I think it would help in this one example, but that's just my opinion.  Doing this would not only obscure distortion measurements in many cases, but it would also freak a lot of people out.

 

Have you ever seen the results of that published anywhere?

 

And, just imagine the ensuing arguments!

I’ve certainly used that mode with oscilloscope, and it does look a lot more ugly than the average, but it does represent reality. Maybe it should be used in more cases and with some published results.

[CG]

1 hour ago, Jud said:

 

USB cables have standard specs they are able to meet, I believe.

These? 

 

https://www.usb.org/sites/default/files/CabConn20.pdf

 

The USB 2.0 cable specs don't have a lot to say about electrical performance, amazingly enough.  The USB 3.1 electrical specs are more comprehensive. 

 

Now, whether the minimum performance specs that are required to allow for the USB logo to be applied are good enough for audio performance, or provide consistent results, I don't know.

 

In addition, the termination transceivers and connectors have similarly loose specification for even high speed mode, which most of us use with digital audio systems.  The combination of tolerance spreads might make for some interesting combinations.

 

There's no specification for such details as differential-to-common mode signal conversion (and vice versa) or noise rejection in USB 2.0 as far as I can tell.  I'd love to be wrong and have somebody point out the error of my ways.

 

USB transmission must not be perfect.  Otherwise, why would the designers add all the extra sophistication to correct for errors if there are none?  Not trying to argue here...

[CG]

22 minutes ago, pkane2001 said:

I’ve certainly used that mode with oscilloscope, and it does look a lot more ugly than the average, but it does represent reality. Maybe it should be used in more cases and with some published results.

I'd think so.  The mass hysteria might be fun to watch...

 

But, I didn't mean to scream that this one aspect of testing is bad.  It was just intended to be a simple example of how what might be significant test information isn't available for 99.999999% of the audio enthusiast public.

[Jud]

7 minutes ago, CG said:

USB transmission must not be perfect.  Otherwise, why would the designers add all the extra sophistication to correct for errors if there are none?

 

Not an expert, but my impression is that errors would not be small, i.e., not just audio a little less nice, but evident like dropouts, etc.

[Jud]

32 minutes ago, jabbr said:

TL:DR - measure common mode noise transmission/EMI

 

Inexpensive methods of doing so for the home audio enthusiast?

[Jud]

1 hour ago, pkane2001 said:

 

That's precisely why I wrote DISTORT. Measurements are great, but they are just an engineering result, showing how well a device is working. They don't tell the whole story of what's audible, or what distortions I may find pleasant.

 

DISTORT lets me apply controlled levels of various distortions (in any combination) to any music track so I can determine if it rises to audible level while listening on my "perfect" system. Anyone can do the same with this app, with their music, in their system.

 

@Archimago's latest internet blind test is an attempt to do this on a larger scale, gathering input on audibility of various levels of THD. 

 

This is a(nother) wonderful tool, though we might have to get at least one step more sophisticated to get to where we could be relevant to the subjective listening experience.

 

What I mean is that when we listen to audio, we are trying to hear music rather than trying to hear distortion. So learning at what level distortion is audible is of course a good thing, but it may not be the same as the level at which it affects the experience of listening to the music. Using DISTORT in combination with DeltaWave to compare two files where one has a level of noise higher than the other (but still below audibility) might do it.

 

[Miska]

1 hour ago, Arpiben said:

Otala/Leinonen/Curl TIM papers are from the seventies. Anything else significant and new to measure at DAC's output? Any new Input stimulation signal rather than sinusoïdal tones?

 

And THD/IMD measurements are even older. 1/3 octave multi-tone is good, 7 kHz square wave (non-aliasing) is good. J-test and the filter tests. And then I use various things like this:

[CG]

27 minutes ago, Jud said:

 

Not an expert, but my impression is that errors would not be small, i.e., not just audio a little less nice, but evident like dropouts, etc.

What is the sound of one corrupted sample?  (I can't answer)

 

Do the processors used to fix up bad packets generate more noise when fixing said packets up that couples to the DAC chip, clock, or analog circuit?  (Dunno)

 

 

[CG]

2 minutes ago, Miska said:

 

And THD/IMD measurements are even older. 1/3 octave multi-tone is good, 7 kHz square wave (non-aliasing) is good. J-test and the filter tests. And then I use various things like this:

 

Have you looked at tests that might, ahh, test circuit memory?  I don't mean memory as in RAM devices, but memory such as dielectric absorption in capacitors, power supply sag, thermal effects, and any others you can think of?

 

 

[CG]

24 minutes ago, Jud said:

 

This is a(nother) wonderful tool, though we might have to get at least one step more sophisticated to get to where we could be relevant to the subjective listening experience.

 

What I mean is that when we listen to audio, we are trying to hear music rather than trying to hear distortion. So learning at what level distortion is audible is of course a good thing, but it may not be the same as the level at which it affects the experience of listening to the music. Using DISTORT in combination with DeltaWave to compare two files where one has a level of noise higher than the other (but still below audibility) might do it.

 

 

Of course, the other approach is to just set aside the human part to a large degree and then work to really test and develop all the individual system components as well as their combined contribution to the system performance.

 

But, this is like going to the physician and telling the health care professional that you have a pain on the right side of your stomach.  Knowing that helps them focus where to start their investigation.

 

Ever see the attached bumper sticker?

 

I think a good one for audio guys would be:  investigate more - conclude less.

 

 

[CG]

48 minutes ago, Jud said:

 

Inexpensive methods of doing so for the home audio enthusiast?

 

It depends on what you call "inexpensive".  $50K?  $50?

 

I'm fond of this:  https://store.digilentinc.com/analog-discovery-2-100msps-usb-oscilloscope-logic-analyzer-and-variable-power-supply/

 

in combination with the current probes here:  http://www.interferencetechnology.com/wp-content/uploads/2012/04/Wyatt_NA_DDG12.pdf

 

Can you start your own certified test lab with these?  How about claiming calibrated-to-a-standard test results?  Nope and nope.  You might need a preamplifier, which you can often buy from vendors on eBay or at DIYAudio.  Or, you might need to build some of it yourself.  Part of the adventure!  (If you want to go out beyond about 20 MHz, you'll need to upgrade.)

 

It's truly amazing what you can buy today to perform pretty exotic testing at home, at least exotic compared to even 10 years ago.  All for not much money and not much space.  (My rule is that all the test gear MUST fit onto two shelves in a spare bedroom closet.)  

[fas42]

5 hours ago, Jud said:

 

Of course this raises what's somewhat the flip side of good measurements correlating with subjective good impressions - what types and levels of distortion and noise are euphonic for at least some people? (Beyond the usual mention of tube electronics euphony, there are things like the Aphex Aural Exciter, literally a piece of electronics to produce noise and hash, used in the production of Born to Run, for example.) 

 

For myself, none. Once one appreciates that one can "leave the system behind", and only hear what's on the recording, subjectively, then anything that gets in the way of achieving that is useless. It's immediately irritating, like having someone pour some substance over the windshield of your car, "to enhance the view" - all you want is completely unimpeded vision of what's on the road in front of you.