Blindtest between sample rates

[adolf512]

I have constructed a blindest between 3 different sample rates. The original file was 24bit 96KS/s and the worst of my files is 44.1KS/s. Please tell your result from blind testing if you have done multiple independent listenings. https://www.dropbox.com/sh/j1535de2zh51ooe/AACMBuey9leK4_5JizDghiAra?dl=0

 

The names is of course random. The bit-rate is also somehow random since i might have added noise to some files in order to increase the bit-rate and make cheating harder(I can also use different compression settings when creating the flac file). Please do not tell which file is which if you have cheated using suitable program, send a PM if you have questions of any kind. Link to the songs Overgrown / Frozen Dandylions - Frozen Dandylions / Overgrown - Encyclopaedia Metallum: The Metal Archives I used "untitled" and edited out 75% of the songs

 

Recommendations

-make sure your audio is bitperfect(mostly relevant for windows users)

-have tweeters with acceptable performance up to at least 30khz

-If you have cheated with suitable software, do not reveal which file who is which. If you(or suitable animal) just have listened you can tell all your findings here

[audiventory]

Hi Adolf512,

 

Are you want check audio resolution of files as itself or check hardware/software?

 

Best regards,

Yuri

[adolf512]

Are you want check audio resolution of files as itself or check hardware/software?

The purpose is to see how down-sampling affect the listening experience. However without speakers with good acceptable Frequency response above 20khz you are doomed to fail these tests which makes the polls useless diyAudio - View Poll Results only 1 in 4 voted that they heard a difference and one vote for frozen_E being the worst(which is different from this poll).

[goldsdad]

However without speakers with good acceptable Frequency response above 20khz you are doomed to fail

 

Since I don't hear beyond 17 kHz, would I be doomed to fail?

[misterspense]

Am I still the only one that wants to see if he/she can hear a difference? For the statistics it would be nice if some more people tried and voted.

[jhwalker]

I listened a and posted my answer above, but I think what you're potentially testing here is the quality of your resampler using your choice of settings (including filtering, dithering, etc.). It is entirely possible to make a good sounding 44/16 version of a 96/24 "master" if you use good settings (either accidentally on purpose); likewise, it is possible to really screw it up by choosing injudicious settings.

[Tony Lauck]

I listened a and posted my answer above, but I think what you're potentially testing here is the quality of your resampler using your choice of settings (including filtering, dithering, etc.). It is entirely possible to make a good sounding 44/16 version of a 96/24 "master" if you use good settings (either accidentally on purpose); likewise, it is possible to really screw it up by choosing injudicious settings.

 

It is possible to convert 96/24 to 44/16 and get "good" results. However, for most music my experience is that it is not possible to get "excellent" results with the 44/16 format. I can hear differences with these conversions on many recordings, despite the fact that my aged ears don't hear sine waves above 13 kHz. This has even happened when starting from 96/24 files made by digitizing cassette tapes off my Nak CR-7a deck.

 

Incidentally, changing from 96/24 to 44/16 involves changing two variables. To get a more manageable experiment I suggest two separate experiments: 96/24 vs. 44.1/24 as one experiment and 44.1/24 vs. 44.1/16 as another experiment.

[adolf512]

Since I don't hear beyond 17 kHz, would I be doomed to fail?

Not if this study is correct http://jn.physiology.org/content/83/6/3548

According to this pdf http://www.tinnitus.vcu.edu/Pages/Ultrasonic%20Hearing.pdf ultrasonic hearing generally requires direct bone contact but is possible up to 150khz

 

I listened a and posted my answer above, but I think what you're potentially testing here is the quality of your resampler using your choice of settings (including filtering, dithering, etc.). It is entirely possible to make a good sounding 44/16 version of a 96/24 "master" if you use good settings (either accidentally on purpose); likewise, it is possible to really screw it up by choosing injudicious settings.

I used audacity and later verified that there wasn't any difference below 25.5 khz between the original version and the 54KS/s version. The file is frozen-delta and the resolution of this FFT is 4096

[goldsdad]

Not if this study is correct Inaudible High-Frequency Sounds Affect Brain Activity: Hypersonic Effect | Journal of Neurophysiology

According to this pdf http://www.tinnitus.vcu.edu/Pages/Ultrasonic%20Hearing.pdf ultrasonic hearing generally requires direct bone contact but is possible up to 150khz

 

Well, I refuse to directly connect my bones to my speakers, so I guess I am doomed.

[adolf512]

I found this about ultrasonic brain stimulation(500khz) http://blogs.discovermagazine.com/neuroskeptic/2014/02/09/ultrasonic-brain/

http://www.unknowncountry.com/news/scientists-can-affect-brain-responses-using-ultrasound-waves

very interesting stuff!

 

Incidentally, changing from 96/24 to 44/16 involves changing two variables. To get a more manageable experiment I suggest two separate experiments: 96/24 vs. 44.1/24 as one experiment and 44.1/24 vs. 44.1/16 as another experiment.

All files is 24bit, this recording is not suitable for comparing 16bits against 24 bits in any case. i choosed this track due to the powerful ultrasonics(this are not noise but overtones).

[Tony Lauck]

All files is 24bit, this recording is not suitable for comparing 16bits against 24 bits in any case. i choosed this track due to the powerful ultrasonics(this are not noise but overtones).

 

That's good. Sorry I complained. Next time I will read more carefully.

[audiventory]

The purpose is to see how down-sampling affect the listening experience. However without speakers with good acceptable Frequency response above 20khz you are doomed to fail these tests which makes the polls useless diyAudio - View Poll Results only 1 in 4 voted that they heard a difference and one vote for frozen_E being the worst(which is different from this poll).

 

Hi Adolf512,

 

You can check my AuI ConverteR 48x44 FREE (quality not limited). It can work with apparatus that have far "non-ideal" ultrasonic range.

 

It achieved due filtrations upper 20kHz (slow math of studio level, of course). I.e. it work as DAC's resampling filter but in power computer domain without time limitation.

 

Also it allow work with DXD or any stuff derived by DSD/DXD without this hi-frequency noise after convertsion even without resampling.

 

Compare on hear my way (minus ultrasound) with "traditional" (0 ... [sample rate]/2) converters, please. I think, it will interesting in frame of the test.

 

Best regards,

Yuri

[blownsi]

Did anyone else perceive volume differences between files? I can't help but think that's to throw us off.

[adolf512]

Did anyone else perceive volume differences between files? I can't help but think that's to throw us off.

I just checked the files using audacity, no difference between the original and 44.1KS/s version below 20.5khz

 

I played the difference files using audacity and the peak volumes where (-23dB,-29dB) for the (44.1, 54)KS/s versions.

[audiventory]

Here need check volume via special pro volume measurement tool due different sample rate.

 

It's integral characteristics.

 

Difference in 1-2 dB we can't see via spectrum, due it distributed in range 0 ... samplerate/2.

 

Also neasurements can differ by listening perception due intermodulation distortion of analog part.

 

In adible range will shifted som ultrasound components what is not considered by measurement tool.

 

Also need consider correction curve of measurement tool to ear perception .

 

More correct result can give pre-calibration of sample rate converter.

 

For more pure experiment better remove ultrasound as possible for avoiding influence of this factor.

[Tony Lauck]

Here need check volume via special pro volume measurement tool due different sample rate.

 

It's integral characteristics.

 

Difference in 1-2 dB we can't see via spectrum, due it distributed in range 0 ... samplerate/2.

 

Also neasurements can differ by listening perception due intermodulation distortion of analog part.

 

In adible range will shifted som ultrasound components what is not considered by measurement tool.

 

Also need consider correction curve of measurement tool to ear perception .

 

More correct result can give pre-calibration of sample rate converter.

 

For more pure experiment better remove ultrasound as possible for avoiding influence of this factor.

 

When one filters one intentionally changes the spectrum. If the loudness matching depends on the spectrum, different filters will require different gain for levels to be "matched". This does not seem like proper experimenter controls to me. I vet the gain of my sample rate converter by using an audio editor to make a 1002 Hz sine wave at -12 dbFS, put this through the SRC and look at the resulting 1002 Hz sine wave at the new sample rate. It should have the same level. (It may not be perfect due to pass band ripple, i.e. gain changes as a function of the sine wave frequency, but this will be tiny with a decent SRC.)

 

It is important to leave lots of headroom in the original test files. If the original file is normalized to just below 0 dBfs then the resampling may try to generate peaks that go over the maximum. If this happens the resultant waveform will be clipped. If this happens even once it can be easy to detect for those of us who have learned to recognize this type of digital harshness. Even if all the test waveforms have some headroom there is still a need for additional headroom to allow for DACs that may clip on loud samples. This can happen in the upsampling DSP portion of a DAC chip or even in the analog portions of the DAC. Distortion in the DAC may provide the tell that unblinds a listening test.

 

With 24 bit audio there is no need for a sample rate conversion test to be anywhere near 0 dBfs. I suggest shooting for -6 dBfs, but if the peaks are below -3 dBfs this is probably OK. For dither tests there can also be a possibility of overflow when the dither noise is added in, but only a small amount of head room will be needed with normal dither algorithms.

[adolf512]

@audiventory what i did a FFT of the delta(for frozen54-frozen96) and then i looked at the frequency components.

With 24 bit audio there is no need for a sample rate conversion test to be anywhere near 0 dBfs. I suggest shooting for -6 dBfs, but if the peaks are below -3 dBfs this is probably OK. For dither tests there can also be a possibility of overflow when the dither noise is added in, but only a small amount of head room will be needed with normal dither algorithms.

That is one problem with sample rate conversions but it appears that the conversion worked out well for these particular files, it is better mastered than the avarage(still not optimal) which reduces the risk of clipping.

 

You are free to compare the files yourself, please tell me if you detected any abnormalities.

[audiventory]

When one filters one intentionally changes the spectrum. If the loudness matching depends on the spectrum, different filters will require different gain for levels to be "matched". This does not seem like proper experimenter controls to me. I vet the gain of my sample rate converter by using an audio editor to make a 1002 Hz sine wave at -12 dbFS, put this through the SRC and look at the resulting 1002 Hz sine wave at the new sample rate. It should have the same level. (It may not be perfect due to pass band ripple, i.e. gain changes as a function of the sine wave frequency, but this will be tiny with a decent SRC.)

 

It is important to leave lots of headroom in the original test files. If the original file is normalized to just below 0 dBfs then the resampling may try to generate peaks that go over the maximum. If this happens the resultant waveform will be clipped. If this happens even once it can be easy to detect for those of us who have learned to recognize this type of digital harshness. Even if all the test waveforms have some headroom there is still a need for additional headroom to allow for DACs that may clip on loud samples. This can happen in the upsampling DSP portion of a DAC chip or even in the analog portions of the DAC. Distortion in the DAC may provide the tell that unblinds a listening test.

 

With 24 bit audio there is no need for a sample rate conversion test to be anywhere near 0 dBfs. I suggest shooting for -6 dBfs, but if the peaks are below -3 dBfs this is probably OK. For dither tests there can also be a possibility of overflow when the dither noise is added in, but only a small amount of head room will be needed with normal dither algorithms.

 

Hi Tony,

 

Main idea provide absolutelly identical spectrum for all sample rates.

 

For eliminating hard controlled factors like artefacts of intermodulation distortion.

 

Oversampling can cause overload. Practically about 1 dB. May be 2 dB.

 

I added some time ago loudness adjustment for it and for decreasing DSD headroom (if record allow do it).

 

When we reasmple, as example, CD to 24/196, we can simply come to overloading. I suppose here enought 2-3 dB headroom.

 

@audiventory what i did a FFT of the delta(for frozen54-frozen96) and then i looked at the frequency components.

 

Hi Adolf512,

 

Visual spectrum analysis is not accurate instrument for loudness control.

 

For accurate control need mathematically process spectrum:

 

1. Normalize to ear perception frequency curve ( Fletcher )

2. Summ all spectral components

 

Ear feel 1-3 dB loudness difference. Thus precision of measurements here must be provided about 0,1…0,5 dB.

 

You control each spectral component, but need control normalized summ all components.

 

Exists group of standards for loudness measurements. For more infor see References part here http://samplerateconverter.com/content/broadcast-wave-format

 

I applied these loudness measurement methodics earlier in my software.

 

Best regards,

Yuri

[adolf512]

1. Normalize to ear perception frequency curve ( Fletcher )

2. Summ all spectral components

are you serious? the most best way is to just switch between the files using audacity, the eye will notice any difference, there is no loudness difference if you exclude the ultrasonics. But you are free do to any analysis you want yourself.

 

I can tell for sure the loudness difference is less than 0.1dB, and my analysis shows 0dB loudness difference when you doesn't count the ultrasonics. I am mostly interested in listening experience, another user has also verified that there isn't any clipping during the conversion.

 

What i dislike about these home-listening tests is that you can't know if someone is honest with his findings or if he has cheated before listening, also people seams to be unwilling to do any real(multiple listenings) blind testing which makes all statistics useless. Currently it's a tie between frozen_E being the worst and frozen_D being the worst(including the vote from diyaudio). Frozen_E has a smallest file size so i guess a lot of people voted for it because of that(the bitrate itself doesn't mean anything).

 

I will do a proper test myself when i have built my next system. What i will do then is

-measure the system in order to estimate how much IMD distortion there will be from the ultrasonic tones, it is likely to be very close to zero since IMD distortion is multiplicative and RAAL 140-15d has low second order distortion.

-do listening tests to see how much IMD distortion that is audible.

-maybe have some trainees(zero salary) which will do listening tests for me, once the trainee period is over they will be fired.

-

[audiventory]

I can't allow myself be non-serious

 

It's half of my work doubt, carefully check and measure. Other half of work fix reasons of bad measurements

 

I agree, remote home test is not laboratory procedure.

 

However, I always collect result of home, subjective, objective, any available for me replies.

 

In general (statistically) it correlate with laboratory measurements.

 

Also it help understand what need improve.

[PewterTA]

For me they all sound very similar (this is not how I voted though). I listed to each of the tracks multiple times. The differences I heard was in the sound stage depth, slightly better sustain in the cymbal hits, better focus (instrument placement) and slightly thinner and less "out of phase sound" on the guitar. To be honest, I don't think I could listen to that track for much longer with how out of phase the guitar seems. I don't like it when the reverb and sustain of the guitar strings are literally beside me if slightly behind. Makes me fell like I'm sitting in a bubble that's echoing the sound. The guitar was the only thing that effected me this way, flipping the phase on my NAD M51 helped some. None of the other sounds bothered me like that (everything else was correctly placed). It really made the guitars seem unrealistic.

 

But anyhow, I liked one of the files better than the others...but they were all very close. After looking at the results the file everyone seems to like the best is the one I liked the least.

[Nikhil]

Adolf,

 

Thank you for an interesting experience. I can see a lot of tangent discussions going on. If I understood correctly all you want us to say is which of the three files "sounded worst". I played each file 2 times completely. I then played different sections of the music to compare where I thought I heard differences.

 

To be honest - asking for the worst sounding is almost like a trick question. To me my brain works better if you had to ask which sounded better. So I did just that and identified what I thought sounded better. By better I felt two of the files to be marginally more relaxed - but really not by that much. The one file that strained my senses a little more got my vote for "worst".

 

Regards.

[adolf512]

You can always PM me and guess which file who is the oridginal, the reason why i ask for the worst file is because i didn't want it to be to difficult, a lot of people think it would be impossible to hear any difference between 96KS/s and 54KS/s when downsampling+upsampling correctly.

 

One user has PM:d me his guess and he correctly identified the best and worst of the files. I recommend not to look at the poll result before voting since it might influence the vote.

 

It is interesting to see that every single user who voted thinks he heard difference between the files but the poll is pretty even now..

[testikoff]

Sorry, but I find the choice of the material for this test a tad strange. The 24/96 original features: firstly, the background flowing water noise that somewhat masks musical performance; secondly, some of the elements in the mix appear to have been captured at lower sampling rate (likely @ 44.1 kHz). If I were to put together this test, I'd rather use the source material recorded, mixed (if this step was used in production) and mastered entirely in Hi-Rez (i.e. 24/88, 24/96, 24/176, 24/192, DXD, etc.). Tracks from, say, Sound Liaison or 2L immediately come to mind. Substantial presence of high frequency (above 20 kHz) spectral content is also a requirement (possibly from several instruments, not just percussions)...

 

BTW, all 3 24/96 test tracks included have identical perceived loudness (the exact same track ReplayGain value throughout) & are lined up time-wise, so they are perfectly suited for ABX comparisons, IMHO.

 

P.S. I got my poll vote wrong due to the request to identify the worst sounding version (instead of the best)...

[adolf512]

P.S. I got my poll vote wrong due to the request to identify the worst sounding version (instead of the best)...

Please tell me how you voted and how you should have voted instead(PM or in this thread).

 

Yes it would be nice with a better track, however it's not that easy to find tracks with powerful ultrasonics above 25khz. If there is another test the 2 of the files will be the original and 2 will be down-sampled.