Audio Blind Testing

[knickerhawk]

6 hours ago, botrytis said:

 

First, that is not blind test. the reason being you are comparing high res vs CD quality. Second, you do not know if the MQA and CD files are from the same master, so this is a HUGE problem with this test. Before you do any test likse this please read on how to actually perform a blind test. You have done at least 4 blunders that basically negates your conclusion.

Perhaps you should take your own advice and read up on what (who, actually) is blinded in a blind experiment. It is the test participant. You seem to be laboring under a false impression that a blind experiment is not blind unless the items being tested are equalized to some common standard. Not so. While such equalization might be important for purposes of isolating perceivable differences and drawing conclusions about the sources of any such perceivable differences, it is not related to whether the test itself is blinded. 

 

In my test I was, indeed, blinded. The personal purpose for conducting the test was confirmation of ANY perceivable differences, REGARDLESS of the cause of such differences. Put another way, the purpose of the test was to eliminate bias as a factor in my differentiations between CD versions and MQA versions of tracks played back on Tidal via my BlueSound front end and DAC. I do not make any claims beyond that (and beyond the assertion that I am capable of reliably distinguishing between CD versions and MQA versions on my own system using Tidal based content). I leave it to others and other threads to hash out the underlying reasons for those audible differences. I mentioned my experiment here because it relates to the subject of the thread - namely, audio BLIND testing.

[Ron Scubadiver]

8 hours ago, Ralf11 said:

Ron - do a search for SACD + meta-analysis here for a cite I posted some time ago.  It appears there a marginal improvement with SACDs over CDs.

 

 

 

I honestly don't think that SACD study is the alpha and omega on SACD vs Redbook, but it might be like coffee after an evening of drinking.  I like the way DSD files play on my system, and now we have high res downloads to entertain us as well.

[fas42]

34 minutes ago, pkane2001 said:

 

Frank, that's a very tenuous explanation for why a DBT might fail to detect a difference. You are basing this on anecdotal evidence of cable 'stabilization'. And, you are missing the point, again: the same cable behavior applies whether or not you are doing sighted or unsighted testing. So, any findings of different sound between cables (or no difference) is just as valid or invalid in both cases.

 

There's nothing special that makes a DBT test more susceptible  to timing due to cable stabilization than a sighted test. Unless, of course, you claim some quantum mechanical effect related to a wave function collapse in a sighted test

 

 

 

Hey, I don't want to get into a full blown argument of why conventional DBT may or may not detect differences, re. cables, . If one wanted to pursue it as a more rigorous exercise the only way I could see it capable of giving worthwhile results would be to have two identical systems, carefully assessed to be effectively impossible to distinguish; and then use cable A on one, and cable B on the other - this would take stabilisation out of the equation.

 

Cables are boring ... so long as they don't affect the sound is all I worry about - and basic stuff is fine, so long as they are installed well.

[STC]

8 hours ago, gmgraves said:

 

And of course, that person shouldn't know which device is being "hooked-up" any more than the listeners shotld. Otherwise it's not a true DBT.

 

The co-inventor (with Ben Muller) of the ABX comparator, one Arny Krueger,  invented the thing (by his own admission) in order to prove that everything (except speakers) sounded the same. This is a point that he argued on Usenet for years against me and others (and notoriously against John Atkinson in a famous, public debate). After debating with his nonsense for years about amplifiers, DACs, disc players and vinyl setups, I came to the conclusion that he couldn't hear. He made so many ridiculous assertions, that as far as I'm concerned, the man has no credibility whatsoever!  Some of his assertions were: the original Dynaco Stereo 120 Solid-State amplifier from the 1960s sounded exactly like the then latest Amps from Krell, Pass, Audio Research, etc. This was at a time when it common knowledge that anyone could see (and ostensibly hear) the nasty crossover notch from a sine wave on the oscilloscope caused by the 2N3055 output transistors on the ST120 being too slow to switch fast enough and too fragile to be biased far enough into class AB to eliminate the notch. The amp sounded awful and was only tolerated by the audiophile public because it was cheap, and powerful (for the day) and the audio press was touting the "transistor sound" as being a good thing! He also asserted that the (then) latest $100 Japanese receivers from Costco sounded exactly like "so-called high-end amplifiers" costing upwards of one-hundred times as much! He also maintained that he was still using the original Sony CDP-101 player from 1982, and that it sounded just exactly like the latest high-end players from MSB, dcS, etc. and that they were a rip-off! Another of Krueger's classic idiotic assertions was that the latest turntable/arm/cartridges were no better than those of the 1960's and that absolutely no progress had been made in that field! While it is true that some decks from those days can still be satisfying performers when restored (Garrard 301, 401, Thorens TD-124, TD 125, the AR turntable (sans arm), etc), arms and cartridges and decks have improved in leaps and bounds. I've had turntables from the '60's, '70's, '80's  up to the present, and I can tell that the best vinyl rigs of today will knock the sox off of the best that any 20th century playback rig had to offer (not to say that these older decks can't sound good, but they simply cannot retrieve from the grooves the level of SQ that today's best vinyl rigs can. It's an eye-opening experience to hear what even old LPs can sound like on a state-of-the-art rig from Walker, VPI, or Air Force or Clearaudio (to name a few)!

My point is how can an ABX comparator designer like Krueger make a totally transparent comparator when he can't hear the difference between transparent and non-transparent or the differences between the equipment likely to be tested by it? 

 

Are you referring to rec.audio-high end newsgroup? I vaguely remember the "debate" which got him banned from the group before readmitting him after about year.

 

Arny never claimed that everything sounded alike. The blind test data clearly showed a difference between Dyna400 and a tube amplfier. Even the Sony and Philips CD players sounded different under blind testing. 

[esldude]

3 hours ago, knickerhawk said:

Perhaps you should take your own advice and read up on what (who, actually) is blinded in a blind experiment. It is the test participant. You seem to be laboring under a false impression that a blind experiment is not blind unless the items being tested are equalized to some common standard. Not so. While such equalization might be important for purposes of isolating perceivable differences and drawing conclusions about the sources of any such perceivable differences, it is not related to whether the test itself is blinded. 

 

In my test I was, indeed, blinded. The personal purpose for conducting the test was confirmation of ANY perceivable differences, REGARDLESS of the cause of such differences. Put another way, the purpose of the test was to eliminate bias as a factor in my differentiations between CD versions and MQA versions of tracks played back on Tidal via my BlueSound front end and DAC. I do not make any claims beyond that (and beyond the assertion that I am capable of reliably distinguishing between CD versions and MQA versions on my own system using Tidal based content). I leave it to others and other threads to hash out the underlying reasons for those audible differences. I mentioned my experiment here because it relates to the subject of the thread - namely, audio BLIND testing.

Were levels precisely the same?  Question #1.  Deciding this by ear is nowhere near good enough. 

 

Given your test conditions it will be difficult to determine this. 

 

I understand your limited claims, but if all you found were louder files sound better than quieter files then this is not much of a useful finding in any regards to MQA.

 

The next question already asked is about the mastering being the same.  Same as above, finding two masters sound different doesn't tell us much about MQA.  

 

So if you can somehow confirm an MQA and non-MQA track are same master, and levels match you'll be onto something. 

 

Without at least those two conditions being met, two files sounded different for reasons unknown.  

[gmgraves]

11 hours ago, fas42 said:

 

Things like interconnect cable differences will always fail in DBT - that's because the time factor aspect is never part of the test - first rule of science: always make sure you are really testing what you think you are attempting to verify, rather than doing a show off demo of the fact that you can play "scientist in a lab coat".

 

Cabling behaviour varies over time - because of material considerations - there are all sorts of subtleties involved here, which matter when the best in reproduction is being aimed for. Claiming that this sort of thing doesn't exist, "because it shouldn't !! ", is not very helpful ...

You're right. Interconnect differences will always fail a DBT because they don't exist. When someone switches between a cheap Radio Shack cable and an expensive interconnect from the likes of Kimber or AudioQuest, etc. and nobody in the room notices even the slightest or subtlest difference or can even tell that there has been a switch, then it must be because there is no difference. Going off half cocked looking for metaphysical reasons why something as simple as an audio cable "fails" a DBT, isn't very helpful either.

[gmgraves]

9 hours ago, Ralf11 said:

I've heard people claim that different masterings are often used for the Redbook layer on SACDs - anybody know for sure?

 

It seems like an odd thing for a recording co. to do...

It's true. I asked Paul Stubblebine that exact question, and he said that as often as not, the CD layer comes from an earlier Red Book master.

[gmgraves]

3 hours ago, STC said:

 

Are you referring to rec.audio-high end newsgroup? I vaguely remember the "debate" which got him banned from the group before readmitting him after about year.

 

Arny never claimed that everything sounded alike. The blind test data clearly showed a difference between Dyna400 and a tube amplfier. Even the Sony and Philips CD players sounded different under blind testing. 

I am referring to rec.audio-high end. And the Arny Krueger that I remember, argued most vociferously that everything sounded the same when I was contributing to that group. After I left, I don't know what might have transpired. 

[firedog]

16 hours ago, Ron Scubadiver said:

There is a well known blind test which showed people could not tell the difference between SACD's and CD's.  Now either they cheated somehow, or a lot of people are wasting money on SACD's.  You can find this on the Wikipedia page for SACD. There is another one where one set of gear was a consumer DVD player hooked up to $200 class AB amplifier (A500) with a $5 interconnect and the other set of gear was $12k of high end CD transport and so forth.  A slight preference was shown for the DVD player and about a third of the subjects had no preference.  Some nice stand mounts were used as speakers.  Note the input level controls were used on the A500 instead of a preamp and these have documented measurable problems with introducing distortion.

 

It kind of makes me wonder...

The "well known blind test" you are referring to is apparently the Meyer Moran study. It was shown to be full of holes - one of which is that much of the "hi-res" material they used were SACDs produced from upsampled Redbook or other content that wasn't recorded in hi-res in the first place, so they were making a false comparison. 

More recently, try the meta analysis which says studies show an ability for listeners to perceive hi-res:

http://www.aes.org/tmpFiles/elib/20180108/18296.pdf

[sandyk]

33 minutes ago, gmgraves said:

You're right. Interconnect differences will always fail a DBT because they don't exist.

 George

 I thought that you had got well past these dogmatic assertions  about Interconnects, but apparently not.

Interconnects of the same length , but of different construction can sound quite different due to differences in capacitance, immunity to RF/EMI  etc. A Blue Jeans LC1 for example , with just over half the capacitance of many other interconnects can sound quite different to a typical coaxial construction type cable of the same length due to interactions with the output stage of the DAC, Preamp etc.  Many commonly used I.C.s used in consumer gear output stages do not like driving higher capacitance loads, and can even become unstable when driving higher than average cable capacitance and often use series output resistors of typically 100 ohms to help prevent this.

The LM4562 for example, is quite fussy in this respect.

Alex

 

P.S.

This is starting to sound a bit like Groundhog Day !

[STC]

53 minutes ago, gmgraves said:

I am referring to rec.audio-high end. And the Arny Krueger that I remember, argued most vociferously that everything sounded the same when I was contributing to that group. After I left, I don't know what might have transpired. 

 

Did he? But the original ABX website contained many equipment that actually sounded different under ABX. 

 

All I remember, he only insisted some did not sound different. 

[pkane2001]

2 hours ago, sandyk said:

different construction can sound quite different due to differences in capacitance, immunity to RF/EMI  etc. A Blue Jeans LC1 for example , with just over half the capacitance of many other interconnects can sound quite different to a typical coaxial construction type cable of the same length due to interactions with the output stage of the DAC, Preamp etc. 

 

Alex, I just looked up LC1 cable. For a 6ft length LC1 has capacitance of 73pF, resistance of 0.204Ω . In comparison, a 25ga zip cord of the same length has 96pF C and 0.3Ω R. If this is enough to cause a problem with an output stage designed for audio frequencies, then I'm afraid it's just not designed well at all!

 

And here are the filtering characteristics of  a 6ft LC1 cable over 10-30,000Hz range:

 

 

 

 

And here's one of a 6ft, 25ga zip cord:

 

 

If you think you can hear the difference of under 1/1,000,000,000 of a dB at 30KHz, I'd be extremely impressed!!! 

 

[Ron Scubadiver]

2 hours ago, firedog said:

The "well known blind test" you are referring to is apparently the Meyer Moran study. It was shown to be full of holes - one of which is that much of the "hi-res" material they used were SACDs produced from upsampled Redbook or other content that wasn't recorded in hi-res in the first place, so they were making a false comparison. 

More recently, try the meta analysis which says studies show an ability for listeners to perceive hi-res:

http://www.aes.org/tmpFiles/elib/20180108/18296.pdf

Yes and no.  The SACD's were not the best, but they were SACD's.  That's a problem with the industry, not the format.  Likewise, there are upsampled high res downloads for sale.  There are all sorts of criticisms of blind testing.  

[firedog]

11 minutes ago, Ron Scubadiver said:

Yes and no.  The SACD's were not the best, but they were SACD's.  That's a problem with the industry, not the format.  Likewise, there are upsampled high res downloads for sale.  There are all sorts of criticisms of blind testing.  

Just no.

You can't compare an SACD that isn't produced from actual hi-res to CD and then conclude that people can't hear the difference betweeh hi-res and Redbook, because you aren't comparing actual hi-res to Redbook. So their test didn't prove what they were trying to or what is claimed for it.

It possibly proved that if you upsample Redbook to hi-res people can't tell the difference. Is that "finding" useful to you? If it is, fine. But don't claim the test proved something it didn't. 

 

This isn't a criticism of blind testing per se, but of the methodology of that test. There are lots of other holes in that paper - enough that it shouldn't be cited as proof of anything. It just was a poorly run session that didn't meet proper standards.

 

There is very little upsampled fake hi-res material being sold as hi-res downloads today. Almost all the hi-res download vendors test product to make sure it is actually hi-res and often provenance information is given on the album page.  Several years ago there was a problem with  labels supplying the vendors with fake hi-res,  but it has almost ceased to exist as an actual issue. When you find some significant number of fake hi-res downloads being sold, let me know. I don't think you can do it. 

[audiventory]

1 hour ago, firedog said:

You can't compare an SACD that isn't produced from actual hi-res to CD and then conclude that people can't hear the difference betweeh hi-res and Redbook, because you aren't comparing actual hi-res to Redbook. So their test didn't prove what they were trying to or what is claimed for it.

 

Comparison of high resolution, SACD and CD is technically impossible. Even for single DAC. Because different electrical circuits work for each of these modes (see "PCM versus DSD DAC" part): https://samplerateconverter.com/educational/what-dsd

[marce]

17 hours ago, fas42 said:

 

Things like interconnect cable differences will always fail in DBT - that's because the time factor aspect is never part of the test - first rule of science: always make sure you are really testing what you think you are attempting to verify, rather than doing a show off demo of the fact that you can play "scientist in a lab coat".

 

Cabling behaviour varies over time - because of material considerations - there are all sorts of subtleties involved here, which matter when the best in reproduction is being aimed for. Claiming that this sort of thing doesn't exist, "because it shouldn't !! ", is not very helpful ...

Provide some evidence of what you claim, we can easily measure and show difference is cables that are so low to be inaudible, all cables are different to an extent. It is whether this difference is audible, and mostly it isn't, unless the cable is either bad or designed to sound different!

[sandyk]

Quote

Alex, I just looked up LC1 cable. For a 6ft length LC1 has capacitance of 73pF, resistance of 0.204Ω . In comparison, a 25ga zip cord of the same length has 96pF C and 0.3Ω R. If this is enough to cause a problem with an output stage designed for audio frequencies, then I'm afraid it's just not designed well at all!

 

And here are the filtering characteristics of  a 6ft LC1 cable over 10-30,000Hz range:

 

 

 

 

And here's one of a 6ft, 25ga zip cord:

 

 

If you think you can hear the difference of under 1/1,000,000,000 of a dB at 30KHz, I'd be extremely impressed!!! 

 

 

For starters, nobody in their right mind would use a Zip cord for an Interconnect !!!

 

I was talking about coaxial type cables as commonly used for Audio interconnects.

Some time back I purchased 2 6ft.Blue Jeans cables for use connecting my DAC to the Preamp at the other side of a wide wooden Entertainment centre with Perspex doors and equipment shelves.

I found that the Blue Jeans LC1 cables made drums sound a little too metallic, and reverted back to the DIY cables as described below.  I gave the Blue Jeans cables to a friend.

 

The attached is from a Jaycar WB1508 cable that I have used for making high quality interconnects with good quality metal RCA plugs. :

High quality OFC mono audio cable, double screened for extra shielding. Ideal for making your own RCA leads etc.
- Conductor Material: OFC
- Insulation Material: PPE
- Stranding: 45/0.12mm
- Braid Stranding: 96/0.10 + Alum
- Conductor Resistance: 0.0365ohms/M
- Capacitance: 66.8pf/1M
- Impedance: 65ohms@1m
 

Before you jump down my throat and tell me that drums are LF and the cables can't possibly have any effect, have a look at the attached below which shows the VERY fast rise times of the envelope.

Click on the image for a larger image.

P.S.

 You would be surprised to see what small differences many members are capable of hearing, even the effects of improved screening on a couple of feet of D.C. cable !

Check out the Uptone area of the forum - DIY DC Cables, and an improved design by E.E. John Swenson. 

 

Yim Hoh_Man_Poem of Chinese Drum

[knickerhawk]

12 hours ago, esldude said:

Were levels precisely the same?  Question #1.  Deciding this by ear is nowhere near good enough. 

 

Given your test conditions it will be difficult to determine this. 

 

I understand your limited claims, but if all you found were louder files sound better than quieter files then this is not much of a useful finding in any regards to MQA.

 

The next question already asked is about the mastering being the same.  Same as above, finding two masters sound different doesn't tell us much about MQA.  

 

So if you can somehow confirm an MQA and non-MQA track are same master, and levels match you'll be onto something. 

 

Without at least those two conditions being met, two files sounded different for reasons unknown.  

Regarding sound levels being the same, the answer is “Yes” per my iPad spectrum analyzer app. If you expect anything more than that, then I can’t accommodate and, frankly, don’t care that much one way or the other. My testing is ultimately for my own edification and confirmation of what I’m hearing (and almost always preferring). In addition to these more “controlled” tests, the fairly serious listening I’ve done over the past month isn’t resulting in a sense of differences in sound levels but, rather, in detail and sweetness of the sound.

 

Regarding mastering sources, your guess is as good as mine, but again my interest is in the results and what sounds better to me personally. In virtually all cases, I’m preferring the MQA version (when I can make out any difference, which is most of the time I listen carefully and especially when doing rapid A/B comparisons). If that’s due to “better” mastering rather than the MQA magic, then so be it. I’ll happily accept the improvement unless/until Tidal makes available for my listening enjoyment remastered CD versions that are just as good/better than the MQA versions. 

[esldude]

20 minutes ago, knickerhawk said:

Regarding sound levels being the same, the answer is “Yes” per my iPad spectrum analyzer app. If you expect anything more than that, then I can’t accommodate and, frankly, don’t care that much one way or the other. My testing is ultimately for my own edification and confirmation of what I’m hearing (and almost always preferring). In addition to these more “controlled” tests, the fairly serious listening I’ve done over the past month isn’t resulting in a sense of differences in sound levels but, rather, in detail and sweetness of the sound.

 

Regarding mastering sources, your guess is as good as mine, but again my interest is in the results and what sounds better to me personally. In virtually all cases, I’m preferring the MQA version (when I can make out any difference, which is most of the time I listen carefully and especially when doing rapid A/B comparisons). If that’s due to “better” mastering rather than the MQA magic, then so be it. I’ll happily accept the improvement unless/until Tidal makes available for my listening enjoyment remastered CD versions that are just as good/better than the MQA versions. 

Nope, ipad spectrum analyzer won't cut it.  If you don't care, I don't either. 

 

I'll reiterate ," Without at least those two conditions being met, two files sounded different for reasons unknown."

 

You'll, like many others feel edified.  And that is about all your methodology is worth.  If that is good enough, then fine.  Simply means not much of anything.   

[sandyk]

4 minutes ago, esldude said:

Nope, ipad spectrum analyzer won't cut it.  If you don't care, I don't either. 

 

I'll reiterate ," Without at least those two conditions being met, two files sounded different for reasons unknown."

 

You'll, like many others feel edified.  And that is about all your methodology is worth.  If that is good enough, then fine.  Simply means not much of anything.   

Dennis

 Given that the normal CD spectrum  area of both files is likely to be the same,(?)  why should it matter, especially as you have stated on numerous occasions that anything higher than 16/44.1 is a waste of time and that high res audio can't be statistically verified as being worthwhile .

Your long standing stance in this area disqualifies you from making a legitimate judgement on MQA sound quality, which can NEVER be better than the original high definition music file that it was sourced from anyway, so why offer MQA as an alternative other than for sinister DRM purposes ? 

Perhaps the Industry regrets making high res files available from HD Tracks etc. ?

Alex

[fas42]

6 hours ago, marce said:

Provide some evidence of what you claim, we can easily measure and show difference is cables that are so low to be inaudible, all cables are different to an extent. It is whether this difference is audible, and mostly it isn't, unless the cable is either bad or designed to sound different!

 

Your claim is that differences in cable materials, construction, how it is connected into the circuit is always inaudible - many who have investigated this as an experimental exercise, as compared with doing a quick switcheroo to confirm a prejudged notion, have found otherwise. I would never have got the quality of sound I talk about, and still couldn't; if I didn't take this aspect seriously.

 

Many audio people can never get the concept, no matter how many times it's repeated to them - a chap with a Ferrari wants to know which are the optimum tyres for him driving his car close to its limits; someone who has a shopping runabout says, he's tried the very expensive, top technical rating variety, and it made not an ounce of difference to the driving experience - well, that was good advice, wasn't it ... ?

 

Which doesn't mean buying expensive stuff - it means, that if one wants more than average performance then everything needs to constantly assessed, to make sure it doesn't make a difference ..

[fas42]

Sound levels don't matter, one iota - if one knows what to listen for  - as soon as the word "better" is tossed around in a conversation about audible differences, I roll my eyes - it means that the people haven't got a clue about how to assess SQ, and perhaps will never have ...

[sandyk]

10 minutes ago, fas42 said:

Your claim is that differences in cable materials, construction, how it is connected into the circuit is always inaudible - many who have investigated this as an experimental exercise, as compared with doing a quick switcheroo to confirm a prejudged notion, have found otherwise.

 

 When Marc gets his act together, he really should try a decent low noise Linear PSU for his antiquated (In SQ terms) beloved SBT ! He may then start to get a bit more adventurous and discover that his theoretical based dogmatic assertions on what is, and isn't audible, aren't always correct, as it's resolving abilities further improve.

I don't doubt that Marc is a whiz in the area he works in, but you can't always directly transfer all of this to other fields such as Audio, or even Digital Video for that matter, which also includes Audio.

[fas42]

1 hour ago, sandyk said:

 

For starters, nobody in their right mind would use a Zip cord for an Interconnect !!!

 

 

Ummm, bits of wire lying around is good enough, most times - if used well. Current experimental rig, NAD components, uses a bit of a single twisted pair, solid strand Cat cable between source and amplifier - works well enough to make all the other weaknesses which are holding back the sound obvious. Perhaps it will be worth making that connection "better" down the track, perhaps not ...

[fas42]

7 hours ago, audiventory said:

 

Comparison of high resolution, SACD and CD is technically impossible. Even for single DAC. Because different electrical circuits work for each of these modes (see "PCM versus DSD DAC" part): https://samplerateconverter.com/educational/what-dsd

 

And this is the point. Playback quality is a result of the full chain in operation, source to speakers. Thinking otherwise is as silly as listening to a demo at an audio show, which "proves" how good an amplifier is - ummm, you mean the player, DAC, preamp, speakers, room, setup skills have absolutely nothing to do with it ... ?