Audio Blind Testing

[esldude]

7 hours ago, GUTB said:

snip

I just wanted to post that whenever I see "audio blind testing" I want to read it as "audio bling testing", which would be a much more entertaining thread.

We already know how important the bling factor is to you. 

[esldude]

2 hours ago, GUTB said:

 

These are areas I don’t think which have undergone any clinical research. They’re very well-known phenomena. @gmgraves admits to have faced it himself, but he chooses to believe the differences he heard were in his head after being influenced by the stress of blind testing. I’ve experienced these phenomena myself in ABX testing and just regular A/B-ing where I do it enough times.

 

Fremer of Analog Planet did a public shoot-out of a bunch of cartridges ranging from $100 or so up to around $1k (as I recall) on a fairly modest turntable and tonearm. He digitized the output of each cartridge, kept secret which file was from which cartridge, and put out a public poll asking which one people liked best. The results showed that, as expected, the tendency was to prefer the more expensive cartridge, with the most expensive (Ortofon Quintet Black?) being the most popular. But what was super interesting was that opinion was not unanimous. I can’t imagine anyone can’t tell the difference between the bundled StudioTracker that came with my Studio Deck and the AT-OC9ML/II that I upgraded to, let alone prefer the former, yet, Fremer’s poll showed just that and worse; some people actually preferred the cheap budget junk the best. What happened very likely was that people listening to all these files one after the other started to mix them together in thier heads. The difference between some cheap elliptical MM and a highly regarded high performance line contact MC isn’t a minor one. You can’t blame setup because it was set up by one of the leading authorities in turntable setup.

 

Another example of audio memory at work. Have you ever listened to some familiar music through some good headphones and you picked up a detail you hadn’t noticed before? From then on you will be able to hear the detail; you’re brain has decided it should be there, so it’s there. The sound IS there, but just much less obvious due to a speaker’s poor low level sound production.

 

In the Music Server forum I recently complained that a tweak (low noise regulator on SSD) I went through a lot of trouble and time to implement resulted in a huge downgrade. I immediately noticed a big increase in glare / sibilance and collapse of soundstage, issues that were fixed after removing the regulator. Shouldn’t I have convinced myself by then that I was opening the gates to audio nirvana? Why did I instead perceive a big reduction in quality? I’m sure I could ABX the difference with certainty — but could I do it after 20 plays? 30? 50? I’m not sure! I say this because I’ve experienced first hand how ABX testing quickly mixes everything together.

 

 

Okay so Mikey didn't match volume levels.  I mean didn't even attempt to match them.  The files you can download have a level difference that ranges over 10 db.  Unless you carefully match levels this is completely useless.  Of those who did try and match levels how many did matching by ear?  Worse than useless because they will think levels are matched (and they will be within 1 or 1.5 db), but in fact the remaining differences are plenty to skew the results dramatically.  Oh and just for good measure two of the files have clipping in them.  

 

Oh and results.  The two loudest (and clipped files) and the loudest non-clipped file garnered the most votes.   

[esldude]

3 hours ago, knickerhawk said:

Perhaps you should take your own advice and read up on what (who, actually) is blinded in a blind experiment. It is the test participant. You seem to be laboring under a false impression that a blind experiment is not blind unless the items being tested are equalized to some common standard. Not so. While such equalization might be important for purposes of isolating perceivable differences and drawing conclusions about the sources of any such perceivable differences, it is not related to whether the test itself is blinded. 

 

In my test I was, indeed, blinded. The personal purpose for conducting the test was confirmation of ANY perceivable differences, REGARDLESS of the cause of such differences. Put another way, the purpose of the test was to eliminate bias as a factor in my differentiations between CD versions and MQA versions of tracks played back on Tidal via my BlueSound front end and DAC. I do not make any claims beyond that (and beyond the assertion that I am capable of reliably distinguishing between CD versions and MQA versions on my own system using Tidal based content). I leave it to others and other threads to hash out the underlying reasons for those audible differences. I mentioned my experiment here because it relates to the subject of the thread - namely, audio BLIND testing.

Were levels precisely the same?  Question #1.  Deciding this by ear is nowhere near good enough. 

 

Given your test conditions it will be difficult to determine this. 

 

I understand your limited claims, but if all you found were louder files sound better than quieter files then this is not much of a useful finding in any regards to MQA.

 

The next question already asked is about the mastering being the same.  Same as above, finding two masters sound different doesn't tell us much about MQA.  

 

So if you can somehow confirm an MQA and non-MQA track are same master, and levels match you'll be onto something. 

 

Without at least those two conditions being met, two files sounded different for reasons unknown.  

[esldude]

20 minutes ago, knickerhawk said:

Regarding sound levels being the same, the answer is “Yes” per my iPad spectrum analyzer app. If you expect anything more than that, then I can’t accommodate and, frankly, don’t care that much one way or the other. My testing is ultimately for my own edification and confirmation of what I’m hearing (and almost always preferring). In addition to these more “controlled” tests, the fairly serious listening I’ve done over the past month isn’t resulting in a sense of differences in sound levels but, rather, in detail and sweetness of the sound.

 

Regarding mastering sources, your guess is as good as mine, but again my interest is in the results and what sounds better to me personally. In virtually all cases, I’m preferring the MQA version (when I can make out any difference, which is most of the time I listen carefully and especially when doing rapid A/B comparisons). If that’s due to “better” mastering rather than the MQA magic, then so be it. I’ll happily accept the improvement unless/until Tidal makes available for my listening enjoyment remastered CD versions that are just as good/better than the MQA versions. 

Nope, ipad spectrum analyzer won't cut it.  If you don't care, I don't either. 

 

I'll reiterate ," Without at least those two conditions being met, two files sounded different for reasons unknown."

 

You'll, like many others feel edified.  And that is about all your methodology is worth.  If that is good enough, then fine.  Simply means not much of anything.   

[esldude]

2 hours ago, Ralf11 said:

Dear Audio Ayatollah - please don't mischaracterize me or my position

 

it just makes you look worse than usual

 

BTW, people can use meters to match levels.  Not hard.

Now, now, apparently matching levels in comparisons is on the order of difficulty equivalent to the LIGO experiments.  Or so one would think with the excuses audiophiles come up with for not doing this one vital, simple and helpful thing.  

 

An incredibly large number of mythical differences vanish like a light fog on a sunny morning if levels are matched.  

 

[esldude]

36 minutes ago, fas42 said:

 

Luckily, you're wrong ... the ear/brain is remarkably adept at recreating a full sound field, if all the cues are right - but is also a harsh critic; and completely rejects an attempted illusion, if all the clues don't add up. 99.9% of systems don't get the clues sufficiently right - which would include all of those done by researchers, who then attempt to fool the brain using more crude techniques.

 

I have had systems of all sorts slip in and out of "3D presentation" over many, many years - depending upon how fussy I was in getting everything right - I know the behaviour characteristics as intimately as the back of my hand, so to speak. That researchers using results from less than competent setups to assert that something can or can't happen is meaningless ... and will eventually move forward to better understanding ...

So do we need stereo?  Can the brain remarkably create full 3D from a mono recording? 

 

In any case we have an authoritative judgement on the issue.  All we had to do was ask.  Oh, and you can't explain to anyone else how to replicate your results as usual...right?

[esldude]

54 minutes ago, pkane2001 said:

 

Just saw this blind test. Look up Mike Lavigne if you don't know who he is. Opus Transparent cables (around $40k at the time of the test) compared to the run of the mill Monster ‘cheap’ cables:

 

http://www.avsforum.com/forum/86-ultra-hi-end-ht-gear-20-000/941184-observations-controlled-cable-test-2.html#post12255000

 

 

Yes good to see that brought up. 

 

Comedy Gold!

 

Comedy because Lavigne learned the wrong thing from this experience.  Doubled down on wrongness.  

[esldude]

3 minutes ago, sandyk said:

 That is incorrect. It has even been measured by the author of the TAS 220 and 221 reports ( Dr. Charles Zellig.)

It has to do with Phase.

Also, the ever sceptical esldude (Dennis) has also discussed this aspect too

 

I have, but Zellig's methodology is ridiculous.

 

I would say generally stereo will not have height info except by accident.  Also with minimal coincidental miking you might get a little effect which will convey height somewhat. 

 

Height is heard by comb filtering of outer ear reflections to the ear canal.  This creates a dip in response that allows one to sense height.  The sensations comes from 6 khz to about 11 khz.  Higher in frequency the response dip the higher the sound is perceived to have originated.  

[esldude]

17 minutes ago, MetalNuts said:

With respect, what proof do you have?  Are you saying that you cannot tell the difference between a $90 dollar wine from $5, that means that they taste as good and how do you define good without reference to one's subjective preference.

 

Wine tasting can be very profession and there are courses and training with different levels.  I learned from my friend who attended the course but fail to proceed to higher level,  the master need to tell the origin, say French wine or Chile, then further the district, i.e. Bordeaux, the year (may be not exact but the range), the kind of grape or multi-grape.  The whole process is a blind test without revealing the label of the wine.

 

The fact that one cannot tell the difference does not make make it a universal truth.  The science today cannot prove does not necessary mean that it is false.  We need to have an open mind and in particular when ourselves are not well equipped with the knowledge of that field unless you believe that only you know everything.

 

 

You can find numerous tests about wine like described saying pricing, color and label have a much larger effect than the wine itself on which is judged superior.  I do mean numerous.  I'll not even bother to provide examples.  With google you have several in seconds if you wish. 

 

As for Sommelier training and certification one must identify 6 wines blind.  So few pass all this I even wonder if most passes aren't simply random luck involved.  Take such a thing often enough and a few guess right.  There are like slightly over 200 Master Sommelier's in the world in the past 40 years.  Maybe not luck, can't be sure.  With 99% of the people in the world, price name, and other factors effect their perception considerably.  

[esldude]

9 minutes ago, MetalNuts said:

 

So, you selectively accept the result of google search to be the truth.  I search MQA and on the 1st page, only Linn commented the negative of MQA.  So, is MQA really good?

That is a tallywhackers and groundhogs comparison there.  

 

STC provided a good starting article.  There are many more where that came from.  

[esldude]

47 minutes ago, Albrecht said:

LOL yawn, yawn, yawn,

 

So sick of the patronizing from the "everything-sounds-the-same" crowd who have no experience, - yet make assertions by what they've read on the internets. Of course, - most everyone with experience, - has the opposite viewpoint.

And, - it doesn't matter what anyone owns, - it's what they've ACTUALLY HEARD; that is what matters. The website is not called "Computer ANTI-audiophile."

 

So everyone that agrees with you has experience.  And anyone who disagrees has no experience.   Neat.  Does it leave room for anyone who has experience to disagree with you?

 

Does it matter whether what they've ACTUALLY HEARD is related to the sound that was ACTUALLY IN THE AIR?

[esldude]

31 minutes ago, gmgraves said:

 

I swore I wasn't going to respond to any more of your nonsense, but this is probably one of the most asinine assertions that you've made yet.

 

A multi-miked recording using a forest of microphones, and indeed any recording where there is a microphone and/or a tape track for each instrument, there is no imaging effect. There is no front-to-back layering nor any image height because the result of pan-potting individual instruments into place puts them all on a single plane. The effect turns out to be that of a single line of instruments stretched across the stage from right-to-left. It results in a group of solo violinists all playing together, but it doesn't sound like a string section because the violins aren't mingling together in the air, they are mingling together electronically in the mixing console, and that does NOT and can NOT sound like a string section.

Only true stereo recordings made with two coincident microphones give real, phase coherent stereo. everything else is multi-channel mono and has no real soundstage. You are wrong, again. Imaging has to be contained on the recording, and playback equipment cannot correct for the lack of it!

You saved me from typing the same thing.  Thanks. 

[esldude]

16 minutes ago, fas42 said:

 

Remember, I'm always talking about an illusion - the ear/brain is taking the raw data as heard coming from the speakers, and reconstitutes a believable sound field. And it turns out that there is enough depth information caught, or added via sound manipulation for the listening mind to make sense of it - with a rig that I'm happy with I never hear a single plane from left to right - there is a stage behind the speakers, always behind; which can be small, or gigantic; in pop productions, or heavy manipulation of the instruments, there are multiple stages, of various sizes and positions, overlaid, one on top of the other - you can move one's focus to each of these layers, and see it as having full integrity, in its own right.

 

The imaging is on the recording, but there are often many layers of such, all still having their own identity. Optimised playback allows all this information to be unraveled, in one's mind - and it is a delight to see clearly the details of the "montage".

Sounds like a messed up rig creating ersatz space, bloom and dimension whether any is on the recording or not.  Then you just imagine something more is there. 

[esldude]

6 hours ago, fas42 said:

 

Not really. Imagine you were in the studio, listening to the master tape, or console, with all the individual tracks that were combined in the final mix at your disposal - playing through a high quality monitoring speaker setup. You can solo any one of those tracks, and hear exactly what was captured on that "stage", with the added effects - with the listening, on a competent rig, that's the subjective impact: all the individual elements can be soloed, or you just let the whole flow over one. Just like being in the middle of a group of musicians fooling around; you can listen to one, and ignore the other fiddlings - the cocktail party effect ...

Typically when you do such you next have the guy at the console add some reverb or delay to "glue" the mix together.  There are a number of tricks to this by experienced people, and it goes against your hearing what you are describing here. 

[esldude]

I think some people have in mind this idea stereo sound can work like holography.  That two sources might set up an interference pattern replicating the actual sound field recorded.  It doesn't work this way however.  It is using how our hearing works to perceptually trick it,  not recreate an actual sound field.  

 

Now holography creates a pair of sources in light that interfere around objects and record that interference pattern that can be recreated with laser light. I think for the same idea to work with sound on the playback end, it might work best if we had a defined area of ultrasonic sound that could be modulated to produce such an interference pattern.  The modulation would be something like class D amps only in the air.  It does NOT work with audible frequencies because they vary too much in length so no stable interference pattern can be established.  

 

Even if you managed this for playback, I am not sure how it would be accurately recorded.  It might be created from nothing for pop type music.  Not sure how you would record it in a venue however. 

 

There has been some work in sound field reconstruction at least back to the 1920's and 1930's.  You end up needing so many channels it is impractical.  

[esldude]

18 minutes ago, STC said:

 

Almost all audiophiles are thinking that’s  possible and regularly claiming hearing so. 

 

Stereo binaural is close enough to 3D but since it comes from headphones it is hard to project the main image in front of you like how you witness them in live performance. 

 

 

Imagine the same sound coming from your two speakers. That’s the first step. The next step is to reproduce the ambiance of the concert hall. The more impulse response the more accurate it is. That’s the second stage. 

So are you saying because many claim it they are managing to recreate soundfields?  

 

I have said before binaural just doesn't work for me.  This one didn't either.  Sounded much better and more dimensional over my speakers actually.  

[esldude]

33 minutes ago, fas42 said:

 

The "in your head" sound with headphones is because the acoustic clues are too indistinct - one's ear/brain can't interpret their true meaning - so the sound "pushes inward". The equivalent with speakers is that the sound is trapped in the cabinets, the imaging doesn't lift out and lay beyond them.

 

The solution in both cases is to improve the reproduction chain; the better resolved acoustic data now is clearly understood by the brain - and the soundfield then stretches beyond the transducers.

If you listen in an anechoic chamber without moving your head, using two speakers, the sound is in your head like with headphones. Has nothing to do with deficiencies in the chain.

[esldude]

1 hour ago, STC said:

 

This cannot be correct unless you are playing a mono sound with two speakers where there is no ITD and ILD.

I think it was in Toole's book or referenced someone else's research.  If the listener is equidistant from speakers in an anechoic chamber and their head is held immobile, they hear imaging mostly inside their head much like with headphones.   If their head is not held immobile apparently very small movements of our head are important.  As the imaging then moves outside their head.  That is one of the advantages of the Smyth Realizer.  Head movement is brought into the sound you hear thru the phones. 

 

I am taking the word of others on this not having the chance to try it out in an anechoic chamber for myself. 

[esldude]

12 hours ago, pkane2001 said:

 

Dennis, so are you saying that for a true 'live' sense and feel of reproduced audio, the listener must be able to move through the sound field and sample this field at slightly different points to get the correct illusion? Similar to holography that looks  2D if you just look at it from a single point, but becomes 3D as you start moving your head around it?

 

If that's the case, would I also get only 2D, not truly 'live' sound impression at a concert hall if I sit with my head perfectly still?

Some info I have read indicates head movement is important for the illusion of space.  So having your head held still or wearing headphones so the soundfield never moves, impacts your ability to hear space and imaging at a distance.  

[esldude]

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0083068

 

https://www.frontiersin.org/articles/10.3389/fnins.2014.00210/full

 

The first link is more directly toward internalization vs head movement. 

 

 

[esldude]

Here is another on head movement and internalization of sound. 

 

http://epubs.surrey.ac.uk/712909/1/2011 Building Acoustics.pdf

 

 

[esldude]

45 minutes ago, STC said:

My ears are not identical but I have no trouble with localization in real world and very good at it. I think it got to do with your early exposure where localization played a crucial role in my early days.

 

However, I have trouble in perceiving sound extending beyond my room walls despite most of the visitors claiming to hear so. Perhaps, my visual clue plays a bigger role.

 

Generally, we could compensate for hearing differences in our left and right ears. There was a recent publication of a research paper that vocal whispered through right ear is perceived differently from left.

Not only that, I read a few years ago where it was found people understood speech better in one ear vs the other using telephones if they had spent years in office work and held the phone to one ear all the time vs the other.  Presumably due to the location of wired desk phones.  

[esldude]

4 hours ago, STC said:

 

I am still reading the first one but I still cannot find reference for two loudspeakers  playing simulataneously. 

I don't think it is in the links I provided.  Those were more to show how under anechoic conditions people have a tendency to hear forward sounds in their head rather than externally if they can't move their head.  Research about such things goes back to at least the 1940s.  

[esldude]

5 minutes ago, fas42 said:

What people who have never heard how powerful the illusion is, as constructed by the brain, from the clues presented from playback, don't appreciate is how rock solid such can be. Movements of the head, movements of the body, walking around the room, to every part of it, does not do the slightest "damage" to the presentation - it's rigidly in place, exactly as if you were listening to a live performance.

Well I don't think anyone believes you for a multitude of reasons. 

 

To change someone's mind you could tell us how we can manage the same trick, what methodology would get us there.  Yet whenever I have asked for this in the end it is a Frank thing only Frank can do with Frank's years of experience. 

 

 

[esldude]

16 minutes ago, gmgraves said:

 

I don't think that's true at all except for Binaural recordings made with an artificial head and embedded microphones in the sides of the head where the ears would be. X-Y, A-B, ORTF, M-S and multi-miked/multi-channel recordings all present the headphone image lined-up from left-to-right through the listener's head to a greater or lesser degree. Binaural recordings will place images outside of the head when listened to on headphones, but even they cannot properly place sound sources behind the head. Try a binaural recording with a artificial head by recording someone walking around the head jingling a set of keys. When the keys pass behind the recording "head" you will notice that on headphones they don't seem to pass behind the listener, but rather they seem to pass in front of the listener! I once did an entire season of binaural recordings of a chamber group and a few well known musicians, including a world renown lute scholar/player for a local radio station. I tried everything I could think of to get the hall ambience to come from behind the listener, I even consulted with Sennheiser's engineers (Sennheiser made the binaural microphone and dummy head), to no avail. Since then I've come to the conclusion that it's the artificial head's lack of an external ear that causes the system to not present binaural images behind the listener. I also think that in order for the trick to work perfectly, the ears on the dummy head have to be replicas of the individual listeners' ears, rather than a generic "ear", and that's simply not feasible. 

One of the links I posted earlier tested binaural recordings.  One of the conclusions of their results is that listeners hear a more accurate recreation when a torso is included on the head.  They found their results didn't vary much with a generic head-sized sphere vs a more accurate one with a nose etc.  (I will note they didn't use outer ears).  The addition of a torso made a large difference in the accuracy of listeners correctly hearing where things were.