What is Immersive Audio?

[STC]

I often come across terms like immersive and 3D sound used to describe the audio system. Unless I missed the development path of what’s now called immersive and 3D audio, we are actually talking about an artificial creation of sound around us and nothing about addressing the spatial hearing of the reproduced sound to sound exactly like hearing them in real life.

 

I am just adding a few charts here and hope readers can share their opinion about my perception of immersive sound reproduction.

 

Fig 1: How sound is perceived in an unamplifed live event. 

 

 

 

 

Fig 2:- The same sound is now being reproduced by stereo.

 

If you compare Fig 1 and 2, you would notice the audiance noise is now shifted to the front and the soundstage and is confined to around 60 degrees. Although, it is possible that the stage width can vary depending on placement speakers and recording mastering but for the purpose of this discussion I am confining to recordings made with standard stereo mics. Depending on the microphones polar patern it is possible the rear sound to be minimized or even eliminated.

 

 

Fig3:- The same stereo recording is now reproduced via heaphones. The separation of the instruments are good but the audio scene is now confined inside the listener's head. This is due to lack of pinna's cues to externalize sound.

 

Fig 4:- This is a typical multichannel playback. The additional 5 channels allow an immersive experience of the same recordings. Such formats requires additional microphones but basically it is capable to replicate the audio scene better than stereo. Notice that the vocal is clear as it is just produced by the center speakers. The recording engineer could channel the piano to SL and the drum to FL. The same with the double bass and trumpet to the other side. Usually, piano would be produced by the front channels as the are big sound. Anyway, the experience of this sound is so much closer to the real event compared to stereo reproduction.

 

 

Fig 5:-  This is the binaural playback of a binaural recording. The original event is replicated close to actual live listening. This example is over simplistic as the soundstage would still be smaller than real life but this is the closest that our current tecnology allows for the most accurate replaction of the live event.

 

Fig 6:- Apple Spatial Audio. Here you do not need binaural recordings as it could feed to pinna cues based on your ear measurements. Further with BRIR, it also allows the images to remain fixed in the orginal location and gives you a sense of direction. The stage is externalized outside your head and closer to the real event.

 

 

Fig 7:- This is Quadraphonics recordings and reproduction.  It is somewhat similar to the current 4.0 concept minus the centre channel. It is too bad that this format did not deserve a place in today's immersive audio format.

 

________________________________________________________________________

 

However, lifelike sound is more than immersive sound. It is about perceiving the sound as how one would hear the real sound. This is truly 3 dimensional. Now let's look at the stereo reproduction over the loudspeakers (Fig 2) and compare the image below.

 

Fig 8:- This presentation is only concerned with the stereo element of audio format. This is what lifelike and natural sound is perceived although here the audiance sound incidence is still reproduced wrongly to the front because that information is in the stereo format and it cannot tell whether the sound is behind or infront of you. But you have the clarity and separation of the instruments more realistic than stereo. For technical papers, refer to BACCH website.

 

 

Fig 9:- Quadraphonics recordings reproduced via front and rear channels without the IAC errors. The same applies to multichannel formats such as Dolby ATMOS, Aura3d, 7.4.2, 5.1 and etc etc where the stereo layer is corrected for realitic 3D sound. This represent the live event more accurately and natural compared to Fig 7 above.

 

 

 

 

 

 

 

 

 

 

[fas42]

For me, immersive audio is creating the sense that I am at the event that was captured in the recording - the conveying of a realness, naturalness, the emotions stimulated by the music itself are all important - I want to feel small compared to the bigness of the projected soundscape; it doesn't have to be "around me" to make that happen. That I don't hear audience noises directly behind me I consider irrelevant - if you stand at the back entrance door of a concert hall to listen, the audience clapping is still in front of you; if you're in the music room, but 'behind' the players, same also applies - in neither case does the quality of connection with the music being played, and the 'size' of the presentation, weaken.

[Archimago]

Nice work @STC. The proliferation of multichannel content now that Apple, Amazon and Tidal are streaming Atmos is a real gift for those of us aiming to achieve a more accurate 3D sound field reproduction for those acoustic recordings (classical mainly) and live concerts.

 

Plus I love how the technology is opening up opportunities with the tools for artists to create more expansive soundscapes for us to enjoy! Wonderful hearing remixes of old favorites and what the new pop, rock, R&B can sound like these days.

 

I suspect over time, this will capture the interest of more audiophiles even if the current "old guard" doesn't quite get it yet and might even resist what IMO is the future of recorded music. Might still be hard to see as audiophile consumers but I suspect this is inevitable at this point on the studio production end.

 

Keep up the good work @The Computer Audiophile for continuing to bring multichannel/Atmos to audiophile attention.

[STC]

Thank you @Archimago . Glad the post is worthy of your attention. 
 

Although I support multichannel sound, I have reservations about the commercialization of music that deviates from the realism of live performances. While the sound can be initially impressive, the artificial placement of instruments around us can feel unnatural to those accustomed to the frontal audio scene of concert hall performances.

[STC]

1 hour ago, fas42 said:

For me, immersive audio is creating the sense that I am at the event that was captured in the recording - the conveying of a realness, naturalness, the emotions stimulated by the music itself are all important - I want to feel small compared to the bigness of the projected soundscape; it doesn't have to be "around me" to make that happen. That I don't hear audience noises directly behind me I consider irrelevant - if you stand at the back entrance door of a concert hall to listen, the audience clapping is still in front of you; if you're in the music room, but 'behind' the players, same also applies - in neither case does the quality of connection with the music being played, and the 'size' of the presentation, weaken.

This could be an early symptoms of Musical Ears Syndrome which is rather a new discovery. Perhaps, entirely a new syndrome. The only time I can relate to your thoughts is when I am drunk where even with one speaker I could see and hear the 100 musicians in my room.  

[El Guapo]

13 hours ago, STC said:

their opinion about my perception of immersive sound reproduction.

Thanks for sharing your opinions. IMHO you’re just representing / explaining the location of the sound (stage, music instruments and audiences). But the value of spatial audio is to reproduce the sound traveling thru and interaction in the space.

Maybe you’ve heard the Eric Kunzel’s 1812 (Telarc SACD 5.1 version). You can hear the cannon’s sound bubble starting from one corner then precisely expands thru your space then disappear in the other corner. The other example one of the 2L’s piano recordings: Home, when playback by properly setup immersive system it can reproduce the sound traveling in the original auditorium space and my friends just shocked. 
Of course the content creators must understand it and use it carefully. Otherwise it will make the creation creation (I mean, too artificial).

[fas42]

11 minutes ago, STC said:

This could be an early symptoms of Musical Ears Syndrome which is rather a new discovery. Perhaps, entirely a new syndrome. The only time I can relate to your thoughts is when I am drunk where even with one speaker I could see and hear the 100 musicians in my room.  

 

Tsk ...

 

In your OP diagrams, the following best represents the presentation by competent playback, as a subjective experience,

 

13 hours ago, STC said:

However, lifelike sound is more than immersive sound. It is about perceiving the sound as how one would hear the real sound. This is truly 3 dimensional. Now let's look at the stereo reproduction over the loudspeakers (Fig 2) and compare the image below.

 

Fig 8:- This presentation is only concerned with the stereo element of audio format. This is what lifelike and natural sound is perceived although here the audiance sound incidence is still reproduced wrongly to the front because that information is in the stereo format and it cannot tell whether the sound is behind or infront of you. But you have the clarity and separation of the instruments more realistic than stereo. For technical papers, refer to BACCH website.

 

 

Yes, the audience sound is 'wrong'; but it remains in a separate layer from that of the musicians; which the above figure somewhat implies. Also, the "100 musicians" are not in the room, but on a stage as big as the captured acoustic 'measures'; that stage begins at a vertical plane through the speakers and extends back 10s of metres from there, as defined by recording setup and mastering.

 

The size of the listener as compared to everything else, and positioning, is spot on, in fig. 8 - as good as anything else I've seen that indicates the landscape, for convincing replay ...

[STC]

17 minutes ago, El Guapo said:

You can hear the cannon’s sound bubble starting from one corner then precisely expands thru your space then disappear in the other corner.

That’s what object audio supposed to do. It doesn’t matter if you actually hear the same explosion and disappearance of the canon sound in a corner in a real concert hall. Here the effect creates a different realm and keeps us hooked. In a way, sound reinforcement adds new perspective to the music. Meyer constellation system is now capable of better any concert hall and it is being used in concert hall already. 

[fas42]

This video, by someone one or two of you may have heard of , explains how it hangs together,

 

 

Immersiveness is controlled by the volume you set the system to play at, and unlike the limits of the illusion as Paul describes - when  equipment is not working well enough - the presentation with adequate SQ corresponds to what he calls at some point in the video, an "ideal" playback scenario ...

[STC]

43 minutes ago, fas42 said:

Yes, the audience sound is 'wrong'; but it remains in a separate layer from that of the musicians; which the above figure somewhat implies.

This is very basic drawing where I didn’t even discuss about reverberation or head shadow.  The very fact that the rear sound got mixed in front shows there is no distinguishable layers. Worst still if the person claps from the position of cone of confusion it could be difficult to localize whether it is back or front since claps can vary in frequencies spectrum depending on how the individual is clapping. 
 

One of the fun demo I do at home is to play the rear speakers where the stage falls within cone of confusion. So far, I am yet to find one listeners to tune back and look for the sound. Their expectation is frontal stage and it projects the sound so. 
 

Of the all the best trickery of mind I experienced was when I was playing “ The Transformer” movie where the intro scenes where the spaceship would fly overhead. I experience the best effect of the ship traveling to the back but when the dialogue came in it was rather low and I kept increasing the level but nothing changed. Only then I realized that my front speakers weren’t even turned on.There was no ATMOS and I was just playing stereo with 4 speakers XTC. At this time, it was just the rear speakers were playing and I forgot to turn on the front speakers amplifier. 
 

You can easily fooled by what you hear. Your brain fills in and reconstruct the sound based on visuals. You hear position based on expectation and experience. 

[STC]

1 hour ago, fas42 said:

This video, by someone one or two of you may have heard of , explains how it hangs together,

 

 

Immersiveness is controlled by the volume you set the system to play at, and unlike the limits of the illusion as Paul describes - when  equipment is not working well enough - the presentation with adequate SQ corresponds to what he calls at some point in the video, an "ideal" playback scenario ...

I would rather discuss his other video, I have the same mics but I do not have the same tracks so I didn’t bother to pursue the matter further. I also had an issue about the authenticity of another video recorded by the same person where the inexplicable mouse clicks were audible and too good stereo separation for a stereo playback acoustic recording.

 

If you or someone could give a short excerpt of the clips he used in this video I would able to investigate further. 
 

 

[fas42]

37 minutes ago, STC said:

You can easily fooled by what you hear. Your brain fills in and reconstruct the sound based on visuals. You hear position based on expectation and experience. 

 

Yes. That's exactly what happens. And a well done system exploits that, in very positive ways.

 

17 minutes ago, STC said:

 

Good choice to bring in ... the first track has some lacking in the treble, as shown by the quality of the hand-claps; but the choral and pipe organ pieces are top notch - acoustics of the recording spaces are well presented, and I hear no issues with clarity and detail in the reproduction.

 

"short experts" - you mean, excerpts?

[STC]

6 minutes ago, fas42 said:

short experts" - you mean, excerpts?

[JoeWhip]

Even in a live acoustic performance you are hearing sound reflections from behind and to the sides of you. The sound also travels over you. Immersive can capture that and present it to you just like being there. The Home recording referenced above is a perfect example of that. Just a piano in a reverberate space. Sounds more lifelike than 2 channel could ever render.

[STC]

10 minutes ago, JoeWhip said:

Even in a live acoustic performance you are hearing sound reflections from behind and to the sides of you. The sound also travels over you. Immersive can capture that and present it to you just like being there. The Home recording referenced above is a perfect example of that. Just a piano in a reverberate space. Sounds more lifelike than 2 channel could ever render.

IIRC, from the fifth row you are already hearing 90% of reverbs vs 10% of the direct sound. Recordings are never made from the listeners perspective. As far as musical contents are concern the additional channels produces the reverbs. The more the better. Having said that, for commercial reasons, we are having more recordings that intentionally placing sound around the listener. That have their own followers. It is more engaging nevertheless. 

[ARQuint]

22 hours ago, Archimago said:

 

I suspect over time, this will capture the interest of more audiophiles even if the current "old guard" doesn't quite get it yet and might even resist what IMO is the future of recorded music. Might still be hard to see as audiophile consumers but I suspect this is inevitable at this point on the studio production end.

 

Keep up the good work @The Computer Audiophile for continuing to bring multichannel/Atmos to audiophile attention.

 

If, Arch, by "old guard", you mean the two print magazines you love to dump on, you've missed a lot of page-inches. Kal Rubinson, in Stereophile, and I, in The Absolute Sound, have been writing extensively about multichannel gear and recordings for more than 20 years, long before Chris developed his current (well-placed) enthusiasms. There was also this guy who wrote mostly for S but also for TAS at the end of his career (maybe you've heard of him—Gordon someone-or-other) who was championing earlier surround formats long before that. There were others. But if you start with a dismissive view of certain publications (not to mention civilian  audiophiles who have been around for a while) you'll tailor the facts to fit your sneering contempt. Not unlike Donald Trump and his references to "the failing New York Times."

When you're operating in your usual scientific work environment and an accurate representation of what's been said or written before matters, I'm pretty sure your review of earlier literature is more rigorous and thoughtful than it was this time.

Best

Andy Quint

[STC]

1 hour ago, ARQuint said:

If, Arch, by "old guard", you mean the two print magazines you love to dump on, you've missed a lot of page-inches. Kal Rubinson, in Stereophile, and I, in The Absolute Sound, have been writing extensively about multichannel gear and recordings for more than 20 years, long before Chris developed his current (well-placed) enthusiasms.

 

Certainly this is progress, the founder of the magazine used to say “ it was at odds with human biology. “We have two ears,” . “We don’t have five ears or 25.” 

[ARQuint]

29 minutes ago, STC said:

Certainly this is progress, the founder of the magazine used to say “ it was at odds with human biology. “We have two ears,” . “We don’t have five ears or 25.” 

 

Harry Pearson? He ended up a big fan of multichannel music! He had a room set up for film but tweaked the system so it was optimized for music—I listened to SACDs with him many times.

 

The execution of MC recordings did improve over time (and continues to improve.) Harry would always acknowledge that when it happened, as he did with digital audio in general.  People can learn and change their minds.

[STC]

30 minutes ago, ARQuint said:

Harry Pearson?

 Yes.

 

———-

 

We all make mistakes. However, the notion that two speakers are sufficient for high-fidelity sound is a common misconception that has been perpetuated by some audio ‘experts’. This belief is based on the idea that we only have two ears, and therefore, two speakers are all we need. However, this is a fallacy. How did he even reach that conclusion? Audiophiles who are reluctant to venture beyond stereo setups, cables, DACs or amplifiers may be influenced by these false narratives created at the very beginning of high-fidelity audio. While it’s true that some people may prefer the simplicity of a stereo setup, it’s important to recognize that there are other options available that can provide a more immersive listening experience but the advocates should have kept open mind about it and not to douse the enthusiasm just because it is beyond their understanding.

 

They were against digital, MC, XTC, digital amp. Of course, now they are embracing them and blame that the early modules lacked the finesse of current products. Is it or just an excuse for their earlier false narratives?

 

[ARQuint]

45 minutes ago, STC said:

 Yes.

 

———-

 

We all make mistakes. However, the notion that two speakers are sufficient for high-fidelity sound is a common misconception that has been perpetuated by some audio ‘experts’. This belief is based on the idea that we only have two ears, and therefore, two speakers are all we need. However, this is a fallacy. How did he even reach that conclusion? Audiophiles who are reluctant to venture beyond stereo setups, cables, DACs or amplifiers may be influenced by these false narratives created at the very beginning of high-fidelity audio. While it’s true that some people may prefer the simplicity of a stereo setup, it’s important to recognize that there are other options available that can provide a more immersive listening experience but the advocates should have kept open mind about it and not to douse the enthusiasm just because it is beyond their understanding.

 

They were against digital, MC, XTC, digital amp. Of course, now they are embracing them and blame that the early modules lacked the finesse of current products. Is it or just an excuse for their earlier false narratives?

 

You are absolutely right. There was and there remains a great deal of resistance to multichannel/immersive audio. I'm just saying that the blame for this shouldn't be placed at the feet of TAS and Stereophile. I don't think it's surprising that the pushback came/comes largely from some consumers who have acheived good sound with a stereo setup. Reviewers have fewer constraints—financial, esthetic, time requirements—to experimenting. When someone who has invested considerable capital (of various kinds) into optimizing the performance of a two-channel is told that a better (or at least different) experience is possible after all the trouble they've gone to, they'll come up with the various BS arguments we've all heard: "I don't have five ears," "I don't want to hear clarinets over my left shoulder" and so on. With the latest progression from discreet MC to object-based technologies, I think we're starting to see chinks in the armor of even the most entrenched partisans.

 

I do feel that it's important to point out that the end objective is improved spatiality with recorded music, not the triumph of one technology or another. Specifically, I'm referring to crosstalk cancellation, as offer by the BACCH filter of Edgar Choueiri. That approach does provide a kind a spatialization that's not as immersive and dynamic (in the sense of movement) as discreet multichannel or Dolby Atmos but is extremely good at specifically localizing musical sounds in a soundfield that's largely in front of the listener. And it requires just two speakers. Watch for an article in an upcoming issue of TAS by Dr. Choueiri (co-authored with Laurie Anderson) that parses the different kinds of spatial information. So, I have a 5.1 system that soon will have Dolby Atmos capabilities (a big part of the impetus to proceed was hearing Joe Whip's excellent system) and I have a BACCH-SP that I use with loudspeakers having restricted lateral dispersion characteristics to maximize XTC. As the saying goes: "That's what makes it a horserace."

 

[STC]

2 hours ago, ARQuint said:

You are absolutely right. There was and there remains a great deal of resistance to multichannel/immersive audio. I'm just saying that the blame for this shouldn't be placed at the feet of TAS and Stereophile. I don't think it's surprising that the pushback came/comes largely from some consumers who have acheived good sound with a stereo setup. Reviewers have fewer constraints—financial, esthetic, time requirements—to experimenting. When someone who has invested considerable capital (of various kinds) into optimizing the performance of a two-channel is told that a better (or at least different) experience is possible after all the trouble they've gone to, they'll come up with the various BS arguments we've all heard: "I don't have five ears," "I don't want to hear clarinets over my left shoulder" and so on. With the latest progression from discreet MC to object-based technologies, I think we're starting to see chinks in the armor of even the most entrenched partisans.

 

Fair enough. 
 

2 hours ago, ARQuint said:

do feel that it's important to point out that the end objective is improved spatiality with recorded music, not the triumph of one technology or another. Specifically, I'm referring to crosstalk cancellation, as offer by the BACCH filter of Edgar Choueiri. That approach does provide a kind a spatialization that's not as immersive and dynamic (in the sense of movement) as discreet multichannel or Dolby Atmos but is extremely good at specifically localizing musical sounds in a soundfield that's largely in front of the listener. And it requires just two speakers. Watch for an article in an upcoming issue of TAS by Dr. Choueiri (co-authored with Laurie Anderson) that parses the different kinds of spatial information. So, I have a 5.1 system that soon will have Dolby Atmos capabilities (a big part of the impetus to proceed was hearing Joe Whip's excellent system) and I have a BACCH-SP that I use with loudspeakers having restricted lateral dispersion characteristics to maximize XTC. As the saying goes: "That's what makes it a horserace."

I have been utilizing the same concept as BACCH for over 15 years, where Choueiri was a co-contributor. I’m not certain why you anticipated BACCH to generate spatiality akin to DolbyATMOS, as that is not the objective of crosstalk cancellation.

 

Although BACCH was initially developed with speakers positioned in close proximity, I am eager to ascertain if it can outperform the well established 20 degrees. Carver achieved a similar feat with his Holographic Amp although not as good as current digital XTC and certainly nowhere holographic compared to current XTC . TaCT and Minidsp already made them years ago but didn’t get due recognition from the magazines. Polk SDA is another one but unlike BACCH or Ambiophonics it is not made using the listeners HRTF so the effect will be minimal.

 

However, it is still impossible to attain side and rear reflections with BACCH or any XTC. You need additional speakers and impulse response. And a powerful computer. 
 

If you happen to meet Choueiri, inquire how he managed to mitigate the influence of direction-finding pinna to prevent speaker localization at 30 degrees. I doubt if BACCH can accomplish this within a 60-degree as well as 20 degrees as demonstrated in his Princeton 3Daudio website. My profile signature below  is about crosstalk cancellation.

[ARQuint]

1 hour ago, STC said:

 

Fair enough. 
 

I have been utilizing the same concept as BACCH for over 15 years, where Choueiri was a co-contributor. I’m not certain why you anticipated BACCH to generate spatiality akin to DolbyATMOS, as that is not the objective of crosstalk cancellation.

 

Although BACCH was initially developed with speakers positioned in close proximity, I am eager to ascertain if it can outperform the well established 20 degrees. Carver achieved a similar feat with his Holographic Amp although not as good as current digital XTC and certainly nowhere holographic compared to current XTC . TaCT and Minidsp already made them years ago but didn’t get due recognition from the magazines. Polk SDA is another one but unlike BACCH or Ambiophonics it is not made using the listeners HRTF so the effect will be minimal.

 

However, it is still impossible to attain side and rear reflections with BACCH or any XTC. You need additional speakers and impulse response. And a powerful computer. 
 

If you happen to meet Choueiri, inquire how he managed to mitigate the influence of direction-finding pinna to prevent speaker localization at 30 degrees. I doubt if BACCH can accomplish this within a 60-degree as well as 20 degrees as demonstrated in his Princeton 3Daudio website. My profile signature below  is about crosstalk cancellation.

 

Correct, and I didn't expect BACCH to generate the kind of 360 degree envelopment that discreet MC and Dolby Atmos create routinely. In my space, with directional loudspeakers and minimal room treatment, with BACCH there's image placement at 9 o'clock and 3 o'clock but only very rarely further back. EC would like me to position myself more near-field but I've declined; IMO, that's the tail wagging the dog. Nonetheless, in my experience, the subtle layering of images in front of the listening position—say, flutes/oboes in the front row of woodwinds and clarinets/bassoons in a row behind them—is superior to what I get with 5.1. DA is a work in progress for me and I'm eager to hear how it fares with this aspect of spatiality.

[STC]

1 hour ago, ARQuint said:

Correct, and I didn't expect BACCH to generate the kind of 360 degree envelopment that discreet MC and Dolby Atmos create routinely.

Even DC or MC couldn’t. You need impulse response of the venue to create the realistic envelopment of the ambience. 
 

Aura3D with more channels is definitely more realistic than 5.1. DC with the height speakers you are adding monaural sound so not much benefit there but the SBL and SBR and the FL and FR can benefit with a very 3D sound. I am not sure if the BACCH allows to do that as it can be easily done by routing any stereo layers through the XTC process. 
 

A correctly done XTC should create a 3D image approximately 180 degrees stage with images floating and distinctly separate. It is always a compromise when the cancellation is precise the sweet spot gets smaller but still allows the normal flexibility of movement as stereo listening. 
 

If I were you, I would focus to implement BACCH to DC or other multi channel, that’s where it is going to make huge difference with immersive sound for orchestra music. For other types of music, DC is probably more exciting like how stereo impressed most hearing the train moving from right to left the first time. 😊
 

 

[ARQuint]

31 minutes ago, STC said:

 

If I were you, I would focus to implement BACCH to DC or other multi channel, that’s where it is going to make huge difference with immersive sound for orchestra music. For other types of music, DC is probably more exciting like how stereo impressed most hearing the train moving from right to left the first time. 😊
 

 

 

That would be wonderful and I'm sure the possibility interests Dr. Choueiri—he loves challenges. As things stand now, I indeed favor BACCH XTC for classical music (unless it's the Berlioz Requiem and there are supposed to be brass bands behind the listening position...) and 5.1 for successful pop mixes. I'll be interested to see how the implementation of DA affects this equation.

 

Thanks very much for your obviously well-informed observations and suggestions.

 

Andy

[ARQuint]

28 minutes ago, STC said:

A correctly done XTC should create a 3D image approximately 180 degrees stage with images floating and distinctly separate. It is always a compromise when the cancellation is precise the sweet spot gets smaller but still allows the normal flexibility of movement as stereo listening. 
 

 

Which is what I'm getting (9 AM and 3 PM) - and sometimes with non-dummy head recordings.