Article: Thoughts On Immersive Audio

[Geoffrey Armstrong]

Thanks Chris for this great series of articles on “Immersive”. 

 

You’ve probably already thought of this; but another useful article could be distinguishing “Immersive” from traditional MCH audio. There’s obviously a tendency for us to lump the two together, as they are in many respects, very similar.

 

As I understand it, the main differences are…

 

1/ Traditional Multichannel is channel based; meaning there’s a hard limit as to the number of channels on any given MCH recording. 8 channels (or 7.1) seems to be the limit. If the number of channels on an MCH recording exceeds the channels in your system, you’ll need to down-mix. For example from 7.1 to 5.1. Though most available MCH recordings are 5.1 (with some at 5.0, 4.1?, 4.0, 3.1?, 3.0, or 2.1, until we’re finally back to good ol’ 2 channel stereo).  Where 7.1 traditional MCH exists, it is designated as, Left front, Right front, Center, Sub, Left rear, Right rear, Left side, Right side. There is no concept of a height channel or channels.

 

2/ “Immersive” is “object” based. So there is no hard limit, as far as the recording is concerned. I suppose, theoretically at least, every instrument/voice on a recording could be its own “object”. This means that an “Immersive” recording is more likely to require down-mixing, or rather allocation/distribution of the “objects” amongst the physical channels in a given system. There is, however, a practical limit as to how many physical channels are likely to be available in a given speaker based system. Perhaps this will converge on 12, in a similar sense that for traditional multi-channel, 5.1 is the most typical.

 

3/ The main contribution that “Immersive” brings to the table, appears to be the addition of one or more height channels. So the question that always comes to my mind, is what advantage do height channels bring to music? I believe you already addressed this to some degree in previous articles. It does also seem that this will depend on the genre of music and where it was recorded. Classical music recorded in a traditional concert hall built, with that music in mind, will generate multiple reflections from all directions within the hall, which will be experienced differently, depending on where the listener is seated. Other genres of music, with far fewer instruments and voices, recorded in a studio also involve reflections, though these can, presumably be more easily controlled. It also becomes a question of which perspective on the music the listener will experience. Do you want to be in the middle of the band or have the band perform in front of you?; etc. Artificial spaces can also be eletronically generated with effects, and these become part of the artistic creation. Perhaps we are entering a time though, where there’ll be greater audience or listener participation in the final result? Two channel recordings, of course, treat all genres of music equally, because they are equally limited.

 

I think it’s useful to distinguish clearly between “Immersive” and traditional multi-channel. If we lump the two together, it can be argued that Qobuz already support immersive, because they have been streaming traditional, full quality 5.1 multi-channel, for some years now. Most of these titles are classical/opera and most are at 96/24 or 192/24 for all 6 channels!

 

4/ Obviously “Immersive” in the form of Dolby Atmos and similar, object based formats can be experienced on headphones (any headphones actually). Whereas with traditional Multichannel recordings, headphone listeners need not apply. For headphone listening, the result is perhaps similar to the old Binaural/Dummy head recordings, which have been around since at least the 70s. I suppose with headphones no down-mixing or allocation of objects to different physical channels is performed, even though there are only two channels available. I’m guessing this is because the room is taken out of the equation, so the “spatial” illusion can be re-created to a degree, because of the proximity of the transducers to our ears. I must admit though, my understanding of this is shaky.  I’ve even wondered if this couldn’t be experienced to some degree with nearfield speaker listening, whereby a pair of speakers could act like giant headphones.

 

5/ Where you have full quality Dolby Atmos recordings available (currently only on some blu-ray discs), you can rip then decode them using the Dolby Reference Player, to the number of physical channels your speaker system has. If you have enough physical channels, these could include at least one height channel. If not, the result of your decoding would be the same as a traditional Multi-channel recording, for example at 5.1, if that’s the number of channels you have; except the sample rate may be limited to 48khz for each channel, as necessary to squeeze all those additional channels onto the disc. In that case an actual traditional MCH recording would, presumably be better, if each channel were >= 96khz.

 

The decoded Dolby Atmos from a Blu-Ray results in regular 48khz/24 bit PCM per channel, which could have convolution performed on the channels, for example via HQPlayer playback.

 

A spatial stream from Apple Music can be decoded by MacOS in real-time and fed into HQPlayer’s input mode.

 

To conclude, I’m sure the results of, particularly full quality Dolby Atmos, on a system, such as the one you’ve built, can be truly spectacular. This seems even more dependant on the job done by the mixing engineer and the genre of music, than with stereo recordings though, and with legacy recordings, risks moving away from the original artsts intentions.

 

I’ve realised I needed to type this out to confirm my own understanding. So please correct me wherever I’ve made any mistakes and keep the great articles coming, if you can tear yourself away from listening!

[Geoffrey Armstrong]

On 7/2/2022 at 9:37 PM, Kal Rubinson said:

All that is true for mainstream multichannel but several labels have, over the years introduced height channels.  I am thinking of Telarc, Chesky and MDG ("2+2+2"0.

Agreed except that the Qobuz app does not support any multichannel at this time and it is necessary to utilize Roon (are there others?) in order to hear those tracks in more than stereo.

Not if you have a Smyth Realiser on hand.

I think you did a great job.

Thanks very much Kal.

 

I wasn’t aware that labels such as the ones you mentioned added height channels to some of their MCH releases.

 

I have also used Audirvana to playback multichannel streams from Qobuz.

 

I also hadn’t heard of the Smyth Realiser. Will check it out.

[Geoffrey Armstrong]

On 7/2/2022 at 10:59 PM, The Computer Audiophile said:

Hi @Geoffrey Armstrong Great post. I’ll do my best to reply from the beach in Northern Minnesota (Independence Day holiday :~)). 
 

Immersive can’t be distinguished from traditional multichannel because it’s object based. The two immersive music formats are Atmos and Auro 3D. Atmos is of course object based, but Auro 3D is channel based. Nobody is doing DTS immersive music, based on what my contacts who mix and master this stuff say. 
 

Immersive to me involves layers beyond ear level, namely height channels. This is where the immersive comes from, being immersed in audio from all angles. 
 

Immersive seems to be settling in 7.1.4 12 channel systems, but others are definitely supported. 
 

I plan many more articles about all of this as you noted. Immersive brings many things to the table for both consumers and creators. Creators are now free from technology dictating what they do. They now have a 3D space in which to play music. Reproducing this as a consumer is great for all different types of music. Sure, a classical concert and recording all now expands to the ceiling. This is fantastic and an obvious use of the format. Others are using it more actively by placing audio everywhere initially. This means we can hear the reverb of sounds originating behind us, in the front channels. I have much more to say about this, and it may surprise people :~)

 

I think mixing engineers will finally get their due rather than mastering engineers getting all the press. Mixing in immersive makes or breaks a recording. 
 

Back to the water for me for now. I’m still digesting your full post. Great stuff. Keep it coming!

I attempted to respond from Marineland near Antibes, where we took our nine year old daughter over the weekend, and got up close and personal with some Dolphins 😀

 

Unfortunately my responses didn't go through, due to a poor internet connection.

 

So the differences between traditional multi-channel and "immersive" are not so clear cut as I'd thought. So I'll just continue to refer to the first as "traditional Multi-Channel" and the second as "Immersive".

 

I'm sure your new "Immersive" system also makes a superb "Traditional Multi-Channel" system and any comparisons between the two, you can make on your system, will be interesting to learn about.

 

I'm looking forward to your further articles.

[Geoffrey Armstrong]

A quick question; is it possible to decode Spatial audio streams from Apple's music app to 6 Channels, using BlackHole 16Ch version, and feed the result to HQPlayer with BackHole 16Channel, set as the input?

 

Apologies in advance if I misunderstood previous discussion on this in these articles; but I got the impression, it should be possible to decode these Spatial streams "on the fly" and feed them into HQPlayer to play though my traditional 5.1 MCH system. All I get though is two channel stereo passed through.

 

Thanks

[Geoffrey Armstrong]

Thanks Chris. That would certainly widen our choices of Immersive/MCH we could play this way. Although we’d obviously prefer it to be lossless.

 

I’m wondering if MacOS needs to “see” an AVReceiver connected via HDMI, before it will decode all 6+ channels?

[Geoffrey Armstrong]

2 hours ago, wisechoice said:

Creating an aggregate device is not necessary if the output is Blackhole 16ch, and shouldn't be necessary if the hardware output device already has sufficient outputs. However, I've noticed that getting more than 6 channels is spotty when outputting directly to at least some interfaces (in my case the Digiface USB), whereas Blackhole 16ch as the intermediary works just fine.

Thanks so much to both Chris and wisechoice. Yes, I confirm it's now working. The references to "aggregate devices", Anubis, VAD; etc, etc. all went over my head a bit as I'm not using Merging devices or Ravenna.

 

I don't know where exactly I was going wrong, as what I had tried appears to be in line with your advice. I have an old Exasound E28, connected to my Mac via USB. I set this as ATMOS 5.1.2 Surround device in Audio Midi Setup, set the default output device as BlackHole 16CH, set that as the input to HQPlayer, which is playing out to the Exasound set as an 8 channel device. Finally I chose the audio:default/48000/2 output option; but then edited the "/2" to replace it with "/6".

 

I'm doing this on my M1 Mac Air. Previously I was trying it on my Intel 2018 Mac Mini. So I don't know if the Intel Mac Mini (also running Monterey) is capable of decoding Atmos, or if it needs to be an M1. I also can't remember if I chose Atmos as "Automatic" (the correct option) in Apple Music app or "Always", which would have been wrong.

 

I also remember I chose standard 5.1 Surround in Audio Mini set-up rather than the Atmos option.

 

Will check tonight or tomorrow the Intel Mac.

 

Anyway, I'm now looking forward to experiencing all that Spatial content decoded to my 5.1 system.

 

👍👍👍

[Geoffrey Armstrong]

I have been spending the last week listening to Immersive/Multi-Channel sources through my traditional 5.1 MCH system and here are my impressions so far.

 

First off the MCH part of the system is a kind of “icing on the cake” add on to my two channel system. I put it together with both music and Home Theatre in mind; but have only listened occasionally to multi-channel music and movies through it.

 

It is by no means a dedicated purpose built Immersive system, such as Chris’s. Mine is a hybrid, consisting of a pair of Avantgarde Trio horns with a pair of the same company’s Short Basshorns. The Multi-channel part consists of one Magnepan center channel and two Magnepan rear channel speakers. The latter rest flat against the wall when not in use, and are swung out to be perpendicular to the wall for listening. So the Multi-Channel part is quite tidy and discrete.

 

I figured that, at least the Magnepan have a speed and transparency which can approach the Trios in that regard, even though they can’t match the Trios on dynamics (perhaps less important, at least for the rears).

 

IMO it actually does work quite well as a way of augmenting the main Trio system.

 

The Magnepan centre channel speaker is placed much higher than normal for a centre speaker. The reason for that is, I use a front projection system with a JVC projector beaming to a Stuart Filmscreen electric tensioned screen. Immediately behind the screen are full length windows and behind those on the outside are electric shutters. So the room can be completely darkened for watching movies. I couldn’t place the center channel either below or behind the screen so it sits way up, above the screen in an especially constructed false wall, with a cut out for the speaker. The speaker is angled down slightly towards the listening position.

 

In this room with 3 meter high ceilings, the sound from the Trio system alone has exaggerated height. I know it’s not exactly correct; but I enjoy it. Think of it as a kind of guilty indulgence. I figure, if I were to sit fairly close to a stage, which I like to do, the artists would be performing on a stage raised above me. So in that sense it does kind of correlate to real life.

 

The exaggerated height presentation of the Trio system alone in this room, often places the main voices and instruments in a position, which coincides quite closely, with the mute centre channel speaker. So to have those voices emanating from that position when the centre channel is playing, doesn’t seem out of place to me.

 

What is crucial though is the correct channel blending for MCH sources. By that I mean, the relative gain of the centre speaker compared to the main LF, RF speakers. If the center speaker is too strong compared to the main speakers, those voices will sound disembodied and coming from the rafters. Either lowering the gain on the center speaker or raising the volume on the main speakers, usually allows me to achieve a balance between the center and mains that satisfies me.

 

The same can be said of the main speakers in relation to the two rear speakers. Here also, the blend of the two rears with the two front mains is crucial to my enjoyment of Immersive/MCH sources. With the right blend on the right sources, the results can be very convincing indeed. I believe, more convincing than can be achieved with a two channel system alone. Still I have my reservations. At least on this compromised MCH system.

 

Anyway, Chris’s articles on the subject have inspired me to dust off my old MCH components and give this system more of a serious listen.The addition of Apple’s Spatial/Atmos streams have provided further incentive.

 

To complete the system description, for the MCH system all channels are fed by an old Exasound E28, which has 8 channel outputs. The two main channels are driven by Class D NCore mono-blocks, the center channel by a Class D Mono amp from Nuforce, providing 160w, and the two rear channels are driven by an integrated amp from Crayon audio, providing 190wpc.

 

So I can vary the volume of the rear speakers with the integrated amp’s volume, as well as apply channel gain adjustments in software.

 

The center channel can only be controlled with the Exasound’s volume slider and other software gain settings (in my case using HQPlayer).

 

I’ve played with this system on and off over the years, and already discovered the settings that work best for me when it comes to channel blending for rippped MCH SACDs and DVD-As. There is, of course, quite a lot of variation from recording to recording. As there is with two channels sources. Though, obviously the situation becomes far more complicated with Immersive/MCH.

 

When it comes to the sound, what I find myself enjoying the most is a more subtle multi-channel mix. By that what I mean is a mix which doesn’t place particular intruments/voices exclusively in the rear channels.

 

We all know the rear’s can be used effectively for audience sounds in a live concert and to capture the natural reverberance of instruments being played on the main stage, as the sounds bounce off the halls rear and side walls.

 

What I enjoy most, I believe, is not exactly or not just, this reverb or echo off the walls; but rather when the sound of an instrument appears to be shared between the mains (or one of the mains) and the corresponding rear speaker(s), in a way that causes the sound of that instrument to reach our further into the room and closer to me. This will vary naturally, depending obviously, on how loud the instrument is being played at any given time, and by movements of the musicians and their instruments.

 

By “channel blending”, I can hit on what, for me, is the most convincing presentation. Too strong from the rear and I might just increase the volume from the mains a little (the easiest for me to do), or the opposite when the rears seem barely audible.

 

If the mixing engineer has exclusively placed an instrument in one of the rear channels though, nothing can be done! This just never sounds natural to me. It’s always an effect; an effect that can be fun, depending on the music; but I’d rather it was used sparingly, if at all.

 

Perhaps the problem is one of perception. If someone has splashed out on an Immersive/MCH system, they want to hear those rears, otherwise what’s the point. I don’t want to hear them as separate sources though.

 

Just as us two channel guys want our speakers to “disappear” as the sources of the sound, so it should be for the rear channels IMO.

 

The situation with the center channel is similar; but a little different, in my view. Here it makes perfect sense for a main instrument/voice to be prominently placed in the center channel. Again, though this has to “blend” nicely with the mains, so that some of that voice/instrument is also being reproduced by the mains. I would say that the center channel are to the mains, what the mains should be to the rear speakers. Although of course, the center channel may also have this relationship with the rears. It’s all about natural sound propagation into the room. So, again, the instruments should (at times anyway) appear to reach out towards you/jump out at you, as they do in a live venue.

 

A really great two channel system might be able to achieve the same thing. From listening over the last week though, I’m coming around to the view, that a two channel system would struggle to achieve this as well as a decent MCH system (even one as compromised as mine). Reviews are full of descriptions of two channel gear that can convey the three dimensionality of instruments. With an Immersive/MCH system, depending on the mix, this effect of instruments reaching towards you, must have a dimensional aspect. They gain in dimensions, I’m sure, at the same time as they reach out into the room. Again though, as I’ve tried to describe, this is all varying with time and the actions of the musicians. Too often, descriptions in reviews discussing the “dimensional” aspect, give the impression of an object fixed in space with fixed dimensions. Not something which is constantly varying with time; etc. Perhaps this apparent fixedness of the dimensions comes from the limitations of two channel systems?

 

Whether an Immersive/MCH system is correctly propogating the intruments into your room, will be very much dependant on the decisions of the mixing engineer.

 

I’m certainly no musical snob; but I would say, in general, the more serious the music is considered to be, the more the engineers appear to take the approach of natural sound propogation. The less serious the music may be considered, the more liberties the engineers might be tempted to make with creative use of the channels.

 

Most of the ripped SACDs I have are of classical and jazz, and with most of these a pretty impressive job has been done in the mixing stage in these respects.

 

Over the last week though, I’ve been mainly listening to Apple’s Atmos Spatial streams. So I will take some examples from those.

 

Apple has “Spatial” playlists, each one given a distinct colour for a particular genre of music.

 

As I’m a big jazz fan, I’ll start with an example from the Jazz Spatial list:

 

Tanya from Dexter Gordon’s One Flight Up, sounds to me like a great example of how it should be done. At times the horns reach out towards me in the way I’ve described, and occasionally a thwack on the drum or a rimshot can bounce off the back wall. It’s a good one for me in getting the balance between the rears and mains right, and for balancing the centre with the mains.

 

Neither the center or the rears call too much attention to themselves. If the rears were too loud relative to the mains, the rears appeared to dominate the sound field. If too low, the rears were barely audible and not contributing enough to that natural sound propagation. Similar comments apply to blending the center channel.

 

For me this track then, is an example of how it should be done. There are many others on that Jazz list, probably from the same period that I could say the same thing about though.

 

Like Someone in Love from Diana Krall’s Turn up the Quiet:

 

Unfortunately for me, this is an example of how not to do it. There is a guitar that appears to emanate solely from the rear left speaker. Why? Is the idea to put me in the middle of the band? If so it doesn’t feel that way, as the rest of the instruments and Diana’s voice all appeared where you’d expect them to be. Perhaps it’s a limitation of my system; but I just don’t get it.

 

I am happy with most of the examples on Apple’s Classical Spatial list, except for one.

 

Hymne à l'amour (Orch. Ducros), from Gautier Capuçon’s album Emotions.

 

There were loud cracking sounds from my rear speakers causing me to fear for them. So I turned the volume way down to barely audible; but I could still detect these “cracking sounds”. It could be something wrong with my set up, coupled with this having some quite dynamic content. I’ve played plenty of other Spatial streams of all genres of music, including classical orchestras at realistic levels though, without encountering this problem. So I suspect there’s something wrong with this recording/stream. A pity as otherwise it appears to have been done quite well, and I like the music.

 

For Apple Music’s Spatial audio, I have used the Dobly Atmos Surround  5.1.2 output in Audio Midi set-up, to the Black Hole 16 channel virtual audio device. This is then fed into HQPlayer’s input and sent out to the Exasound as an 8 channels device in HQPlayer.

 

I have nothing to say about height channels, as my system is 5.1 only. The last two channels (7,8) are designated as “Left Top Middle” and “Right Top middle”.

 

I imagine when the height channels are present in a system, they would further add to that natural sound propagation, as it occurs in all directions and it might further add to the dynamic three dimensional sound I’ve tried to described. Since HQPlayer has its very useful Matrix Pipeline feature though, I thought I’d make use of it to blend channels 7 and 8 into the centre channel (channel 3). Since this is blending two channels into one channel (which is also receiving its own output) I reduced the blend of each channel 7 and 8 to -24db. I have absolutely no idea if I’m gaining anything from this. Since my Center channel is up on high though, I thought, what the heck, why not? At some stage I might mute the center channels actual output, and just listen for anything I might be getting from channels 7 and 8.

 

Finally, after making the search again for “Spatial” in the Music app I scrolled down beyond Apple’s own curated lists and found the list “Gramophon’s Choice” .

 

These are some truly classic performances, many taken from Deutch Gramophon and, so far, they all sound done right to me. After my all so subtle approach, though, I found I can listen to these with the rear channel volumes higher in relation to the mains than previously, and very much enjoying the result. The rears are more prominent; but in terms of room filling sound, as you would experience in a hall they are sounding quite natural to me. Highly recommended!

 

 

 

 

[Geoffrey Armstrong]

9 hours ago, The Computer Audiophile said:

Hi @Geoffrey Armstrong thanks for the detailed post. Deutsche Grammophon has some amazing Atmos releases for sure! I'd say it's the leading label for this stuff. I just wish it would release more on Blu-ray Disc for the lossless TrueHD Atmos.

 

I'm going to publish an article tomorrow in which you can see how I'm using convolution filters from Mitch Barnett to correct timing issues with speakers being different distances from the listening position and others room correction items. It's pretty remarkable. 

I'll read that with great interest Chris. In the past I've applied DRC to the main two channels; but wasn't fully convinced of the results. At the moment I just have correction applied to the Short Basshorns, after measuring the room with Avantgarde's own measuring tools and submitting the recorded files for them to create the optimised curves for my room.

[Geoffrey Armstrong]

2 hours ago, wisechoice said:

That said, please permit me to point out that you've made an error in your macOS speaker configuration. If you only have a 5.1 setup, with no height speakers, the correct setting in Audio Midi Setup is 5.1, not 5.1.2. Then your system will render Atmos correctly to the 5.1 configuration if that's what you're listening with. If you set it to 5.1.2, there is no guarantee that it will be rendered correctly. In fact, because Atmos is object-based rather than channel-based, there's a good chance that any musical or sound elements above the listening position (let's say a balcony choir in a classical piece, for example) will be improperly rendered, without the detail they should have. The difference may be subtle for some music, but more dramatic for other music. That's because the rendering in an object-based system like Atmos isn't merely additive. The format is supposed to be totally independent of the number of speakers you're listening with. So you should even be able to listen to an Atmos mix rendered in mono or stereo (not binaural) and still hear all the elements *somewhere*, even those that were mixed to rear or height channels. Note that this is not a "mixdown," either, because there were never discrete channels in the original Atmos mix, only objects.

Thank you for your clear explanation and for correcting my error. In fact 5.1 is the first option I chose in Audio Midi set-up, since as you point out, that corresponds with the number of speakers I actually have.

 

With my first attempt though, I couldn't get this to work through BlackHole 16Channel version as input to HQPlayer for playing out to my Exasound 8 channel DAC. When I changed to 5.1.2 Atmos, I got it working. So I figured I may as well blend the height channels into the Center Channel with HQPlayer's Matrix Pipeline tools.

 

Having said that, I have now reverted to 5.1 in Audio-Midi set-up and set HQPlayer as outputting to 6 channels and it's all working fine. I must have made some other mistake in my first attempt.