Audiolinux Server configurations, Software, Hardware, and Listening Impressions

[Miska]

2 minutes ago, lmitche said:

For the traffic generated by roon, optane is ideal as it is the fastest nonvolatile memory available for short bursty traffic.

 

Yes, that I agree with...

 

3 minutes ago, lmitche said:

We could debate why a ram disk based OS sounds better then a demand paged OS all day long. For most of us it just does, and we don't know why.

 

All the code pages stay hot and in RAM anyway, especially when there is no memory pressure, so there's no storage traffic needed...

 

Most of my bigger server OS installations are on Samsung 960 Pro and Intel M.2 SSDs. And on smaller devices on the internal eMMC FLASH or microSD card. I don't use RAM disk (tmpfs), other than for /tmp and /run.

 

5 minutes ago, lmitche said:

Music is either fed from a nas or external usb disk.

 

NAS is good, but I personally really really hate USB disks for music content and never use them for such. I use either M.2, SATA, eSATA or Thunderbolt that allow fast and efficient storage access.

 

[bobfa]

9 minutes ago, lmitche said:

My servers all ram boot the AL OS from a 32gb Optane stick. This is especially fast with Optanes crazy sequential read speeds.

 

At startup the Roon scripts are pointed to an Optane based Roonserver software and database in a separate optane partition. This way any database or software update is stored in non volatile memory. Everything else runs in ram. Music is either fed from a nas or external usb disk.

 

For the traffic generated by roon, optane is ideal as it is the fastest nonvolatile memory available for short bursty traffic.

 

We could debate why a ram disk based OS sounds better then a demand paged OS all day long. For most of us it just does, and we don't know why.

 

With AL it is trivial to test this one way or another just by turning ramboot on or off at boot when the yes or no question  is asked so people can have at it.

 

I still prefer zero nonvolatile storage in the endpoint, so the Usb stick boot and remove method or iscsi boot from server is preferred. Unfortunately Iscsi over wifi is unavailable, so the latter only works with wired connections. 

So do you put two Optane m.2 drives in the system Roon Server system?  

[Miska]

6 minutes ago, luisma said:

Thank you, I understand now, I have Roon and HQPE running along on the same server 18.04 NUCi7 and Roon Bridge / NAA on Celeron with 18.04.

 

For me, I have the normal full Roon installed on macOS on my Mac Mini. Content is stored on external Thunderbolt HDD.

 

I have another copy of all the music content on my Xeon workstation with Samba-server to operate as a "NAS" for all other computers. However, this "NAS" is accessed only by HQPlayer library features, not by Roon which is on the Mac Mini and it's local storage.

 

[austinpop]

7 hours ago, Miska said:

 

Why would you have entire Optane just for the database and not running the OS from there?

 

Yes, that is the second part of the recommendation, but I didn't mention it as it does take a little bit of effort to set up, and a lot of people like the USB boot option for its simplicity.

 

Why Optane, and not just a regular NVMe SSD? Well, this whole AL/RAM/NUC experiment was driven by  many people observing that diskless units sounded better, when divested of SATA and NVMe storage devices (HDD and SSD). Larry, Roy, myself and others thought that Optane might be a good option to experiment with because:

1. Optane has extremely low latency, and to the extent that the SQ benefits we hear with AL in RAM is due to latency, this would be beneficial
2. Optane is electrically lower power than a standard NVMe SSD. 32GB of Optane consumes 2.5w when active and only 0.8w at idle.

Based on these, it was worth trying out. To my ears, the Optane drive adds no noise penalty to the server, while adding a very functionally useful persistence capability: both as a boot alternative to USB stick, and for Roon DB, LMS cache, etc.

[austinpop]

6 minutes ago, bobfa said:

So do you put two Optane m.2 drives in the system Roon Server system?  

 

No need, Bob. One 32GB Optane m.2 SSD is plenty to store your AL OS, and your Roon DB.

 

It does take some steps to boot from Optane. You have to create / and /boot partitions and dd over the contents from the USB disk.

 

Maybe @hifi25nl would publish a procedure to do so?

[Miska]

11 minutes ago, austinpop said:

Why Optane, and not just a regular NVMe SSD? Well, this whole AL/RAM/NUC experiment was driven by  many people observing that diskless units sounded better, when divested of SATA and NVMe storage devices (HDD and SSD). Larry, Roy, myself and others thought that Optane might be a good option to experiment with because:

1. Optane has extremely low latency, and to the extent that the SQ benefits we hear with AL in RAM is due to latency, this would be beneficial
2. Optane is electrically lower power than a standard NVMe SSD. 32GB of Optane consumes 2.5w when active and only 0.8w at idle.

Based on these, it was worth trying out. To my ears, the Optane drive adds no noise penalty to the server, while adding a very functionally useful persistence capability: both as a boot alternative to USB stick, and for Roon DB, LMS cache, etc.

 

Yeah, I know Optane quite well...

And I agree that it is very good choice for use in music servers. Now quite a bunch of motherboards have one or more Optane slots! For the reasons you have stated I've also been recommending it.

 

[gsquared]

35 minutes ago, lmitche said:

At startup the Roon scripts are pointed to an Optane based Roonserver software and database in a separate optane partition.

 

Larry what size is the partition that you allocate to the Roonserver software and db?

[bobfa]

48 minutes ago, austinpop said:

 

No need, Bob. One 32GB Optane m.2 SSD is plenty to store your AL OS, and your Roon DB.

 

It does take some steps to boot from Optane. You have to create / and /boot partitions and dd over the contents from the USB disk.

 

Maybe @hifi25nl would publish a procedure to do so?

I am going to order an Optane NVME SSD stick today and will work with (pay for as needed) @hifi25nl to help.  I have AL setup with Roon server right now.  I will run this on my NUC7i7BNH for now and if it works out as a test then decide if I need to build one of the other boards in a fanless case.

[lmitche]

2 hours ago, gsquared said:

 

Larry what size is the partition that you allocate to the Roonserver software and db?

Usually 12gb for boot, root and iscsi lun, 16gb for roon.

[gsquared]

21 minutes ago, lmitche said:

Usually 12gb for boot, root and iscsi lun, 16gb for roon.

 

Thank you!

[lmitche]

8 hours ago, Forehaven said:

Larry, " I expect to replace the i7-6700k and z270 with a modern coffee lake and related motherboard in the next few weeks."  Really?  So interesting!  TBH, I'm surprised still to hear again that an i7 NUC sounds better than our super tweaked out pc's.  

 

Amazing progress in so little time

Hi Forehaven,

 

There are people that are looking for new tweaked out servers that are not NUC based, so the coffee lake and z370 type motherboard seem to meet those needs best.

 

This allows tweaking with special nic, USB cards and clocks in a way that is not possible on a NUC.

 

Will these solutions sound better then a NUC, I don't know.  How many angels dance on the head of a pin?

 

 

 

 

[Forehaven]

16 hours ago, lmitche said:

Hi Forehaven,

 

There are people that are looking for new tweaked out servers that are not NUC based, so the coffee lake and z370 type motherboard seem to meet those needs best.

 

This allows tweaking with special nic, USB cards and clocks in a way that is not possible on a NUC.

 

Will these solutions sound better then a NUC, I don't know.  How many angels dance on the head of a pin?

 

 

 

 

Ah!!  Now that makes sense Larry.  Despite all the improvements we've seen recently, there's still more to be had perhaps with clocks in particular.  I'll be following with much anticipation

[austinpop]

On 12/31/2018 at 3:04 PM, austinpop said:

 

The experiment I am proposing, which I did last night, and plan to do again tomorrow, does not require a reboot, and can be accomplished within seconds on A and B. This does not require your (human) memory buffer to wait multiple minutes between comparisons!

 

The experiment is to compare 2 servers A and B driving an identical endpoint, E. In my case:

• A = my Dell XPS 8700 i7-4770 box
• B = NUC7i7DNBE
• E = TLS DS-1

with all of these running AL in RAM.

 

For me, the baseline comparison was Roon Core on A and B, and RoonBridge on E. Switching cores is easy: go into settings, and click on the Core to bring up a list of Cores on the network, then select the other Core, and listen. Switch back and forth.

 

Here, I can clearly tell that A driving E sounds different than B driver E. Make sense so far?

 

So the new experiment is to run squeezelite (with large buffers - either my settings ,or Ray's settings) on E. You don't need more RAM. My settings ( -b 2097152:2097152 -a 52428800:4::) require 4GB of memory consumption, and fits comfortably on my 8GB endpoint.

 

You can drive E running squeezelite with either LMS or Roon Core on A and B. If you do LMS, then it is again very easy. Use the web UI to control LMS (not iPeng or another mobile app).  Squeezelite by default always attaches to one LMS instance. Say it's connected to LMS A. Play your test track on LMS A and listen. Now go to the web UI of LMS B. On the top right, pull down to select the squeezelite instance. It will prompt to ask if you want to connect LMS B to it. Say yes. Now play back your test track on B. Listen. Rinse and repeat.

 

The situation is messier with Roon Core as server. There is no easy way to detach squeezelite from Core A and attach it to Core B. Probably the best way to do this would be (type menu, then select Configuration):

• Start and Enable RoonServer on A, then Start and Enable Squeezelite on Endpoint
• Verify the Endpoint is visible
• Run test
• Stop and disable all running Audio Services on A, then on Endpoint
• Start and Enable RoonServer on B, then Start and Enable Squeezelite on Endpoint
• Verify the Endpoint is visible
• Run test

Repeat until done.

 

Due to travel, and other projects, it took a while, but I finally completed this study to my satisfaction. The procedure I followed was more or less as enumerated in my quoted post.

 

To reiterate - the key question was this: does the (Roon) server make a difference in SQ, when running:

• a) Roon Bridge on the endpoint?
• b) Squeezelite with large buffers on the endpoint?

I had already found the answer to a) to be yes, definitely. With regard to b), I found the answer was also yes, with the following specifics:

• endpoint was TLS DS-1
• Servers compared were my Dell and NUC7i7DNBE/Plato X7D.

However, there was a wrinkle. Looking at iftop on the endpoint, I noticed that while Roon Bridge maintains a throughput of about 4.5 Mbps, even with Squeezelite (SL), the throughput was only at about 9.2 Mbps. This was puzzling, as I had previously seen the initial throughput with SL to be up to 200Mbps with the i7DNBE as the endpoint. What was going on?

 

It turns out that the DXD streaming limitation I referred to in my DS-1 review was due to the DS-1's ethernet port auto negotiating a speed of 10 Mbps (yes 10, not 100) with the switch. This is the issue Adrian is working with Intel, but as of now, this is the case. What does this mean? It means that even with SL, the initial network traffic is not a firehose, but just a fatter hose. On a 10 minute 24/96 track, it took 3:45 before the network throughput dropped down to 0, once the file was in the SL buffer.

 

Could this be the reason I still answered yes to question b) above? Time for a different test. I now used:

• i7DNBE as the endpoint
• Server 1 was my Dell
• But what to use as server 2? I had run out of options, until I decided to just use my Dell M3800 laptop.

Surely, the Dell laptop (running Roon Server on Windows 10) would sound significantly worse than my more powerful Dell desktop running AL in RAM? Don't call me Shirley.  

 

First, I verified the network throughput behavior. As expected, Roon Bridge still drives a nice, flow controlled throughput of 4.5Mbps. But with SL, as expected, the firehose is observed for a few seconds, and then throughput drops to 0. Much better! So with this new configuration, back to the question:  does the (Roon) server make a difference in SQ, when running:

• a) Roon Bridge on the endpoint? heck yes. 
• b) Squeezelite with large buffers on the endpoint? yes, but... the delta is much smaller.

As @ray-dude analyzed - at some level, there should be no difference at all, since the server is essentially idle for the bulk of the playback. Yet, I do hear a (small, but repeatable) difference. I have no explanation, other than to report what I observed.

 

Certainly, one actionable conclusion is that when using SL, the server contribution to SQ is much smaller, so perhaps it isn't necessary to take heroic (or expensive) measures to optimize the server.

 

I welcome other reports comparing servers between Roon Bridge and SL.

 

[lmitche]

54 minutes ago, austinpop said:

 

Due to travel, and other projects, it took a while, but I finally completed this study to my satisfaction. The procedure I followed was more or less as enumerated in my quoted post.

 

To reiterate - the key question was this: does the (Roon) server make a difference in SQ, when running:

• a) Roon Bridge on the endpoint?
• b) Squeezelite with large buffers on the endpoint?

I had already found the answer to a) to be yes, definitely. With regard to b), I found the answer was also yes, with the following specifics:

• endpoint was TLS DS-1
• Servers compared were my Dell and NUC7i7DNBE/Plato X7D.

However, there was a wrinkle. Looking at iftop on the endpoint, I noticed that while Roon Bridge maintains a throughput of about 4.5 Mbps, even with Squeezelite (SL), the throughput was only at about 9.2 Mbps. This was puzzling, as I had previously seen the initial throughput with SL to be up to 200Mbps with the i7DNBE as the endpoint. What was going on?

 

It turns out that the DXD streaming limitation I referred to in my DS-1 review was due to the DS-1's ethernet port auto negotiating a speed of 10 Mbps (yes 10, not 100) with the switch. This is the issue Adrian is working with Intel, but as of now, this is the case. What does this mean? It means that even with SL, the initial network traffic is not a firehose, but just a fatter hose. On a 10 minute 24/96 track, it took 3:45 before the network throughput dropped down to 0, once the file was in the SL buffer.

 

Could this be the reason I still answered yes to question b) above? Time for a different test. I now used:

• i7DNBE as the endpoint
• Server 1 was my Dell
• But what to use as server 2? I had run out of options, until I decided to just use my Dell M3800 laptop.

Surely, the Dell laptop (running Roon Server on Windows 10) would sound significantly worse than my more powerful Dell desktop running AL in RAM? Don't call me Shirley.  

 

First, I verified the network throughput behavior. As expected, Roon Bridge still drives a nice, flow controlled throughput of 4.5Mbps. But with SL, as expected, the firehose is observed for a few seconds, and then throughput drops to 0. Much better! So with this new configuration, back to the question:  does the (Roon) server make a difference in SQ, when running:

• a) Roon Bridge on the endpoint? heck yes. 
• b) Squeezelite with large buffers on the endpoint? yes, but... the delta is much smaller.

As @ray-dude analyzed - at some level, there should be no difference at all, since the server is essentially idle for the bulk of the playback. Yet, I do hear a (small, but repeatable) difference. I have no explanation, other than to report what I observed.

 

Certainly, one actionable conclusion is that when using SL, the server contribution to SQ is much smaller, so perhaps it isn't necessary to take heroic (or expensive) measures to optimize the server.

 

I welcome other reports comparing servers between Roon Bridge and SL.

 

Rajiv,

 

Nice report and lots to think about here.  Off hand, it would be interesting to have you set the NIC on the NUCI7DNXX to 10 mbps and compare that to the TLS-DS1 box both running Roonbridge.

 

I recently terminated my own Cat 6a Ethernet cables. OMG, they sounded great. Later I realized I'd screwed up, and crossed wires meant the cable was running at 10 mbps.

 

They have been left that way, but now I'm running wireless so don't use them.

 

More to come.

[austinpop]

31 minutes ago, lmitche said:

Rajiv,

 

Nice report and lots to think about here.  Off hand, it would be interesting to have you set the NIC on the NUCI7DNXX to 10 mbps and compare that to the TLS-DS1 box both running Roonbridge.

 

Yeah, that might be worth a try... but not sure when/if I'll get to it. I've been playing around with AL so much the past month that I've promised myself to buckle down and finish a couple of reviews that are pending.

 

One thing I did notice before I set it aside is that even in the SL case, the size of the "firehose" was different between my Dell server running AL/RAM, and my Dell laptop running W10. Whereas the desktop firehose was 200+ Mbps, the W10 laptop seemed to top out at about 50 Mbps between Roon server and the i7 endpoint.

 

So yes, perhaps this explains why I still hear a difference between the 2 servers on SL. But this is of course just a theory that the network throughput between Roon Server and SL endpoint IS the cause of the SQ difference.

 

I really can't devote time to this experiment for a few weeks. If anyone else wants to experiment, I would be very interested.

 

31 minutes ago, lmitche said:

I recently terminated my own Cat 6a Ethernet cables. OMG, they sounded great. Later I realized I'd screwed up, and crossed wires meant the cable was running at 10 mbps.

 

They have been left that way, but now I'm running wireless so don't use them.

 

More to come.

 

Intriguing! Keep us posted.

[BigAlMc]

6 hours ago, austinpop said:

• a) Roon Bridge on the endpoint? heck yes. 
• b) Squeezelite with large buffers on the endpoint? yes, but... the delta is much smaller.

As @ray-dude analyzed - at some level, there should be no difference at all, since the server is essentially idle for the bulk of the playback. Yet, I do hear a (small, but repeatable) difference. I have no explanation, other than to report what I observed.

 

Interesting stuff Rajiv and looking forward to those reviews.

 

Meantime any commentary on the difference in SQ of using Roon Bridge vs Squeezelite with large buffers on the same Roon server?

 

Put simply is it worthwhile for me to open my Endpoint, add another 4gb of RAM (which I have anyway) and try the large buffer size in the Squeezelite endpoint?

 

Thanks,

Alan

[austinpop]

3 minutes ago, BigAlMc said:

 

Interesting stuff Rajiv and looking forward to those reviews.

 

Meantime any commentary on the difference in SQ of using Roon Bridge vs Squeezelite with large buffers on the same Roon server?

 

Put simply is it worthwhile for me to open my Endpoint, add another 4gb of RAM (which I have anyway) and try the large buffer size in the Squeezelite endpoint?

 

Thanks,

Alan

 

Very worthwhile! I’ve described the improvement on the novel thread.

 

i assume you’ve tried the new “experimental” feature on your SE? This is the same thing.

[BigAlMc]

Thanks Rajiv,

 

Yup, I tried that on the SE.

 

Knew it used an SL endpoint but didn't realise it was also large buffers. Didn't think the SE had a lot of RAM.

 

I played around last weekend, made progress but ultimately unsuccessfully, at trying Ray-Dudes instructions.

 

Guess I'll reach for the screwdriver and some Putty later this morning 🙂

 

Cheers,

Alan

[austinpop]

7 minutes ago, BigAlMc said:

Thanks Rajiv,

 

Yup, I tried that on the SE.

 

Knew it used an SL endpoint but didn't realise it was also large buffers. Didn't think the SE had a lot of RAM.

 

I played around last weekend, made progress but ultimately unsuccessfully, at trying Ray-Dudes instructions.

 

Guess I'll reach for the screwdriver and some Putty later this morning 🙂

 

Cheers,

Alan

 

Nuno’s the one who I credit for the idea of running with large buffers. We just replicated their results in the NUC endpoint.

 

Yes, the SE has 8GB RAM.

[HeeBroG]

59 minutes ago, austinpop said:

 

Very worthwhile! I’ve described the improvement on the novel thread.

 

i assume you’ve tried the new “experimental” feature on your SE? This is the same thing.

Hi Rajiv. 

 

If Squeezelite is used as the endpoint rather than Roonbridge; do you then lose all the beauty of using the Roon interface?

 

G

[BigAlMc]

18 minutes ago, HeeBroG said:

Hi Rajiv. 

 

If Squeezelite is used as the endpoint rather than Roonbridge; do you then lose all the beauty of using the Roon interface?

 

G

 

Hi @HeeBroG,

 

Nope. You still use Roon app as controller and Roon server, so interface is the same. You just select Squeezelite as player under the Audio menu in Roon.

 

So Squeezelite is where you're sending the music to, but your still using Roon to select what you're sending.

 

Cheers,

Alan

 

[BigAlMc]

7 hours ago, BigAlMc said:

Guess I'll reach for the screwdriver and some Putty later this morning 🙂

 

Fell at the first hurdle. The Endpoint won't boot on the LPS-1.2 with both 4GB ram sticks in it.

 

Considering buying an 8GB stick on the assumption that one 8GB stick uses less power than 2 x 4GB sticks. Its 1.2v per stick, right?

 

Cheers,

Alan

[seatrope]

I had no luck booting with 1x 8GB stick, Alan. You might have luck with the appropriate 8GB stick selected for low current but it’s borderline even with 4GB on a LPS1.2 for me.

[austinpop]

1 hour ago, BigAlMc said:

 

Fell at the first hurdle. The Endpoint won't boot on the LPS-1.2 with both 4GB ram sticks in it.

 

Considering buying an 8GB stick on the assumption that one 8GB stick uses less power than 2 x 4GB sticks. Its 1.2v per stick, right?

 

Cheers,

Alan

 

6 minutes ago, seatrope said:

I had no luck booting with 1x 8GB stick, Alan. You might have luck with the appropriate 8GB stick selected for low current but it’s borderline even with 4GB on a LPS1.2 for me.

 

Guys, I just checked the DS-1. it does indeed have 2 x 4GB SODIMMs.

• Quick aside: I installed the dmidecode utility (pacman -S dmidecode) to get detailed info about the HW on the box. Try it - for those inclined to.

I am able to run with an LPS-1.2. I can think of 2 reasons:

• have you done all the low power tunings in the BIOS? As well as turning off all the unused things? 
• What is downstream of the endpoint? The current draw of the USB device can be the biggest factor in whether the LPS-1.2 can handle the load.
• Alan, are you using your tX-USBultra? That is what I have. This should effectively shield you from whatever the current draw is from the downstream DAC.
• For the moment, my downstream DAC is the Ayre QX-5 Twenty, which is not bus powered, so its current draw is very low.

[ciccio1112]

1 hour ago, BigAlMc said:

 

Fell at the first hurdle. The Endpoint won't boot on the LPS-1.2 with both 4GB ram sticks in it.

 

Considering buying an 8GB stick on the assumption that one 8GB stick uses less power than 2 x 4GB sticks. Its 1.2v per stick, right?

 

Cheers,

Alan

 

Hi BigMac my nuc with celeron as an endpoint with 2 4GB banks works perfectly with the LPS1.2, you will still have something to disable in the bios.