From 0 to Xeon/AL/Roon Server in 2 days

[bobfa]

 

 

Update February  2020

 

I recently has the opportunity to test the JCAT USB and Net cards in my server.  These cards are a worthy upgrade and I would highly suggest that you consider them in your systems design.  My testing was done using the Xeon machine as a single box solution and the two cards made a lot of difference.  

 

 

Update October 2019

 

Over the summer I have been testing a lot of things around my server and the current place I have landed is the server is now connected to the Kii Control in my main system via a txUSBultra.  I am running Euphony Stylus software as the player.    I am editing this post to reflect where I have arrived.

 

Update April 2019

 

There are a few changes in the server that are important.  First, I have moved to Euphony OS to run the Roon server software.  Next I  removed the USB storage drive and am pulling from a NAS for the music.  The third change was to change the power supply.  I removed the DC-ATX converter and replaced the HDPLEX200 with the HDPLEX400.

 

 

February 2019

 

I have built a custom music server to run Roon Core.  BECAUSE!

 

I have a perfectly good server or rather a GREAT one from Small Green Computer.  Andrew makes excellent hardware, and the OS in the machine is stable and supports a lot of music player software.  It is "Plug and Play” and very well supported. It does not lend itself to tinkering!  I want to listen and tinker with some different ideas with, setup, power, networking, and connectivity.  Thus my build.  Just like my build of the NUC endpoint, I am doing this build in public so that others can duplicate it or follow along. 

 

A Side Note: John Darko put up a very timely video.  It is ostensibly a review of sparkling water and the Innuos ZEN MK3. John speaks the way I think.  

This is not hard if you have “The Knack.” 
And

No Soldering iron required!

General Goals:

 

• Roon Core Server
• Euphony OS $389
•  "Swap-able" components, MB, CPU, Power
•  I want this build to sound GREAT

 

Key design features:

    

• Multiple network interfaces for bridging and more
• No WiFi
• High TDP capable
• Xeon, ECC RAM, Optane Storage
• Option for mass storage
• Fan-less
• Powered by an outboard LPS 
• PCIe expansion card with SFP for network testing

 

System Description

 

A dedicated silent workstation / small business class server. The case is fan-less, with an external power supply.  There is 16GB of ECC RAM and a 32GB Optane  SSD for boot, and database storage.  There are two onboard network interfaces which are different Intel generations.  I have a PCIe network interface card with a single SFP slot to be the optical network bridge to the "best" endpoint that I will be testing with later.

 

The motherboard supports either Xeon or 8th Generation core Intel processors in the LGA1151 socket.  The chipset is VPRO and thus supports Optane storage.  The board also supports both ECC and non-ECC RAM with four slots.

 

I am using the entry-level E-Series  Xeon E-2124G processor to start with as the price and availability are good. The E-2124G has a TDP of 71 which is well within the case specs. I am using ECC RAM that has been validated to work with the motherboard.

 

Music storage is an interesting problem.  I read through the threads on AS about data storage and to be honest, without some summary there is too much data to compile into a “best practice.”   In my testing with a different NAS and a different server than I am running now, the NAS did not sound “as good.” I do not want the drive inside the server, as indications are that internal storage degrades sound quality. 

 

The Case for the Case!

 

I have been researching this for several months, and I picked the HDPLEX case due to the flexibility of their design. The H5 case fits up to a full ATX motherboard like I am using.  The H5 case can also support adding a graphics card to your build which I am not using.  There is room for both 2.5in and 3.5in storage devices. 

The H5 case is designed to support alternative power supply configurations which is one of the bigger keys to performance.    

 

After a lot of testing I am now using the HDPLEX 400 direct into the machine.  I have not yet changed to the Ghent cables that others are using.:

 

 

Buying "stuff." 

 

Picking suppliers is a problem today.  Many online stores are marketplaces with other vendors inside of them, and that adds risk to the purchase process.  I purchased most of the server computer components from Newegg.com.  The Power supplies and case came directly from HDPLEX.  I want to thank Larry at HDPLEX for helping me validate the fit of my motherboard selection during the pre-purchase process.  Great customer service there!

 

I purchased these from Newegg

 

    $248 ASUS WS C246 Pro Motherboard
    $250 Intel Xeon E-2124G
    $214 Two Kensington 8GB ECC RAM (check MB validated models)
 

    

I purchased this from Amazon.

 

  $62 Intel 32GB Optane M.2 SSD

 

From HDPLEX I purchased the following items:

 

    $485 HDPLEX200 LPS
    $388 2nd Gen H5 Case with 400W DC to ATX internal supply

 

 

 

I  have added an Element H USB card $250

I am now using an HDPLEX 400 power supply 

 

For Music Storage I am now using my QNAP NAS

 

 

 

 

I will not count research time as that is endless.  Actual construction and setup were completed over two days.  I estimate about 10 hours of review, assembly, software configuration, and BIOS learning.  As a reminder, I do this stuff for FUN.  If this sounds like work, please buy a commercial server you will be happier!

————

BIOS Setup

 I have turned off all the fan controls and monitoring. The fancy signed secure boot had to be turned off. I have NOT changed any of the default CPU settings.  I have not turned off all of the “other” hardware that I will not be using.  I had some issues with the boot order.  I have not found the way to boot off of USB first every time.  (I will try to do BIOS screenshots later)

————

Euphony Stylus Setup

 

As of about April 2019 I have fully moved off of AudioLinux to Euphony OS and I switch between Roon and Euphony Stylus as my playback software.  Follow the instructions on the website for trial download and setup.  This custom port of linux and player software cost $289 USD and I was very happy to pay that.

 

https://euphony-audio.com

 

 

—————

Build Process

 

I am an experienced PC builder, and they usually go together pretty quickly.  This build took a bit longer than I expected.  Taking great care with the heatsink, the heat-pipes, as well as the first time is this kind of case slowed me down.  If I built a second one I could do it in half a day.  

 

Please remember to use proper anti-static protection.  Do not over-use heat-sink compound!  And have fun!

 

 

Let's get started!

 

 

Here are the "goodies" to start building the case

 

This is the heatsink system

 

 

 

A little out of context! This is a single Motherboard stud.  You have to place them properly for your board.

 

 

Next we put the posts and the power switch button on the side panel and mount the PCB for the USB ports, etc.

 

 

 

Here is the side panel mounted to the base and you can see the motherboard standoffs already inserted.

 

Now it is time to start on the CPU heatsink

 

 

 

 

 

On the LGA11551 socket, the pins for the CPU are on the motherboard.  CAREFULLY place the CPU on the socket and close the locking cover and bar.

 

 

 

I did not take a picture of the underside of the motherboard where the bottom part of the headsink system attaches.  The casemanul covers this pretty well

Add the heatsink compound to the base of the copper heatsink

 

Here is the base of the heatsink attached to the motherboard

 

Another view of the case with the side panel, MB and heatsink ready for the heat pipes.

 

The manual covers the intall of the heatpipes well.  Again donot over use heatsink compound and proceed slowly.

 

 

Here are some shots of the system installed and hooked up

 

You can see the RAM and the Optane drive installed here.  I have since removed the DC to ATX supply!

 

 

 

 

 

 

This is the other side of the case.  I hate how black things collect dust and stuff.

 

 

 

 

 

I purchased the HDPLEX 400 power supply and here are the changes to the server to remove the DC-ATX converter and connect the new supply.

 

I disconnected the DC-ATX converter and removed it from the case.  (sorry no picture)

 

First the old 19V socket and mount have to be removed to place the back to back ATX boards.  This really requires you to remove the back panel first.

 

 

There are four studs installed and the board pair is screwed into the back plate.

 

 

Then the two internal cables are run from the connector board to the Motherboard. 

 

 

The two longer cables are connected to the outside of the case and  run to the back of the HDPLEX supply.

 

 

 

NOTE DO NOT FORGET TO SET THE AC INPUT VOLTAGE SELECTOR PROPERLY!

 

Not hard at all.  

 

 

 

 

 

 

 

[bobfa]

This thread is parallel to my NUC Thread.  I will edit the thread as I finish the project.  (IF it is ever finished)

 

 

[bibo01]

Are you sure that HDPLEX200 LPS is enough cpu/ram/mb/optane/hd (?) ?

[bobfa]

With ZERO issues so far.  There is no graphics card for one thing and Roon hardly tickles the CPU load playing to one or two endpoints. I will be taking some measurements over time.

 

 

[bibo01]

1 minute ago, bobfa said:

With ZERO issues so far.  There is no graphics card for one thing and Roon hardly tickles the CPU load playing to one or two endpoints. I will be taking some measurements over time.

 

 

Well, there is no GPU on-board Xeon!

[bobfa]

The chip I have does have a GPU on it> I have no PCIe board.  The E-2124G has P630 graphics. That matches up with the motherboard that has no onboard graphics and I did not want to have a PCIe graphics card.   

 https://ark.intel.com/products/134854/Intel-Xeon-E-2124G-Processor-8M-Cache-up-to-4-50-GHz-

[bobfa]

Status report

 

 I have not had time to finish my music transfer.  The external drive is now hooked up to the server, and I am SFTPed into the system from my iMac using Transmit5.  Transmit5 is sending 4Tb of data over.  

 

I was looking for other ways to do this and found that I like this method.  It is simple to do, and the performance over the gigabit wired LAN is reasonable.  I drug over several folders in the application, and I thought it did one at a time.  NOPE, it was busy with one, and after that finished, it is now doing several.  When the smaller folders finish, it will be faster per file...  

 

I have Roon watching the folder and importing the music. The transfer will take some time but I can stream from the internet.

 

 

[austinpop]

Fun project. Can't wait to hear the results!

[Blackmorec]

I would never have the guts to start a project like this as I’m far too likely to run out of knowledge should things go even slightly awry. But I do love reading about them and understanding the type of engineering choices that contribute to improved SQ. So excellent work!! 

[bobfa]

I would never suggest that this is a project that anyone should start without the support and knowledge you need.  There are bumps along the road.  

 

I am having fun.  A commercial product might have some additional features.  From watching the John Darko video I linked in the first post the new software on the Innous ZEN MK3 looks nice.  The Sonic Transporter I have has support in a different way but good.  AudioLinux headless menu system also provides a clean way to do most management.

 

I am working with several different ways to manage the music on the drive attached to the server.  I use a terminal program or a web browser to log into the server.  I have used an SFTP program to transfer data to the storage drive.  I am looking at remote sync.

[bobfa]

Building a 

16 hours ago, austinpop said:

Fun project. Can't wait to hear the results!

I wish you could hear the results.  I do have a few others that come by and help/listen. 

[bibo01]

21 hours ago, bobfa said:

The chip I have does have a GPU on it> I have no PCIe board.  The E-2124G has P630 graphics. That matches up with the motherboard that has no onboard graphics and I did not want to have a PCIe graphics card.   

 https://ark.intel.com/products/134854/Intel-Xeon-E-2124G-Processor-8M-Cache-up-to-4-50-GHz-

Do you think that your ext. power supply would still match an E-2128G ?

[bobfa]

My math (it is not perfect) and thus the systems design says that I can use a CPU with a 95W TDP.  That would mean that it should be able to run any of the CPUs in the E-21xxG series:

 

https://ark.intel.com/products/series/134861/Intel-Xeon-E-Processor

 

The top E-2186G is a 95W DTP with six cores and twelve threads.  

 

 

My "MAIN" concern here is to see if it sounds better than what I have.   My listening so far has been limited.  I am still fussing with overall systems design.  I did get a listen to Emerson Lake and Palmer Pictures and an Exhibition today.  I did not want to leave the listening room, but a broken dishwasher is interrupting my music time.  Along with some snow!

 

 

[Bricki]

Absolutely excellent write up!

 

I'm looking forward to reading your listening impressions 👍

[rickca]

10 hours ago, bibo01 said:

Do you think that your ext. power supply would still match an E-2128G ?

I asked HDPLEX if they have tested the 200W LPS at 19V/10A (or even 19V/8A) for an extended period.  

HDPLEX answered:  Yes, for 72 hours.

 

I am skeptical because I see XR-160 written on the side of the transformer.

HDPLEX answered:  XR-160 is not referring to wattage. It is the model number for that core structure which the transformer factory attached the coil to.  120W-160W-200W-240W all use this same core structure.

[bobfa]

Some notes observing the thermal behavior of the server during Roon fingerprinting.

 

I have all of my music loaded into the server. Roon does background audio analysis of all new tracks loaded.  I had left it to throttled the default.  The CPUs were not any warmer than on idle around 35C.  CPU load was up a little bit from idle.  So I "turned it up a bit"!   I set it to run in Fast (4 cores). The Xeon is flying through this faster than what I have used before.  I have been watching it and the HDPLEX200 power supply.  I have not opened the case to see how the DC - ATX converter is doing, but the bottom of the case where it is attached is slightly warm.!

 

 After about 3 hours running. Here are a couple of screen grabs from the Audiolinux status menus.  The temps are around 57 to 60C. and the CPU is around 50%

 

 

For a little fun, I fired up my Flir One camera on the iPhone and took a couple of shots with it.  Note I do not know the actual temperatures here.  The heat-pipes from the CPU connect to the right side of the H5 case.  The hard drive on top of the server is just "warm."  The left side of the HDPLEX 200 is pretty warm.  Not hot you can hold your hand on it no problem.  It is winter here, and the basement where all this gear is running is about 65F ambient.

 

 

These numbers and pictures indicate normal behavior.  After all of the analysis is done I will grab some more numbers.

 

Bob

 

 

 

 

[bobfa]

My DIY Xeon Roon Core server up and running, and along with the fun, there are challenges. 

 

One of those challenges is the local music file management.  My server is a stand alone music-appliance; the music is a copy of the master library stored on my iMac.  To that end, I have attached an 8TB drive in a USB enclosure and connected that to the server via USB 3.0 and powered directly by the HDPLEX200. 

 

Audiolinux does not appear to have file sharing services on by default, and I did not want to set up those services when SSH and SFTP work just fine.  Over the past couple of days, I have been testing different methods of managing the music data on the server.  I tried standard file copy methods by attaching the drive to my iMac, two SFTP programs and a few other crazy methods.

 

I settled on using the Linux ext4 disk format as it is native.  The Mac (or PC) needs to have individual drivers to access that format.  I tried using the Paragon driver, and it caused my iMac to crash (black screen reboot) during data transfer.  It is not viable in the long run.

 

 I have a couple of programs for SFTP, but I like Yummy FTP Pro for this project.  It has the controls I need to manage the transfers and maintains the best speed over time in my testing.  I can enable automation to automatically update the server.  If you have a Setapp subscription, Yummy FTP Pro is free.

 

Transmit 5 works very well, but I observed more slowdowns during significant transfers.  Transfer speed is not a problem during regular updates.  The total transfer time for 3.5TB of music using SFTP over gigabit Ethernet was 20 hours.  There were a bunch of weird slowdowns, some of which matched up with Time Machine backups.  Moreover, I got to do it twice because I changed my mind on which disk format I was going to use.

 

NOTE:  I currently do NOT make metadata changes using Roon as there as I do not have a feedback method to update the master.  I hope to be able to address this in the future.

[bibo01]

18 hours ago, rickca said:

I asked HDPLEX if they have tested the 200W LPS at 19V/10A (or even 19V/8A) for an extended period.  

HDPLEX answered:  Yes, for 72 hours.

 

I am skeptical because I see XR-160 written on the side of the transformer.

HDPLEX answered:  XR-160 is not referring to wattage. It is the model number for that core structure which the transformer factory attached the coil to.  120W-160W-200W-240W all use this same core structure.

What are your considerations then? Would it be safe to employ a HDPlaex with an E-2128G?

[Miska]

34 minutes ago, bibo01 said:

What are your considerations then? Would it be safe to employ a HDPlaex with an E-2128G?

 

Xeon E and E3 are practically like Core i5/i7, with memory controller support for ECC RAM, so the power requirements are similar too.

 

From Xeon E5 and E7 it begins to differ more from Core processors. My development workstation is Xeon E5 and it supports for example much more RAM than the smaller ones can support. For that reason, the motherboard also has eight DIMM slots. And going to two or four socket boards you also gain higher total RAM bandwidth because each CPU has it's own memory controller and memory.

 

[rickca]

42 minutes ago, bibo01 said:

Would it be safe to employ a HDPlaex with an E-2128G?

I can't find any specs for a Xeon E-2128G.  Is this just a typo?

 

On my i7-6700K Windows 10 system, I encountered a random reboot during a Windows Update similar to what would happen with a brief power outage.  I'm not talking about the expected reboots that happen during a Windows Update.  After that I was in a boot loop and had to reinstall Windows 10.

 

I may have exceeded the maximum load the 200W LPS can manage, but I don't think so since HDPLEX says they have tested the 200W LPS at 19V/10A for 72 hours.  My unit couldn't sustain it and I'm sending it to HDPLEX for a thorough checkup.  Perhaps there's something wrong with my specific unit.  It worked perfectly well just playing music and sounded great.

 

I don't want to sidetrack @bobfa's documentation thread so I'll report back about my unit on the HDPLEX 200W LPS thread.

[Miska]

On 2/18/2019 at 4:42 AM, bobfa said:

The top E-2186G is a 95W DTP with six cores and twelve threads.

 

https://ark.intel.com/products/134855/Intel-Xeon-E-2186G-Processor-12M-Cache-up-to-4-70-GHz-

 

Quote

$250 Intel Xeon E-2124G

 

https://ark.intel.com/products/134854/Intel-Xeon-E-2124G-Processor-8M-Cache-up-to-4-50-GHz-

[johnli]

hello bobfa.

 

Awesome post! I love how you built it.

 

I'm completely new to DIY server, thought I have a few years of experience of using "brand" servers, and now I'm interested and want to start to build my own server.

 

Can you kindly break down the steps into smaller ones, or even with included video? So that I can buy the components, just like yours, and built it step by step like building lego.

 

Many thanks for your great help!

 

happy listening!

[bobfa]

John,

 

Welcome to the Forums!  I hope that you enjoy what you find here.

 

I am sorry I do not have time to generate that work it is a big job to do that. There are a LOT of PC build videos on YouTube that can help you.  There are very few steps past what is shown. There is quite a bit of assumed knowledge here.  You will need experience with basic Linux commands, some mechanical skills and basic tools.  

 

bob

 

[bibo01]

16 hours ago, Miska said:

Xeon E and E3 are practically like Core i5/i7, with memory controller support for ECC RAM, so the power requirements are similar too.

 

From Xeon E5 and E7 it begins to differ more from Core processors.

In this perspective, how are Xeon Platinum, Gold and Silver placed?

[Miska]

5 minutes ago, bibo01 said:

In this perspective, how are Xeon Platinum, Gold and Silver placed?

 

They are newer parallel line. I think idea behind the Scalable series where those belong is to have a single socket that can take different levels of CPU, so you can upgrade easier. Of course within other limits of the system (cooling, PSU, etc). So there's the socket 3647 for those.

 

Somewhat like Silver = E/E3, Gold = E5, Platinum = E7.