Building a DIY Music Server with custom made parts

[OAudio]

23 hours ago, Peter Avgeris said:

IMHO the first issue that needs to be discussed is the specifications that need to be met.

 

Peter hi,

 

Agreed, I would go with:

• optimised sound quality first,
• then component  compatibility, manufacture, and ease of build.

I think for optimised sound quality you might want to think about some research. I use active cooling at the moment, but this is only to eliminate thermal considerations whilst developing my server (no impact on sound quality). Passive cooling is coming in time which I have looked at for the C620 series. I am not going into details of the research here but it may help to know that the cooling design, even the passive approach you are looking at has a bearing on sound quality. Defiantly the case for C620 / LGA3647 but also for X99 and X79 and LGA 2011 builds, I have no experience of LGA115x. 

 

This said this the lions share of the cooling related SQ probably comes from move to passive which I am positive you know , so making this accessible to people doing builds is a great goal.

 

OAudio

[OAudio]

21 hours ago, LTG2010 said:

https://diyaudiostore.com/collections/chassis/products/deluxe-5u-ultimate-amplifier-chassis?variant=12212123204

 

This might be a good case to mod - it has giant heatsinks. Its made by modushop and can also be bought from Italy. They are able to replace the back panel with one for a computer board, ie cut out the openings for the board faceplate and vertical PCIE slots.

They could also cut the grooves for heatpipes but they need cad drawings. I suppose they could also mill the cpu heatsink from the drawings.

It also has more space internally and the height aids passive ventilation.

The alternative is to provide them with a drawing for a case from scratch, but I suppose you already have access to such facilities in Greece.

 

Yes this is a good looking case I have been looking at it as well as an interim step to finalise packaging.

 

Its worth thinking about the one or two box build decision a little more. IMHO a single case is the way to go for sound quality, no contest. I have gone down both routes and looked at the impact of umbilical's. A single box setups does take time to optimise component positions, particularly the magnetics and they are just well "big" which is not cool depending on what you looking for. But for sound quality I think they are the way to go.

 

The HDPLEX case is a beautiful piece of engineering, no question its beautifully designed and manufactured but if it doesn't have the volume for an internal power supply of the quality your happy with then its might be worth spending some time considering this point.

 

Powering a PC for SQ is not the same as powering an amplifier. An amp needs a relatively slow transient response from its supplies compared to a PC motherboard. For an amp a two box set up with 0.5m to 0.7m of umbilical and internal wiring and umbilical chassis connectors can be ok. For a Audio PC this setup tends to limit SQ performance particularly on more powerful and current hungry motherboards.

 

Many leading commercial server designs lean towards a single box design, I don't think its by chance, and worth considering.

 

OAudio 

 

 

[OAudio]

1 hour ago, Peter Avgeris said:

OAudio, you are sooo right!

PCs work on the current domain, they require current, while amps require voltage. Power hungry motherboards do exist. The umbilical cables of thin copper conductors are a catastrophe for powering hungry motherboards.

Even in my audio power amplifiers, I do consider the second chassis and the umbilical cables a very serious restriction that can be worth doing but only following very strict design specs without losing sound quality.

 

Finally! This is a really constructive and supporting comment!

 

Good that we are on the same wavelength.

 

To put some figures against this.

 

A C621 LGA3647 (pretty much the worst case I have measured) easily pulls  uSec transients of 7 amps from a LPSU that in turn feeds a DC ATX like an HDPLEX. This is a 140va transient going into the DC ATX. On the output of the DC ATX this is likely to be an 11amp transient on the 12v rail into the mother board. Actually probably more than 11 amps because the DC ATX has a few thousand uf of capacitance that shields the LPS from the true size of the transient. If an umbilical and chassis connectors have just a few 10s of mOhms in them, then your into many millivolts of rail drop on input to the DC ATX, above the threshold for audible effects to happen.

 

The resistive element is simple the inductive impact of the long umbilical is not so much. DC ATXs put out a ton of switching noise to their input terminals back towards the LPSU, there are at least 3 buck convertors most DC ATXs. This arrangement can ring like a bell with long inputs leads.  

 

Even moving an open window on screen, starting a network or local file copy can generate an up to an amp or two of uSec transients at the LPSU. This is as CPU cores, GPU Networks and drives wake up. I Haven't looked at the load the software DSPs generate but if they need 4Ghz processing to work as I am reading, you can guess. This is why I have personally avoided this type of real time software processing  

 

Measuring and quantifying all this for a number of Motherboards and DC ATXs was my starting point to understand the requirements of LPSU's before moving onto designing and building them.

 

Summary: It can be a good thing for SQ to put your supply in the same server chassis, with the caveat that packaging its will take some real work. Where you put things matters for sound quality :-) 

 

OAudio

 

 

[OAudio]

@Jabbr, @Peter Avgeris

 

It's the combined architecture of the Scalable CPUs and c621 PCH together that really matters. Intel have really delivered a hand up with this platform.

 

What I think are important (backed by much research and experimentation on most of the points) is;

 

1. The clock subsystem. Wow this is where Intel have really helped out in the design. 

• A single oscillator on the PCH generates the CPU, RAM, DMI, PCIe, SATA, USB and Network (for on-chip N/W) clock domains signals,
• All these domains and subsystems now locked to each via the single clock - the PPLs generating the clocks for the domains all reference the same master,
• The master clock is even an integer multiple of the USB clock frequency ! so exceptional for good timing of on board USB interfaces for music if you want,
• Finally the power system for the master clock is very different to consumer motherboards, the design effectively carves out almost all of the clock subsystem, (its PLLs and transmission buffers) away from most of the motherboards power rails and other systems.

1. RAM speed - CPUs have 3 x individual memory controllers supporting 6 channels in total,
2. PCH to CPU speed - 4 x high bandwidth DMI channel between CPU and PCH,
3. Generous amounts of on chip CPU L1 - 3 cache,

   

The above in particular the clock system, puts the c620 architecture in a completely new league for audio servers compared to the earlier Intel X series (x79, x99 x299) and C series (C600) PCH boards.

 

I don't think you really need to worry much about fast processor frequency with a c621 system unless you have playback software that must run on fast cores. I tend to go as slow as possible on core speeds deliberately in order to ease power supply design and CPU electrical noise. Processor core count dose also make at least some difference.

 

I don't have direct experience of the sage twin CPU board (by choice, going a slightly different direction here) but the 621 architecture is defiantly the way to go Peter. 

 

OAudio

[OAudio]

1 hour ago, jabbr said:

That’s your personal opinion however if you can’t run EC modulators in HQPlayer that’s a complete deal breaker for me. I place real math capabilities way over theoretical differences. 

Jabbr hi,

 

I didn't claim any experience running the EC modulators I don't know them (I think I mentioned them as DSPs in my post) so defiantly take your views on that.

 

I am a relatively new user of this forum so its understandable to assume that I may by putting forward baseless ideas or opinion about the 621 architecture for instance. I want to reassure you that the comments are not purely speculation and are in fact based on many, many years of research both practical and paper based into a number of Intel Architectures. For sure I would say I am not your typical server builder. Where I haven't had direct experience of something you will see that I have been calling this out in my posts and then may offer opinion or just leave the matter as I cannot add any value.

 

A few examples of what I have been up to and the type of activity that leads me to be confident about the statements made about the c620 architecture.

 

Worth mentioning I am on the 5th instance of the c621 board I use this is because a lot of the stuff I do is at board level and can and does break stuff. I risk the expense this brings because much of the time it helps learn. Thinking about it I also got through 4 x X99 boards and 5 x X79s boards for the same reason. I don't just put parts together in different combinations.

 

How can I claim understanding the importance of locked clock domains ?

 

Well as a small example (but there is much more experience behind the statement) many years ago I built an X79 system 3 separate commercially available reference level clocks (quite unusual for the time) which were set up especially to be able to "pull" each oscillators frequency from nominal by up to a few thousand herts. The clocks drove 1) the CPU / RAM domains 2) the PCIe buss domain and 3) the USB domain. I then spent months working on what the optimum relative frequencies were between each domains for best audio performance. Pulling clocks by as little as 0.000002 % can be heard. Get the set frequencies right and the music really comes to life... very important to SQ. After much subsequent work in the many builds following that I "know" that interdomain clock speed matters and how to set them. The 621 does much of this for you which is a gift I think to us as music lovers from Intel .

 

How do I know how the c621 clock sub system is powered and why this is important and what the c621 clock subsystem architecture is ?

 

I read the relevant Intel technical documents (and have done at least as far back as the X79 architecture) and then I experiment hands on with the boards to work this stuff out (sometimes breaking them  but its worth it for the SQ). I used to use expensive commercially available clocks but found that they do not meet the specifications I want so I build my own these days. There is simply a huge amount or work needed to work out how to make a clock really work well within a specific audio server environment - as apposed to simply bolting in a commercial clock and crossing fingers - but that is probably for another thread maybe.

 

RAM Channels ?

A simple test of ram memory channels (but not the only evidence I base the observation on) just take out DIMM pairs 2 at a time to drop from 6 to 4 to 2 channels and listen. You should clearly hear the impact. Other things would be in play too doing this but believe me your listening to the channel numbers in play as well.

 

I don't want to be rude or get into post exchanges, but I did want to provide just a very small bit of background on type of activity behind the views I'm posting. Of course that doesn't mean they are guaranteed to be right . 

 

Best regards,

 

OAudio

[OAudio]

7 minutes ago, jabbr said:

The newest architecture which bears looking into and which I am running is the Xeon W - LGA1200 / 422. These really are ideal, in terms of Xeons, for audio workstations because of:

 

1) high base clock rates

2) multiple PCIe lanes

3) ECC RAM / quad channel 

 

Now, I am not saying that this is better than the 470 series (non ECC) and ASUS makes both, but for example my 100 Gbe NIC as well as RTX 2080 GPU ... both now NVIDIA and both requiring PCIe x16 ... 

 

Arguably if one wants the very best clocking then wait for PCIe-4.0 motherboards — Intel is still at PCIex3.0 because the current boards can’t handle the right jitter requirements of PCIe 4.0

Ver interesting,

 

I have had my head down working on the c621 and had missed the 422 I will pull the documents and take a look.

 

I think I understand our different level of emphasis on PCIe now with your network and graphics setup.

 

I have gone down the route of all interfaces on-chip on the PCH, so network, USB, and storage.  I think this gives CPU > DMI > PCH > direct on chip to USB / and SATA / and Network (the network uses another serial standard from PCI chip to a media controller on the motherboard, I forget what its called, but I think its still in the PCH network clock domain which is nice).

 

If I have this right I don't think I use PCIe this way at all (must check again ) but it puts me more at the mercy of the motherboard implementation of the relevant ports for electrical signal integrity of the ports. Still the clocking is good for the reasons mentioned earlier.

 

You seem to have quite a collection of systems

 

[OAudio]

1 minute ago, rickca said:

Can you please provide a link to the source of this information?  Thanks in advance.

Its spread across multiple sources and I tend not to compile as I read, but its out there if you look at c621 architecture information sources and board reviews.

 

The summary above covers the key points regards the clock subsystem.

 

OAudio

[OAudio]

All,

 

I was taking a moment to step back though earlier to think about the 2 CPU direction. It is quite complex and expensive in relative terms to a single CPU c621 build. I was trying to fathom why go that route ?

 

Is it following the established Sage build taken by Taiko with the Extreme server so likely to assure a good SQ outcome, this could be a reason in and of itself. Are there any known "USPs" or theories that people are driving to realise with the Sage dual 2 CPU route ?

 

A few things circulating in my mind comparing with a single processor c621 build.

Cost, motherboard, 2* CPUs, 12* Memory DIMMs

More demanding on PSU requirements

Needs PCIe cards for network and USB interfacing to perform

Use of NUMA routing for cross CPU1 acceding CPU2s memory (slower than direct CPU to Memory access)

Cooling

Unusual form factor to accommodate

 

The main reason for dual processors that I am coming up with is: there may be a possibility to allocate end point and music server software processes (control and file storage tasks) to separate physical processors. Might by good if the music server streams files to the end point during playback maybe.

 

I don't have any experience of Dual CPU builds and why you would do them so thanks for any thoughts / experiences.

 

O Audio

[OAudio]

1 hour ago, austinpop said:

 

This is exactly the discussion that can derail this thread. Can we just take the above as a good enough reason?

 

Remember, this thread started with Peter observing that people attempting such extreme builds could benefit from his access to customization resources. Let’s focus on that.

 

If we get into the “why on earth would you want that?” discussion... well, we know how those threads end up.

 

Austinpop hi

 

I have read a lot of you posts in the past, its nice to say hello.

 

There is no agenda here just a few relevant questions to the tread that people may want to discuss and we all may learn from the discussion. The thread has been growing quickly so you may not have seen a few of my posts (couple of links for convenience below). I know a reasonable amount about the c621 platform, and I hope these posts added to collective knowledge here.

 

I try where can understand how hardware might be helping music reproduction, I don't know much about dual CPU systems hence the questions, they are not loaded.

 

At the first sign of defensive exchanges I will be off, I am here to enjoy the company of other enthusiasts.

 

OAudio

 

 

 

[OAudio]

Guys,

 

Great conversation,

 

@Dev

 

I am not familiar with the Euphony software package, that's a really interesting point about root access to UNIX and how that plays into a decision about processor count.

 

@Nenon

 

So seperation and optimisation of OS / audio software / device driver across physical CPUs (and RAM) is the targeted benifit.

 

Thinking broadly, a model of how to configure resource optimisations will emerge, sort of "good practice", and details of this could vary for different OS types / playback software / device drivers depending on peoples software selections and system configurations. I think tools like process Lasoo could template this for commonly used setups, but as you say there's certainly some work here to understand all this.

 

I certainly recognose your points about the difficulty of comparison, this is far from plug and play stuff, many many vairiables involved :-). Usefull comparisons could be made however but only with great care. This alone would be quite some project in its own right and not really productive I would say.

 

The real purpose of my question was to understand what people see as the key  areas of attack on sound quality using dual CPUs and this helps, thanks for the answer.

 

 

OAudio

[OAudio]

5 hours ago, bobfa said:

Is anyone thinking about putting Music Storage in the box?  What kind?  SSD? HD?

 

 

My c621 server has one SSD, it's an Intel 128GB so nothing exotic. The purpose is only to hold the OS and playback software. No music is stored in the server.

 

The disk can be turned of or physically removed from the machine after it has been booted from a RAM image.

This allows for some good insights on the impact of the disk on SQ.

 

Sound quality best first.

 

Disk removed from system booted from RAM

Disk in system booted from RAM

Disk in system booted of disk

 

Top line is very special SQ, but my c621 sounds so "real" that I don't get up to walk across the room to take it out most of the time and so listen disk in system booted from RAM  (line 2).

 

No dedicated power needed to the disk. My LPSU and DC ATX setup make the added complexity unnecessary.

 

OAudio.

 

Sorry miss read the question slightly. The notes above give a feel for the impact of an SSD in my c621. I have placed music storage in the system on SSD, Crucial 500gb just for testing reasons. It would not be my preferred playback (because with network storage and the configurations above SQ is clearly improved)  but music on the second disk still sounds good in absolute terms. If this were the only  config I used, I would consider better isolation of the power to the disk because disk is them in live use  whilst playback is taking place.

[OAudio]

On 5/30/2020 at 3:30 PM, rickca said:

Can you please provide a link to the source of this information?  Thanks in advance.

Martin hi,

 

I used early Plextor M.2 SSDs for a while. They could be bought with the M.2 memory stick mounted to a full height PCIe card. The riser card looked easy to modify.

 

This is just a random PCIe SSD card selection (I haven't researched it) but if I were trying to do I might research a card like this to see if the go faster shroud can be removed. Applying power means cutting traces from the PCIe board and wiring in your own, I am sure if this is something you would be comfortable with.

 

https://www.goplextor.com/Product/Detail/M9P(Y)_Plus#/Spec

 

A riser board with a power socket would be helpful, but is not something I would normally look for as I tend to modify boards directly if needed.

 

Regards,

 

OAudio

[OAudio]

Some good points being made here.

 

Personally I ended up spending months and months on the characterisation of motherboards power behavior to be sure of capturing complete requirements (mentioned a little of this is the earlier post). As well as longer duration trace recording its important look extensively, in both time and frequency domains, at real time behaviours. Understanding dynamic behavior and relating it back to what is driving it in the PC is very important.  Some systems for instance will throw out a mSec transient current spike as POST completes and processors are initiated, easy to miss but similar in size to maximum draw on the EPS rail at maximum CPU load. 

 

You could almost weld with some of the LPS prototypes I made up during development. I tried a variety of designs based on different devices types and topologies to understand what works best. It was lot of work but,  but important to properly understand what actually delivers optimal SQ. I'm glad to say (at last) that the supplies are finnished on smart PCBs.

 

OAudio

 

 

 

 

 

 

[OAudio]

Hi All,

 

This is a rendering of one of the parts I'm making for packaging of my audio server development. There are a few final details to iron out with my engineerings but they will start cutting them from billet in the next couple of weeks with luck (waiting to get a production slot agreed). 

 

The cooler is designed to handle up to 200 watts of dissipation from an LGA3647. This is far higher than is required even for heavily loaded CPU. At 200w you would need a < 0.2C/w rated heat-sink which is a very large passive sink, but this is not really necessary for an audio server  ).

 

Care has been taken to make the heat pipes exit the CPU's footprint area 40mm above the motherboard. This means that  heatpipe routing remains simple and most sensible sized memory DIMMS should be insertable and removable without moving either the cooler or heat pipes. It can be reversed by 180 degrees when fitted meaning cooling pipes can exit over either bank of memory. I dont need this feature but it just seemed a good thing to do. End point clamps for the far end of the heatpipes are also being produced. Material will be engineering grade aluminium with clear hard anodizing to keep it looking pretty as time passes.

 

 

OAudio. 

 

[OAudio]

17 hours ago, Tatomek7 said:

Hi All,

 

I refresh the thread, as I just joined the C621 camp with two LGA3647 processors motherboard and I’m highly interested to exchange the current coolers ( Dynatron B14 ) with passive solution. I was following with big hope @Peter Avgeris involvement, but no news recently. @OAudio, do you think, your cooler has chance to become real? I was looking here around to find the local artisan with right CNC , but with no success at the moment. It seems more complicated, I thought before.

Hi,

 

Details are finalised and the my engineers have a production slot scheduled to cut the coolers and other parts (below) from billet in two weeks time. I still have not decided on colour of the finish either black or clear anodized. I plan to visit the factory to see some finish samples and look at the units in production so will confirm colour then.

 

The rendering (in the above post)  shows the cpu "riser" and its clamp for the securing heat pipes. In addition I am having four half clamps produced per riser. The half clamps are used in pairs clamp three heat pipes and allow them to be bolted to whatever heatsink is to be used for disipating heat away from the server's case. The correct hardware to fixings will be used to bolt the risers to the standard LGA3647 motherboard socket fitting and I went to some lengths in the design to ensure that the plastic CPU retainer that is used on LGA 3647 coolers will also work as normal.

 

I have had to commit to a batch of production in order to get them made which has not been cheap ! There may be a few sets left over that I do not use for my project. Will keep you posted when the physical unit show up in a couple of weeks.

 

OAudio.

[OAudio]

On 7/31/2020 at 12:04 PM, Solstice380 said:

@OAudio The emissivity of black color is better - slightly more heat will be dissipated.

 

Black would work better for heat emission from the CPU cooling block, but the extra heat emitted would be inside the case rather than being transferred out of the case via the heat pipes / case's external heat sinks. I don't think the difference between black v's silver finish should be too great so aesthetics will probably be the decider . 

 

Black would look cool though..... 

[OAudio]

11 hours ago, Downtheline said:

Any luck?

 

The production slot has moved back a week because of a small delay in a prior jobs, machining starts a week on Monday, 

I will be dropping by the factory then to decide the finish and look over some other work.

 

So all good thanks. I will post some updates as things progress.