Passive CPU Cooling - High Power CPU

[OAudio]

The time has come to deal with cooling of the c621 server I am working on. For the usual SQ and ambient noise reasons the server must to be entirely passively cooled. The cooling solution must deal with the heat dissipation requirements of a high power Xeon SP series CPU and it has to integrate into my server's quite dense packaging of power electronics in the area of the CPU. Of course there are no off the shelf passive cooling solutions for LGA 3647 socket CPUs 😦. 

 

Some time ago I posted details of the CPU cooling solution I had in mind.

 

 

On 7/13/2020 at 6:00 PM, OAudio said:

Hi All,

 

This is a rendering of one of the parts I'm making for packaging of my audio server development. There are a few final details to iron out with my engineerings but they will start cutting them from billet in the next couple of weeks with luck (waiting to get a production slot agreed). 

 

The cooler is designed to handle up to 200 watts of dissipation from an LGA3647. This is far higher than is required even for heavily loaded CPU. At 200w you would need a < 0.2C/w rated heat-sink which is a very large passive sink, but this is not really necessary for an audio server  ).

 

Care has been taken to make the heat pipes exit the CPU's footprint area 40mm above the motherboard. This means that  heatpipe routing remains simple and most sensible sized memory DIMMS should be insertable and removable without moving either the cooler or heat pipes. It can be reversed by 180 degrees when fitted meaning cooling pipes can exit over either bank of memory. I dont need this feature but it just seemed a good thing to do. End point clamps for the far end of the heatpipes are also being produced. Material will be engineering grade aluminium with clear hard anodizing to keep it looking pretty as time passes.

 

 

OAudio. 

 

 

 

 

As the server's design and packaging developed there were opportunities to refine the cooling design. The final CPU block design for instance no longer routes heat pipes out over the motherboard's RAM so it will be simpler fit. The block and clamps are produced now, they are CNC'd from billet and are pictured below with just anodizing is outstanding. 

 

 

 

 

Defiantly looking forwards to getting these fitted, I will update on how installation and testing goes. 

 

OAudio. 

 

  

[Tatomek7]

Hi,

I was waiting for the “production stage” of your idea and it is 😛

 

I’ve tried to move forward initiative of @Peter Avgeris sending him RAM chips, but since then no news. 
Fingers crossed, as I have active cooling of my C621 and fans are not noiseless  😡

[R1200CL]

...you know you can place the whole computer in a aquarium of oil 😉

[Dev]

@OAudiolooks grt8. Are you trying to build a product to sell or is it for your own use only ? How about the chassis ? There is some interest in this chipset and the Asus sage c621 motherboard with some of us and hence curious what you are upto.

[J.J]

If you are willing to sell, I would like to buy

[Exocer]

Interested in a pair as well if you do decide to mass produce these :)

Which case did you intend to use these with?

[gerardiw]

I would also be interested in these for the Asus motherboard. Looks promising!

[R1200CL]

On 11/28/2020 at 6:41 PM, OAudio said:

Of course there are no off the shelf passive cooling solutions for LGA 3647 socket CPUs 😦. 

 

https://www.coolerguys.com/collections/intel-xeon-platinum-lga-3647/passive-cooler

http://www.akasa.com.tw/update.php?tpl=product/cpu.product.tpl&no=181&type=CPU coolers&type_sub=Low Profile&model=AK-CC9203

[OAudio]

34 minutes ago, R1200CL said:

https://www.coolerguys.com/collections/intel-xeon-platinum-lga-3647/passive-cooler

http://www.akasa.com.tw/update.php?tpl=product/cpu.product.tpl&no=181&type=CPU coolers&type_sub=Low Profile&model=AK-CC9203

Hi R1200CL,

 

These are great coolers.

I use the Supermicro versions in 2u size. Unfortunatly they are not efficient enough to the cool 3647 cpus passively.  They are designed for server cases with fans and ducts to make sure the coolers see some forced air moving through them.

 

Whilst working on the server for the moment I am still using the Supermicro versions of these coolers with electrically isolated fans. It's far easier to strip and reassemble the server with the Supermicro 2U coolers than with the new CNC coolers and heat pipe system installed. 

 

OAudio.

[botrytis]

You also have to understand server farms are kept Cool, like 60F and have high rate of air flow through (more than 4 volumes of air per hour changed). 

[vhs]

On 11/29/2020 at 1:41 AM, OAudio said:

Defiantly looking forwards to getting these fitted, I will update on how installation and testing goes. 

Hi there, any news yet ?

[Nenon]

4 hours ago, vhs said:

Hi there, any news yet ?

 

Just in case you have missed the news that Taiko is working on a case already:

 

On 1/7/2021 at 5:05 PM, Nenon said:

Here is the first draft of the DIY computer chassis by Taiko. 

 

 

 

 

 

 

It's big! Bigger than the Extreme chassis actually. Thick panels, high quality, no flimsy stuff... the thinnest panel is 6 mm.

It can handle any motherboard. You can see the dual CPU Asus Sage motherboard fitted in the last photo. That is the biggest motherboard I've used. And there is so much space around it for a a power supply. 

 

Emile and I had a long discussion regarding the size of the chassis. It could be smaller. And that would be 20-30% cheaper. But this is DIY, and it would be excellent to have some extra space to work with inside the chassis. 

 

The cooling capabilities of this chassis would be extraordinary. That's another reason for the decision to make it so big. It should be able to properly cool down any consumer CPU with some headroom if we utilize both heatsinks. But even if you don't cool down crazy CPUs, keeping your CPU and everything inside the chassis a few degrees lower typically leads to better sound.

 

There will be CPU coolers (at least for the Xeon CPUs and the SAGE motherboard) and heat pipes included. Perhaps for other CPUs in the future too. 

 

We can fit full height PCIe cards. That would be great for those of us who prefer PCIe based storage. We can fit one or more Asus Hyper M.2 cards with NVME drives inside. 

 

There will be an opening on the back that we can fit with a plate that has either a DC connector or a IEC inlet / fuse. There will also be ventilation holes on the bottom panel... these are not reflected on the pictures. An LED on the front panel and power button on the back. 

 

We would like to hear your comments. But don't ask about price. It's too early. Taiko is not doing this for profit. But good quality costs money, so don't expect it to be a couple hundred dollars. I had to buy 4 x HDPlex H5 cases to build my DIY coolers for those Xeon CPUs. That is a $1,200 expense plus a lot of work and not even close to this passive cooling solution.

 

The main reason Emile is doing this is to enable the DIY community to build a proper dual CPU server, so we can explore new discoveries everyone would benefit from. Just like we did in the legendary Novel thread not so long ago. It's interesting how many current manufacturers were inspired or influenced by the Novel thread and used ideas from there! There was a time when DIY was ahead of the commercial sector. 

 

Also, I know there are a lot of opinions on what CPUs should be used, CUDA offloading and cooling, etc. etc. Let's stay focused on this project. This is not designed to be an universal audio / home theatre chassis like the Hdplex and Streacom cases that should be able to accomodate any computer. The purpose of this chassis is to go with a specific DIY recipe utilizing the dual CPU Asus SAGE motherboard for now. If we find something that sounds better, it can be tweaked in that direction. There is plenty of space inside for anything and plenty of cooling capacity.

 

There is one particular question we are wondering about. For those who will be following this DIY recipe and building this computer - you are willing to pay 20-30% more for the extra space in a bigger chassis like this. 

 

There is of course plans to fill that space with a state of the art power supply. That's something that is still in the works and will be a part of the complete DIY recipe. It seems like there is a lot of excellent technology Taiko is willing to share with us. 

 

[gerardiw]

I was on the list for the turemetal case last year which did not result in a solution but I really like what you have worked on and presented here so I'd pay a little extra for the bigger case, taking the view that this would be a long-term purchase.

[rossb]

Is passive cooling really necessary? The fans on my Noctua NH-D15 are currently spinning at about 200rpm and are completely silent.

[Nenon]

1 minute ago, rossb said:

Is passive cooling really necessary? The fans on my Noctua NH-D15 are currently spinning at about 200rpm and are completely silent.

 

It's not about the mechanical noise from the fan as much as vibration. If you are building a $10K server, you most likely have a very resolving system and can clearly hear the effects of different vibration treatment under the server. All these clocks and capacitors are super sensitive to vibrations that we can't even feel. Ask some hardcore headphone users - they use vibration treatment too, even though they don't have huge speakers blasting sound waves. 

 

Adding fans to a computer of that class is like buying an expensive telescope to watch the stars and using it from the car while driving on a dirty road. Sorry, I could not come up with a better example. In simple words, it's completely unacceptable.

[OAudio]

51 minutes ago, rossb said:

Is passive cooling really necessary? The fans on my Noctua NH-D15 are currently spinning at about 200rpm and are completely silent.

Hi Rossb,

 

There is a lot of dogma about fans which really makes me 🙂sometimes. You would think they are a complete disaster the amount of attention that "fan-less" setups get, however the truth is that if you: 

 

• run them slow so you are happy / can live with the with the sound noise levels,
• isolate them from the servers sensitive power (don't connect via the mother board headers),
• make sure that the fans ground is not referenced to the servers supplies,
• run fan wiring well away from the servers wiring and the motherboard etc,

the sound quality will be absolutely fine.

 

Sounds like a small auxiliary supply for your fans could be a good thing to try as well. 

 

Your fan setup is similar to the setup I am using to develop with here. The fans are electrically isolated from the server's power supplies for good measure. In a really high resolution system the set up is very good for sound quality, the impact of turning the fans off completely can just be heard but frankly its not much of a difference.

 

Last thought is that for DIY a passive heat pipe setup needs careful consideration. There are benefits in terms of physical sound levels from the server and "some" sound quality improvement to be had BUT you may lose the ability to easily move the motherboard from the chassis due to the complication of the pipe system. Not a problem ? possibly, but I need access to the motherboard to do "stuff" 🙂 which means its been in and out of the server 30-40 times at least.  

 

OAudio

 

 

 

 

 

 

     

[kyoya78]

46 minutes ago, rossb said:

パッシブ冷却は本当に必要ですか？私のNoctuaNH-D15のファンは現在、約200rpmで回転しており、完全に無音です。

 

I replaced one of the CPU coolers with my own (A7075). The thin fins are excellent for cooling but still have a negative effect on the sound. I was surprised at the improvement in sound quality even after replacing one of the CPU coolers.

[OAudio]

On 1/27/2021 at 5:12 AM, vhs said:

Hi there, any news yet ?

Hi VHS,

 

Thanks for the interest, yes things have moved forwards. I wanted to see  if the cooling system performed as per the thermal design I did. I couldn't wait any longer so I did do a trail fit a week ago. They are just about bang on the thermal design which was really nice to see  and they look absolutely superb in the chassis.  The coolers were made by an engineer with decades of work for a jet engine manufacturer, and it really shows. Its such a shame to shut them away inside the server chassis.

 

They are back out of the server again and the fans are back in for now for the access reasons I mentioned above.

 

Here is another picture of them with CPU mount attached.

 

OAudio.

 

 

 

 

[OAudio]

6 minutes ago, kyoya78 said:

 

I replaced one of the CPU coolers with my own (A7075). The thin fins are excellent for cooling but still have a negative effect on the sound. I was surprised at the improvement in sound quality even after replacing one of the CPU coolers.

 

 

Hi kyoya78,

 

What a difference to the radiator on the other CPU. 

That's a nice looking CPU block !

[rossb]

10 hours ago, Nenon said:

 

It's not about the mechanical noise from the fan as much as vibration. If you are building a $10K server, you most likely have a very resolving system and can clearly hear the effects of different vibration treatment under the server. All these clocks and capacitors are super sensitive to vibrations that we can't even feel. Ask some hardcore headphone users - they use vibration treatment too, even though they don't have huge speakers blasting sound waves. 

 

Adding fans to a computer of that class is like buying an expensive telescope to watch the stars and using it from the car while driving on a dirty road. Sorry, I could not come up with a better example. In simple words, it's completely unacceptable.

 

I'm far from being an objectivist, but I think a lot of these statements are articles of faith more than anything else.

 

I have an extremely resolving high end headphone setup - An Nagra HD DAC/Classic PSU, EAR HP4 and Niimbus US4 headphone amps and Audeze LCD4 headphones (amonst others).

 

I have tried completely fanless audiophile servers run on linear power supplies - an Antipodes DX3, for example - and found that my high end gaming PC with its i9 processor and about 12 fans connected to the motherboard and ATX PSU sounded better.  This is consistent with my experience in general that higher powered PCs sound better than lower powered ones, even where the high powered PC is run with a conventional switching PSU. In fact, I once built a mid range server with an i5 processor and an HD Plex linear PSU, and it sounded much worse with the linear PSU than with a conventional ATX power supply.

 

So my experience is that having a very high powered PC which necessarily requires fans to run at acceptable temperatures, even with a conventional switching power supply, sounds considerably better than a lower powered fanless one. I have built and used PCs with fans and without fans. The processing power of the CPU in my experience has a far bigger impact on sound quality than a few fans.

 

[Nenon]

9 hours ago, rossb said:

The processing power of the CPU in my experience has a far bigger impact on sound quality than a few fans.

That is my experience as well. But with everything else equal removing the fan is another step up.

There is also another factor - keeping the temperature lower. So a good active cooling solution that keeps the components cool might work better than a bad passive solution that can't keep up. It's the sum of all parts at the end. 

 

I currently use two 85W TDP CPUs in a HDPlex H5 chassis, one connected to each heatsink. I would say that is the limit of the Hdplex. The chassis from Taiko would be able to handle a lot more than that. The bottleneck in heat dissipation would actually be the heatpipes, not the coolers or the heatsinks. I am expecting to be able to passively cool down two 100-120W TDP CPUs, but that is to be determined. If the heatpipes were not a bottleneck, I would have said 200W+. 

 

I agree that there are many other things that make a bigger difference. But as I go through every detail on my music server and sort out all the things that make a bigger difference, there is a point I reach where replacing the active cooling with a good passive cooling is the next logical step. Once you get to that point it's a really nice improvement. 

 

In other words, don't take my comment out of context. When I said "If you are building a $10K server..." I implied that attention to every detail has been taken. $10K is a lot of money for a DIY server! Passive cooling is not the most important aspect, but at some point it becomes quite important factor, and easy to hear. That point is probably closer to $2K... at 5 times that price, quoting myself "it's completely unacceptable" to have fans. 

[firedog]

I replaced a high powered fanless server (with an a fancy external PS) with an even more high powered server with active cooling: Internal "low noise fan" computer PS. Large, slow, virtually silent case and motherboard fans. Soundproofing material on inner walls of the case.

 

Result: the active server is actually quieter than my fanless server (I could sometimes hear the storage HDD on the fanless server); I can't hear the new server at all-even from half a meter away. It also runs much cooler than the fanless server. The fanless server often got pushed to it's limits in cooling, and I worried it would fail. The actively cooled server never gets even close to that level of internal heat.

 

SQ: I don't hear any degradation in SQ.

[vhs]

On 1/27/2021 at 5:20 PM, Nenon said:

 

Just in case you have missed the news that Taiko is working on a case already:

 

 

Thx Nenon,

 

Please let me know once the case is available.

 

 

[seeteeyou]

On 1/30/2021 at 4:02 PM, Nenon said:

I currently use two 85W TDP CPUs in a HDPlex H5 chassis, one connected to each heatsink. I would say that is the limit of the Hdplex. The chassis from Taiko would be able to handle a lot more than that. The bottleneck in heat dissipation would actually be the heatpipes, not the coolers or the heatsinks. I am expecting to be able to passively cool down two 100-120W TDP CPUs, but that is to be determined.

 

Good enough for Xeon Silver 4309Y then

 

https://wccftech.com/amd-3rd-gen-epyc-milan-intel-3rd-gen-xeon-ice-lake-sp-server-cpu-lineup-detailed/

[vhs]

https://wccftech.com/nvidia-geforce-rtx-3080-goes-all-passive-cooling-87c-temps-turemetal-up10-chassis/