Boom. It conjured the Earl of Pi himself.

Very much so. That is why we did another video while we were up there on how these are tested. Look for that soon.

@@ServeTheHomeVideo I'd Love to see a video on creative solutions data centers have used for cool source water - like loops into a large body of water like an onsite pond or lake, or something like the new "space air conditioning" which basically converts the heat into infrared via special polymer tubing, and irradiates it up into and out of the atmosphere - almost like a reverse solar panel.

Yea we tend to do these boxes before Linus does. E.g. A100 box, we did 3x and including liquid cooled ones a year ago. There is a MI250X node that makes a cameo in this video. That is faster at FP64 than the next-gen NVIDIA H100 GPUs. We just do more informative rather than entertainment aimed at gamers.

Well Linus kinda failed epically with his DIY whole room water cooling back in 2015. It's probably a good idea he didn't get anywhere near this equipment.

And Linus' humour is for kids...

pains me when i see linus building awesome systems just to run games. datacenters throwing and crunching gigs/tera of data to me is what these bad boys are for minus the linus... just ignore my ranting

Cool IT knows that Patrick is less likely to drop stuff, especially after months of having to lift hardware to show it in the new set.

Ahh, now i should use the hot water convert it into steam to generate electricity 🔥

4

Glad you liked it!

Heat from Datacenter should be used to heat homes. How cool would be home colocation of quiet, compact mining rigs during the winter? I know it would be challenging technicaly and perhaps legally (who'd be responsible for what in case of sth goes wrong etc) but just based on ecological and cost of electricity terms it would be a win-win. But realistically, unfortunately it's hard for me to imagine that. Unless doing some centralised heating, like locating many rigs in big boiler rooms that serves buildings or small communities etc.

@@aeonikus1 "Heat from Datacenter should be used to heat homes." I agree. If there is an efficient way to store and transfer the heat. Case in point, older cities in the US used to have a municipal steam generation to provide heat to buildings. There's a reason why they don't do that anymore, and a LOT of that has to do with the fact that even with really heavily insulated pipes, a LOT of that heat is lost during transmission to the surrounding environment, before you even get to your intended building. So, you would have to "overdesign" the heat generation capacity to make up for those transmission losses so that you would actually get the heat that you need, in the building. "How cool would be home colocation of quiet, compact mining rigs during the winter?" I can tell you that with the mining rig that I have at home, it is literally more than enough to heat our ~1000 ft^2 house with only about 1.5 kW "space heater" running (read: mining) 24/7. It's not technically a cost efficient way of heating the house (turning electrical energy into thermal energy) - our natural gas furnance is a LOT more efficient for that purpose, but so long as proof of work crypto mining is still relatively profitable, (i.e. it is enough to cover the cost of electricity), then it is still a net profit PLUS heating the house. In fact, we didn't turn off our A/C until WELL into December, and we had to turn it back on I think as early as like February this year. (And the electrical costs that I am talking about is the cost of my electrical bill and NOT just the mining rig itself - so it's both mining and cooling/cost of running the A/C starting in February as well.) "Unless doing some centralised heating, like locating many rigs in big boiler rooms that serves buildings or small communities etc." So....yes and no. There are some data centers that actually don't bother with chilling the incoming air (or water if they're using liquid cooled servers), but the problem is that then in the data center, you end up with hot spots which lowers the computational efficiency of the systems themselves. So, the entire thermal management loop needs to be optimised so as to optimise the performance of the individual systems whilst allowing them to get at close to the "hot" temperature threshold without exceeding it so that you're also not spending a lot of energy cooling the incoming air/liquid supply (which just means that you need something else to be able to dump the heat somewhere else). As a computational infrastructure problem, it's actually quite challenging to hit that balance over a wide range of environmental and operating conditions.

Glad you enjoyed it

Ha

That is why we did it in the lab

There are leak detectors people use. We also have another video with a tour of the lab coming to help show what is done to test and build reliable liquid cooling

You can also just use a non water cooling fluid like Galden HT200 in your primary loop and move your water cooled secondary loop away from your electrical equipment.

That is why the next video is going to be on the Liquid Lab we are in and how they test these systems. Starting later this year, not having liquid cooling is going to mean that the data center will only host lower performance/ lower density servers. We have been doing more on server liquid cooling over the past year to start pushing this concept so our readers/ viewers are ready for the transition.

I selfishly snuck out to Lake Louise the Saturday after we filmed the two videos in this series. I am a big fan

It is actually super hot underneath. :D

Are you thinking the -ZL1 with the RAM and NIC? In the next video (and there was a cameo in this video) there is a MI250X node that is fully liquid cooled.

@@ServeTheHomeVideo Correct! Oh that’s awesome! Looking forward to the next video! To be fair I always look forward to your content. This because what you guys share with us is quite unique here on KZfaq so immensely appreciated! 👊👌🤓

It is not standard yet, but the hyperscalers are looking at it or deploying it already

Heck yes! Back in the US now but hopefully Canada again soon

The exchange is basically a normal radiator but instead of air flowing through it they flow water through it. That outside water (now warm) goes back to the datacenter cooling system, through the chillers (large AC equipment designed to cool the water) and back into a storage/buffer tank to then be pumped right around again and again. This sort of equipment is usually already mostly in place at most datacenters as they provide the cooling used by the air conditioning.

CoXuS.

Immersion is still going forward. Just two different technologies. Using direct to chip liquid integrates easily into existing racks.

It was -16C the night before filming this!

Each node has access to one quarter of the front drive bays

@@ServeTheHomeVideo thanks!

If you see our PhoenixNAP data center tour you can see those.

@@ServeTheHomeVideo went and watched that video and it reminds me of a lot of specialty installations that were shown on der8auers tour through a Hetzner data center with like the diesel generators and raised floor.

I used to say it as well not I T.

Id imagine so many people have mispronounced it that both are acceptable to the company now.

We do not use the "L-word" in the lab. :-) But the next video that we recorded there is how they test in the lab to make sure no L-word events happen.

To make that work would require the water to be heated to well over 100C, which means that whatever is heating the water would also have to be well over 100C.

@@gloop6589 I think you're right, but surely you could use a heat pump to focus the heat? Things like sterling engines can sap energy out of much less heat so I'm sure you could do some energy recovery...

There are several projects looking at re-using the heat from servers for heating water and building heat applications. Steam less so.

I'd think that generating an appreciable amount of electricity would thermodynamically require a sufficient temperature differential to ambient temperature, which is the exact thing that we're trying to minimize when cooling a datacenter. Could be wrong though.

Usually there are chillers outside the data center floor at the facility. You can see them in our PhoenixNAP data center tour video.

I think CoolIT was using RPi's because they were easy to source. I also think they are being phased out on newer products. This was also an older version of the CHx80 because we were running it on its side.

Energy effiency is the thing that matters, it is going up.

transistor count will always go up exponentially. but efficiency limits the increase in power draw. They are as efficient as they get right now. you have to consider both aspects

liquid cooling is used in industrial applications (and vehicles) 24/7 with no problems. The point here is what grade is the equipment. Consumer liquid cooling, heck no. They are using industrial-grade stuff here so it is fine