Why are we double glazing our servers?

It is well known that one of the major contributors to data centre inefficiency is the cooling plant. Most data centres are air conditioned and this is accepted practice, with Air Conditioning bodies recommending the operating ranges but why is air conditioning necessary at all?

The major heat emitting components, the processor, memory and mainboard chipsets are happy to run with internal temperatures of up to 80C. We also know that heat will naturally move from a hotter material to a cooler one such as the external air surrounding a data centre.

Double glazing

The problem we have is that the servers are insulated from the coolant by an air gap in much the same way as we insulate our homes with an air gap between two sheets of glass.

Traditional data centre

As seen in the diagram below, a traditional “optimised” air flow data centre relies on a chiller plant. The heat manages to get from the processor core out through the case and into the heat sink quite effectively but there is then a very effective insulating air gap between the heat sink and the chilled water system. The chilled water system is so well insulated that it is below the external air temperature for most of the time and requires a chiller to increase the coolant temperature to a level where it can be rejected to external air. This insulating air gap is so effective that it costs the data centre 40+ degrees C.

However, we know that many operators are running data centres either at higher temperatures or as “chiller-less” designs. How do these designs look?

Contained air flow data centre

Higher temperature and chiller less data centres mostly use air flow containment to enable them to avoid supply air temperature rise through supplied air mixing with heated return air. This enables the data centre to supply air at a higher temperature without actually increasing the server intake temperature. This increase in supply air temperature provides much greater opportunity to use economiser technology. The diagram below shows a more modern, contained air flow design which is able to make some use of an economiser but still requires a chiller system.

Even this contained air flow design, with its much more predictable behaviour and higher supply air temperature loses 30+ degrees C to the insulating air gap.

What about close coupled?

What these diagrams show us is that air is the major problem with high power density compute. Whether the cooling fluid is in the rack with the servers or 20 metres away in a CRAC unit, the insulating air prevents us getting the our of the building efficiently and means that we need to spend money and energy on installing and operating an expensive chiller plant.

What is the alternative?

If we take the insulating air gap out of our design and deliver the coolant fluid (it doesn’t need to be water) directly to the chip then we have a fundamentally different system. We can get the heat out of the silicon and into the cooling fluid (whatever that is) at a temperature high enough to send directly to an external cooler without requiring any chiller plant at all as shown in the final diagram below.

As this diagram shows, it is those final few inches taking the coolant fluid to the source of the heat that make the big difference and allow us to run the data centre without the burden of huge chiller plant.

But they don’t make servers like that

The rising cost of both power and the cooling infrastructure taken against the falling cost of commodity servers is making this sort of technology viable. There are several companies making these sort of direct cooling solutions, there is the relatively well known Spraycool, http://www.spraycool.com/ and the newer Iceotope (nothing on their website yet but you can watch an intro video here) who are both bringing this technology to a rack near you.

I don’t want to work in an oven

There is one other upside of taking all the heat out of our servers in a fluid, we won’t be having to work in a ‘hot aisle’ at 40+C behind a wall of blade servers. Once the majority of the waste heat is in the fluid loop we can run the data floor at air temperatures that are suitable for engineers to work in.