Thermal Design of Crypto Mining and Multi-GPU Rigs

When using computers with large number of GPU cards, cooling and thermal management quickly becomes a major headache. Here we discuss an approach of building rigs with rudimentary but still quite effective heat management: We create cold and hot zones in each rig and evacuate only the air from the hot zones out of the room. The basic idea is to create a maximum temperature differential between hot and cold, and removing only the hot portion using an exhaust fan such as a bathroom fan in a bathroom.

With 800W of equipment in a bathroom with central A/C and in summer with 80-90 degrees F air temperatures this appeared to be sufficient, but at the cost of increased infiltration of air from outside or from the attic, increasing the load on the central A/C and increasing humidity. Certainly beyond this level of power consumption a dedicated server room dual hose A/C unit will become necessary, which will be the topic of a later post.

Disclaimer: The approach in this post is an approach that may not be suitable for your specific situation. The approach you use personally must be approved by all relevant experts, and you assume all responsibility. Fire hazard is a serious and real risk in this field and you really need to know what you are doing. All equipment has max. temperature ratings that should never be exceeded. Safety devices are recommended to handle equipment failure.

Steps for creating hot-cold zone thermal management:

 [ ] Create cold and hot zone in each mining rig.
 [ ] Capture air from hot zone.
 [ ] Evacuate hot zone air out of the room.
 [ ] Implement temperature limit based equipment shutdown switch.

Multi-GPU rigs with hot and cold zones

We feel that systems with GPUs consuming more than about 200W in total are best built without conventional computer cases. Crates work much better at the expense of density. See our post on building a mining rig in a crate here.

Density will not be a concern as the real limiting factor will typically be the rating of a typical electrical circuit. For instance it could be limited to about 1,875 Watts (15 Amps at 125 Volts). High quality electrical connections are needed to operate at such currents, or heat generated at less then perfect contacts can melt materials and even cause fire.

Air cooled GPUs have fans blowing air over the heat sink, keeping the electronics at reasonable temperatures. Even if ambient temperature is around 90 degrees F, temperatures can be kept below 80 F, but more typically around 70 F. Ideally the hot air would flow towards the end of the card with display connectors, but generally hot air flows in all directions away from the card. This is especially a problem with 100W+ cards.

The hot zone can start at the surface with display connectors, but with larger cards the entire card should be shielded from CPU and other components with another barrier. We constructed zones and barriers at very low cost using UL listed non-flammable tape, and metal wire mesh.

Video card primary hot air exit

The primary separation between hot and cold is at the GPU primary hot air exhaust at the back of the card. The EVGA cards have seen have nice large holes, which is good for air flow. We built our rigs in crates with GPU card ends pointing up, so we just need to cover the top and keep holes for the GPU card ends.

On the left, a top view of a crate with six EVGA 1050Ti GPU cards and a barrier with wire mesh openings just for the card ends blowing hot air. A close-up of one card is shown on the right. Wire mesh was cut out around the connectors for a flush alignment of barrier with card end.

Hot air capturing hood

The second part of the solution is a hood for capturing and holding hot air, and an exhaust system. We built the hood very cheaply using a second crate, and more UL listed metallic duct tape. This crate can be placed on top of the crate with the computer. A small opening at the top of the crate is needed for the exhaust.

A crate with tape to build a low cost hood for collecting hot air.

Evacuation of hot zone air

To help air flow we ended up installing a small 4″ 100CFM, properly temperature rated, duct fan on top of each hood. We used 4″ flexible duct to connect to the bathroom exhaust fan. The basic bathroom fan could handle two or three smaller rigs (total up to about 1,300 W) connected together but not much more. Using 5″ or 6″ ducts would have been better for easier air flow. More tape was used to connect the ducts to the bathroom fan and to ensure that the fan was pulling air only from the duct and not from the room.

Et voila – a cold zone inside the crate with the compute hardware, a hot zone on top of it, and exhaust ducts for evacuation of the hot air.

Temperature Safety shut-off switch

One possible (but certainly not the most reliable) safety measure was to install a wireless temperature sensor in the bathroom fan, and a properly rated and high quality wireless power switch, and monitoring the exhaust air temperature. We used our SmartThings hub to shut down the rigs when temperatures exceeded a certain limit.

The temperature of the collected hot air could be as high as 120 degrees F in case of a room ambient of about 90 F, and GPU temperature of the worst cards (outliers) at 82 F. Therefore sufficient cold air was needed to keep the ambient in the 80-85 F range, for example by keeping the bathroom door open; not an ideal situation.

Considering an Upgrade to Dedicated A/C

This setup without dedicated A/C could work for small server rooms, but a portable A/C is certainly a good addition that becomes a requirement for rooms with more equipment, beyond about 1,000 W in total. However. if you are thinking to install an A/C unit don’t forget that you will need another breaker and circuit for the A/C, and a drain for condensation. The hot air temperature from the A/C can be in the range of 120-130 F and the air flow in the order of 300-400 CFM, which is too much for the bathroom fan. As for the A/C. install one with two hoses for cooling the compressor to minimize infiltration of hot and humid air.