Cooling rack-based equipment - GoFuckYourself.com

edgeprod · 12-24-2013, 10:57 AM

Server guys, your time to shine!

For racks of bare components (motherboards, video cards, power supplies), what are some good cooling methods? Can I make the rack out of thick aluminum? The video cards are already on risers, and the machines are built into milk crates for airflow, but the footprint is sub-optimal. I want to be able to put them in racks that maximize cooling while minimizing square footage in the data center cage (which is how I am billed).

Any and all suggestions welcome.

American Psycho · 12-24-2013, 01:11 PM

Pour cold water on it.

SplitInfinity · 12-24-2013, 02:06 PM

Submerge boards and power supplies in non-conductive cooling oil. That's how lots of the bitcoin miners are doing it. We have a rig here with a radiator cycling oil through it and the tank has many many ASIC miners.

livexxx · 12-24-2013, 02:48 PM

Rack vertically and have the floor tile off to allow the air in, also put vents on the side to suck the neighbours cool air in

FlowerKid · 12-24-2013, 05:59 PM

Why do you need video cards in servers?

Zeiss · 12-24-2013, 06:36 PM

Quote:

Originally Posted by FlowerKid

Why do you need video cards in servers?

Yes, it's possible to run xserver without a graphics card

edgeprod · 12-24-2013, 08:10 PM

Quote:

Originally Posted by American Psycho

Pour cold water on it.

I'm going to pass on that, thank you though.

Quote:

Originally Posted by SplitInfinity

Submerge boards and power supplies in non-conductive cooling oil. That's how lots of the bitcoin miners are doing it. We have a rig here with a radiator cycling oil through it and the tank has many many ASIC miners.

This is a great idea, thank you very much. I'm going to see how feasible this is (datacenter may not want liquids in proximity to other machines).

Quote:

Originally Posted by livexxx

Rack vertically and have the floor tile off to allow the air in, also put vents on the side to suck the neighbours cool air in

This is what we're doing right now. It's slowing the graphics cards down to be as warm as they are. With just a couple of machines, it's fine, but when you start adding more, it just gets to the point where it's too warm. I want to solve the problem before it really becomes one.

Quote:

Originally Posted by FlowerKid

Why do you need video cards in servers?

They're not servers. Each machine has three Radeon R9 290 4GB GDDR5 in it, all running 24/7, all overclocked. The oldest machines have Radeon 7950's, but I am phasing those out (anyone buying 7950 cards?) in favor of the 290's (650 kH/s versus 850 kH/s).

Quote:

Originally Posted by Zealotry

Yes, it's possible to run xserver without a graphics card

-- I've seen people try to install cgminer on server-based Linux platforms, forgetting that they need the X components. Hilarity ensued.

freecartoonporn · 12-24-2013, 10:05 PM

hope this helps.

btw how many gh/s you getting? or is it th/s ?

edgeprod · 12-24-2013, 10:39 PM

Quote:

Originally Posted by freecartoonporn

hope this helps.

btw how many gh/s you getting? or is it th/s ?

All joking aside, fans like that worked up to a certain point for me.

johnnyloadproductions · 12-24-2013, 10:40 PM

What are you planning to do?

Glad to see you try new things...

Hope it's not mining http://m.techspot.com/news/55111-ins...n-iceland.html

edgeprod · 12-24-2013, 11:05 PM

Quote:

Originally Posted by johnnyloadproductions

What are you planning to do?

Glad to see you try new things...

Hope it's not mining http://m.techspot.com/news/55111-ins...n-iceland.html

New things? Where do you get that from?

And who the fuck mines Bitcoins anymore?

johnnyloadproductions · 12-24-2013, 11:11 PM

Quote:

Originally Posted by edgeprod

New things? Where do you get that from?

And who the fuck mines Bitcoins anymore?

The people in the story I linked to

Have a merry christmas

edgeprod · 12-24-2013, 11:13 PM

Quote:

Originally Posted by johnnyloadproductions

The people in the story I linked to

Have a merry christmas

Haha, fair point! I should rephrase: who the fuck GPU-mines Bitcoin anymore? Idiots, that's who.

When ASICs arrived, that shit shut down.

DAMNMAN · 12-25-2013, 09:49 AM

Quote:

Originally Posted by SplitInfinity

Submerge boards and power supplies in non-conductive cooling oil. That's how lots of the bitcoin miners are doing it. We have a rig here with a radiator cycling oil through it and the tank has many many ASIC miners.

We looked at this option, but some cables and wiring are melted by mineral oil. Also Harddrives need to be kept out of it. Resale of components is impossible after being submerged in mineral oil. Fluorine inert liquid is prohibitively expensive.
Etc........

DAMNMAN · 12-25-2013, 09:51 AM

Quote:

Originally Posted by FlowerKid

Why do you need video cards in servers?

Yer, joking right?

DAMNMAN · 12-25-2013, 09:52 AM

Quote:

Originally Posted by Zealotry

Yes, it's possible to run xserver without a graphics card

It ain't possible to mine LTC without video card unless you use FPAC and ASIC ain't available yet.

edgeprod · 12-25-2013, 04:32 PM

Quote:

Originally Posted by DAMNMAN

We looked at this option, but some cables and wiring are melted by mineral oil. Also Harddrives need to be kept out of it. Resale of components is impossible after being submerged in mineral oil. Fluorine inert liquid is prohibitively expensive.
Etc........

That's what I found when I looked into it yesterday also. A shame, but it seems like a dead end, at least right now.

Grapesoda · 12-25-2013, 05:13 PM

Quote:

Originally Posted by edgeprod

Server guys, your time to shine!

For racks of bare components (motherboards, video cards, power supplies), what are some good cooling methods? Can I make the rack out of thick aluminum? The video cards are already on risers, and the machines are built into milk crates for airflow, but the footprint is sub-optimal. I want to be able to put them in racks that maximize cooling while minimizing square footage in the data center cage (which is how I am billed).

Any and all suggestions welcome.

how about a cabinet with AC?

edgeprod · 12-25-2013, 05:36 PM

Quote:

Originally Posted by Grapesoda

how about a cabinet with AC?

The form-factor is weird, sadly.

Grapesoda · 12-25-2013, 06:02 PM

Quote:

Originally Posted by edgeprod

The form-factor is weird, sadly.

that's too bad... get a rather larger fridge for $1700

put the HD's in the freezer and the rest in the fridge

http://www.amazon.com/Whirlpool-GX5F...s=refrigerator

rowan · 12-25-2013, 06:19 PM

Quote:

Originally Posted by DAMNMAN

We looked at this option, but some cables and wiring are melted by mineral oil. Also Harddrives need to be kept out of it. Resale of components is impossible after being submerged in mineral oil. Fluorine inert liquid is prohibitively expensive.
Etc........

The other thing to be aware of with liquid is possible wicking, where the oil travels along the cable (against gravity) and ends up depositing itself at the other end... in your power supply, ethernet switch, etc.

One cooling method I am currently experimenting with is fast moving fresh air. Filtered inlet at ground level, m/b and GPUs in a sealed box, big-ass fan exhausting out via the roof space. The idea being that even with high outside ambient temps the airflow is still sufficient to draw the heat away from the GPUs. As a bonus it means you do not need to cool the entire room. (Obviously this sort of thing is not possible in a data centre.)

rowan · 12-25-2013, 06:23 PM

One other thought. Is the D/C going to be happy about letting you install extreme heat producing equipment that you've self-assembled into a milk crate?

DAMNMAN · 12-25-2013, 06:38 PM

Quote:

Originally Posted by DAMNMAN

It ain't possible to mine LTC without video card unless you use FPAC and ASIC ain't available yet.

Mistype FPGA

DAMNMAN · 12-25-2013, 06:40 PM

Quote:

Originally Posted by rowan

The other thing to be aware of with liquid is possible wicking, where the oil travels along the cable (against gravity) and ends up depositing itself at the other end... in your power supply, ethernet switch, etc.

One cooling method I am currently experimenting with is fast moving fresh air. Filtered inlet at ground level, m/b and GPUs in a sealed box, big-ass fan exhausting out via the roof space. The idea being that even with high outside ambient temps the airflow is still sufficient to draw the heat away from the GPUs. As a bonus it means you do not need to cool the entire room. (Obviously this sort of thing is not possible in a data centre.)

This is the best path we've found so far as well. = No Data Center

Due · 12-25-2013, 06:55 PM

Your datacenter should be able to supply sufficient cooling, you just need to design it so air can flow.
If you have airflow it might be better with less airspace since it well be easier to maintain the flow.
Look on eBay or similar for refurbished server hardware, you can buy some older units cheaper than what the chassis would cost you.
I know in the past people that was looking for GPU power used a modified PlayStation (or possible Xbox can't remember).

edgeprod · 12-26-2013, 12:25 PM

Quote:

Originally Posted by rowan

One other thought. Is the D/C going to be happy about letting you install extreme heat producing equipment that you've self-assembled into a milk crate?

They weren't crazy about it when I first started, but became more comfortable the more bills that got paid on time.

Looking at two different data centers now, so I'm going to have to do the whole song-and-dance again, it looks like. Not using milk carts in the new data centers, though, looking at using these:

https://openrigs.com/arianna-grid-gpu-frame

12-24-2013, 10:57 AM	#1
edgeprod Permanently Gone Industry Role: Join Date: Mar 2004 Posts: 10,019	Cooling rack-based equipment Server guys, your time to shine! For racks of bare components (motherboards, video cards, power supplies), what are some good cooling methods? Can I make the rack out of thick aluminum? The video cards are already on risers, and the machines are built into milk crates for airflow, but the footprint is sub-optimal. I want to be able to put them in racks that maximize cooling while minimizing square footage in the data center cage (which is how I am billed). Any and all suggestions welcome.

12-24-2013, 02:48 PM	#4
livexxx Confirmed User Industry Role: Join Date: May 2005 Location: UK Posts: 1,201	Rack vertically and have the floor tile off to allow the air in, also put vents on the side to suck the neighbours cool air in __________________ http://www.webcamalerts.com for auto tweets for web cam operators Last edited by livexxx; 12-24-2013 at 02:50 PM..

12-24-2013, 05:59 PM	#5
FlowerKid Confirmed User Join Date: Sep 2005 Location: Zurich Posts: 1,045	Why do you need video cards in servers? __________________

12-24-2013, 10:05 PM	#8
freecartoonporn Confirmed User Industry Role: Join Date: Jan 2012 Location: NC Posts: 7,683	hope this helps. btw how many gh/s you getting? or is it th/s ? __________________ SSD Cloud Server, VPS Server, Simple Cloud Hosting \| DigitalOcean Last edited by freecartoonporn; 12-24-2013 at 10:10 PM..

12-25-2013, 06:55 PM	#25
Due Confirmed User Industry Role: Join Date: Mar 2001 Location: Murrieta, CA Posts: 3,620	Your datacenter should be able to supply sufficient cooling, you just need to design it so air can flow. If you have airflow it might be better with less airspace since it well be easier to maintain the flow. Look on eBay or similar for refurbished server hardware, you can buy some older units cheaper than what the chassis would cost you. I know in the past people that was looking for GPU power used a modified PlayStation (or possible Xbox can't remember). __________________ I buy plugs Skype: Due_Global /Due

12-24-2013, 01:11 PM	#2
American Psycho Confirmed User Industry Role: Join Date: Feb 2009 Posts: 3,068	Pour cold water on it.

12-24-2013, 02:06 PM	#3
SplitInfinity Confirmed User Join Date: Dec 2002 Location: San Diego, CA Posts: 3,047	Submerge boards and power supplies in non-conductive cooling oil. That's how lots of the bitcoin miners are doing it. We have a rig here with a radiator cycling oil through it and the tank has many many ASIC miners.

12-24-2013, 10:40 PM	#10
johnnyloadproductions Account Shutdown Industry Role: Join Date: Oct 2008 Location: Gone Posts: 3,611	What are you planning to do? Glad to see you try new things... Hope it's not mining http://m.techspot.com/news/55111-ins...n-iceland.html

12-25-2013, 06:23 PM	#22
rowan Too lazy to set a custom title Join Date: Mar 2002 Location: Australia Posts: 17,393	One other thought. Is the D/C going to be happy about letting you install extreme heat producing equipment that you've self-assembled into a milk crate?