From Tim.Tenhave@compaq.com Wed Jan 10 16:56:10 2001
Date: Wed, 10 Jan 2001 16:24:25 -0500
From: "Tenhave, Tim" <Tim.Tenhave@compaq.com>
To: 'Paul Hoffman' <paul@pxh.oac.ucla.edu>, beowulf@beowulf.org
Subject: RE: Rack mounted systems.

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "US-ASCII" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hi Paul,

Don't forget power needs.  Don't forget cooling needs == power needs.

You can get X86 2P boxes in 1U form factors.  Many venders sell them. You
can place up to 40-42 units in most tall 19" wide cabinets offered.  This
would be 84 processors in a cab, meaning about 7 cabinets in a fully loaded
rack situation.  (Note: each cabinet with many boxes, side panels and doors
could then be considered a "heat" chimney of sorts.)

The down side is you need to figure networking, if the nodes are simply to
cramped, and other needs so you may blow out to 10 or 12 cabinets to do this
cluster.  Given the 19" form factor rack is about 21-22" wide with side
panels and it is about 30-32" deep, you get the feel for floor space of one
rack. 

When I mentioned power above, don't forget each node has a power cord.  The
cords need to be grouped and the "grouped" cord needs to go to some sort of
power outlet box.  Either way you do it, you will have many power outlets
used as well.

This is just a few thoughts...

Tim

_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf

From glindahl@hpti.com Wed Jan 10 18:07:06 2001
Date: Wed, 10 Jan 2001 16:59:51 -0500
From: Greg Lindahl <glindahl@hpti.com>
To: Paul Hoffman <paul@pxh.oac.ucla.edu>
Cc: beowulf@beowulf.org
Subject: RE: Rack mounted systems.

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "US-ASCII" character set.  ]
    [ Some characters may be displayed incorrectly. ]

> - what is the minimum width of a 2-way node? (1U, 2U?)

You can buy 1U nodes. 2U nodes are still cheaper. You can also buy half
depth cases such as the ones from rackable.com and get 2 2U cases in 2U, or
2 1U cases in 1U. racksaver.com seems to be selling 3 systems (potentially
dual intels) mounted in 1U.

> - how many nodes can reasonably be put in a standard
>   10" rack? i.e. can the rack be fully loaded or is
>   some spacing required?

If correctly designed, no spacing is needed. Note that a standard rack is
19" (typo) and they come in a variety of heights. Normal is 42U, I think,
but I've purchased as tall as 50U.

At that many nodes, you should check how much excess air conditioning you
have in the room. When you build a machine room you guess how much cooling
you'll need, usually set by the volume. If you then put in unusually dense
computers, you'll run out of cooling.

-- g


_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf

From joelja@darkwing.uoregon.edu Wed Jan 10 18:07:19 2001
Date: Wed, 10 Jan 2001 14:10:23 -0800 (PST)
From: Joel Jaeggli <joelja@darkwing.uoregon.edu>
To: Paul Hoffman <paul@pxh.oac.ucla.edu>
Cc: beowulf@beowulf.org
Subject: Re: Rack mounted systems.

On Wed, 10 Jan 2001, Paul Hoffman wrote:

>
> Am attempting to estimate machine room floor space for a fairly
> large Beowulf cluster (~500 processors.) Here are my questions
> re: loading these into racks: (assume Pentium processors)
>
> - what is the minimum width of a 2-way node? (1U, 2U?)

that's height... but  yeah various vendors do 1u duals. 19" racks are
generally when all is said and done at least 22" wide. and the 1u dual
boxes we have are about 24" deep...

> - how many nodes can reasonably be put in a standard
>   10" rack? i.e. can the rack be fully loaded or is
>   some spacing required?

if the rack is sitting on a raised floor with hvac blowing from underneath
and a fan tray on top. and the pc's are vented out the ends there's no
reason you can't get 42 1u pc's in 1 7' rack... things like switches,
terminal servers, rackmount upses, and so on take additional space...

the space that the Ac/chiller takes up can be considerable as well and
needs to be factored in. the ibm netfinity 330r's in the akamai rack in
our data center have a nominal current draw of slighly more than one amp
each. so to extrapolate that to one rack that's ~44amps, ~4900 watts,
since you're gonna need ~1ton of ac for every 3.5-4kw that you're dumping
into the room so for 500 1amp loads on 110v you'll need a ac unit rated
for something like a 14.5ton load which is a pretty big although not huge
ac unit, it's about the size of 5 residential airconditioners...

> Thanks.
> paul /
>
>
>
> _______________________________________________
> Beowulf mailing list
> Beowulf@beowulf.org
> http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
--------------------------------------------------------------------------
Joel Jaeggli				       joelja@darkwing.uoregon.edu
Academic User Services			     consult@gladstone.uoregon.edu
     PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E
--------------------------------------------------------------------------
It is clear that the arm of criticism cannot replace the criticism of
arms.  Karl Marx -- Introduction to the critique of Hegel's Philosophy of
the right, 1843.



_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf

From walt@parl.ces.clemson.edu Wed Jan 10 18:43:17 2001
Date: Wed, 10 Jan 2001 17:34:46 -0600
From: Walter B. Ligon III <walt@parl.ces.clemson.edu>
To: Paul Hoffman <paul@pxh.oac.ucla.edu>
Cc: beowulf@beowulf.org
Subject: Re: Rack mounted systems. 

--------

There is a company that sells a case that will put 3 nodes in 1U.  That is
the most dense I have seen.  Of course, you have to buy from them (including
their mainboard, I think) and I have doubts about cooling in those things.

We're building 264 nodes (528 cpus) using 2U cases.  A "standard" rack is 42U. 
There is no wasted space, but you need to think about network switches, UPSs, 
console management hubs, etc. which will take more space.  I think we're
planning on 16 racks.

You can also get 1/2 depth cases that let you populate the back of the rack.
I'm not sure I'd want to deal the resulting rat's nest.  Might be a way to
get the auxilliary stuff out the way though.

Good luck, and let us know how it turns out.

Walt

> 
> Am attempting to estimate machine room floor space for a fairly
> large Beowulf cluster (~500 processors.) Here are my questions
> re: loading these into racks: (assume Pentium processors)
> 
> - what is the minimum width of a 2-way node? (1U, 2U?)
> 
> - how many nodes can reasonably be put in a standard 
>   10" rack? i.e. can the rack be fully loaded or is
>   some spacing required?
> 
> Thanks.
> paul /
> 
> 
> 
> _______________________________________________
> Beowulf mailing list
> Beowulf@beowulf.org
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Dr. Walter B. Ligon III
Associate Professor
ECE Department
Clemson University



_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf

From emintz@staff.mail.com Wed Jan 10 18:43:32 2001
Date: Wed, 10 Jan 2001 17:46:05 -0500
From: Erik Mintz <emintz@staff.mail.com>
To: Paul Hoffman <paul@pxh.oac.ucla.edu>, beowulf@beowulf.org
Subject: RE: Rack mounted systems.

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "US-ASCII" character set.  ]
    [ Some characters may be displayed incorrectly. ]

	The tightest density in today's market is with the RIA model servers from
Crystal, they are less than 1U, with a special hot-plug mounting system.
Check them out on their
site -http://crystalpc.com/products/computers/ria.asp You can fit 52 in a
standard 7ft rack. they have built custom backplanes for me, and they have
twin processor boards under development.
To load a cabinet with 52 servers, you'll need to order extra power, and
good cooling (raised floor, low ambient temp, etc.). If you'd like, I can
set you up with a sales engineer. I use them extensively, great systems.
	The next step down from that would be Compaq with their dl360 models at 1U
(42 servers in a 7ft rack). They come standard with dual capable
motherboards, and two PCI slots. They also have a built in twin ethernet
chipset, but I've had problems with this chipset and Beowulf (intel). The
compaqs also need more cooling and power than the Crystals, but they have a
fast built in scsi with hot-swap drives.

The Crystals spec out to .87U. That = 520 nodes in 10 racks. You also need
supporting network hardware. Right now I think you can get the most ports/U
from a cisco 3548 series switch at 1U each. it's a stackable, but if you
need density over fault tolerance, it will work nicely.

-Erik

Erik Mintz
Domestic Field Engineering Manager
Mail.com
732-516-2767 emintz@staff.mail.com

-----Original Message-----
From: beowulf-admin@beowulf.org [mailto:beowulf-admin@beowulf.org]On
Behalf Of Paul Hoffman
Sent: Wednesday, January 10, 2001 3:55 PM
To: beowulf@beowulf.org
Subject: Rack mounted systems.



Am attempting to estimate machine room floor space for a fairly
large Beowulf cluster (~500 processors.) Here are my questions
re: loading these into racks: (assume Pentium processors)

- what is the minimum width of a 2-way node? (1U, 2U?)

- how many nodes can reasonably be put in a standard
  10" rack? i.e. can the rack be fully loaded or is
  some spacing required?

Thanks.
paul /



_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf



_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf

From marsden@scripps.edu Wed Jan 10 18:43:48 2001
Date: Wed, 10 Jan 2001 15:20:47 -0800 (PST)
From: Brian Marsden <marsden@scripps.edu>
To: beowulf@beowulf.org
Subject: Re: Rack mounted systems. 

On Wed, 10 Jan 2001, Walter B. Ligon III wrote:

> There is a company that sells a case that will put 3 nodes in 1U.  That is
> the most dense I have seen.  Of course, you have to buy from them (including
> their mainboard, I think) and I have doubts about cooling in those things.

Racksaver (racksaver.com) do these. I have experience of these nodes (6
800MHz Pentium III's per 1U box). Yes they are hotter than a 4P1U system
(unsurprsingly) but I have experienced no problems with heating issues
(and this is a 128 processor system in an enclosed rack). Each 1U box does
not seem to pump out much more heat than a 2U API UP2000 box (although
this is purely qualitative).

Brian Marsden

-- 
---------------------------------------------------------------------
 Brian Marsden        			  Email: marsden@scripps.edu
 TSRI, San Diego, USA.  Phone: +1 858 784 8698  Fax: +1 858 784 8299
---------------------------------------------------------------------




_______________________________________________
Beowulf mailing list
Beowulf@beowulf.org
http://www.beowulf.org/mailman/listinfo/beowulf
