next up previous
Next: Beowulf Architecture Characteristics Up: Achieving a Balanced Low-Cost Previous: Achieving a Balanced Low-Cost

Introduction

The Beowulf Parallel Workstation integrates off-the-shelf commodity subsystems to create a new operating point in price-performance for single-user scientific workstation environments. Beowulf's capabilities include a Gops peak performance, half a GByte of main memory, and disk storage capacity of 20 GBytes achieved at the cost of a conventional high-end scientific workstation (under $50K). These capabilities are accomplished through a parallel configuration of multiple processor subsystems, disks, and interconnection networks; all commodity components derived from the PC marketplace. The challenge is to define structures of such components that provide a balanced ensemble of resources in support of user needs. This paper looks at the specific problem of balancing parallel disk capacity and file transfer bandwidth with the message passing bandwidth of the system internal interconnect network.

To achieve high intra-system interconnect bandwidth, the Beowulf project has pioneered the use of multiple commodity networks within a single workstation. It has been shown [7] that network bandwidth can be scaled, at least across several parallel Ethernet networks (10 Mbps) to achieve useful sustained throughput gain. For example, dual Ethernets have delivered sustained bandwidth of almost 2 MBytes per second under favorable conditions of packet size. This was achieved in a user transparent manner through changes in the operating system. Beowulf employs the Linux operating system [4] which, among its other features, comes with source code and therefore is ideal for this class of research. Modifications to Linux were made to support channel bonding, allowing message packets to use any of the available networks.

Even with multiple networks and channel bonding, it was shown [6] that dual parallel 10 Mbps Ethernet can impose a bottleneck on disk file transfers under unfavorable conditions. One experiment revealed a discrepancy of about a factor of four with respect to disk throughput demand. Two approaches are being pursued by the Beowulf project. One approach alters the topology of interconnection to segment the two busses into four segments, each using moderate cost switches. The results of these experiments will be presented in a separate paper. The second approach is to exploit the very recent advances in 100 Mbps Ethernet (also referred to as Fast Ethernet). Only in the last few months has this technology reached the commodity market and at a price level commensurate with the objectives of the Beowulf workstation project. Early analysis showed that one such network might be marginal but that dual Fast Ethernets should be able to provide sufficient useful bandwidth to remove the internal interconnect as the limiting factor for transferring files among remote processor subsystems.

A new realm of system architecture has been created by the opportunities implied by the emerging low cost processor and network technology base. The PopC or Pile-of-PCs approach enables on-site configuration of essentially interchangeable components easily procured locally from multiple vendors and distributors. The flexibility in configuration permits the end user to match needs specific to the immediate workload requirements and to adapt resources as requirements evolve. However, PopC is still experimental and only made possible by the Linux operating system. Linux has provided a sophisticated and robust system software platform with source code availability and essentially no legal constraints. From this perspective, it is an almost unique tool for systems research and was the catalyst for PopC and the Beowulf experiment.

This paper presents the first published results of experiments conducted to evaluate multiple Fast Ethernet channels as the interconnection medium for a parallel workstation. Through empirical means, this new form of communication is characterized. It is shown that its capacity is not fully utilized by a single processor because of software limitations. However, it is demonstrated that multiple processor subsystems using a single channel concurrently can exploit most of its available capacity which is precisely the mode of use required by the Beowulf approach. Even in single source mode, achievable bandwidth will be shown to exceed sustainable disk file transfer rates. Finally, quantitative data will be provided that demonstrates a balanced architecture for mass storage management based on Fast Ethernet technology and channel bonding techniques employed by Beowulf.



next up previous
Next: Beowulf Architecture Characteristics Up: Achieving a Balanced Low-Cost Previous: Achieving a Balanced Low-Cost



Chance Reschke
Mon Nov 4 12:29:54 EST 1996