The Implementation of MPFS (MPI-based Parallel File System)

• Main Authors: Hsieh, Te-Ching, 謝德慶
• Other Authors: Ming-Feng Chang
• Format: Others
• Language: zh-TW
• Published: 1997
• Online Access: http://ndltd.ncl.edu.tw/handle/56415177223750286241

碩士 === 國立交通大學 === 資訊工程學系 === 85 === The performance of CPU and memory system has been increasing dramatically in recent years, while I/O sub-systems have not been improving at the same rate and becomes the performance bottleneck of modern computer systems specially in parallel computing systems. One way to improve I/O system bandwidth is to stripe file data across several disks or data servers so that file access can be carried out in a parallel fashion. Moreover, a parallel file interface is needed so that a programmer can effectively exploit the parallel I/O bandwidth in multi- processor systems. Most of the existing parallel file systems were designed for dedicated platform with low portability. This thesis presents the design of MPFS, a MPI-based parallel file system. MPI-IO is an extension of MPI to support a flexible user-defined logical file partition and a rich set of file access functions. Our MPFS implementation includes full supports of a high-level interface to describe the partitioning of file data among processes, a collective interface describing data transfers between process and file, asynchronous I/O operations allowing computation to be overlapped with I/O, shared file pointer among a group of processes, and limited support of hints to specify physical data layout across data servers. MPFS is now built on a workstation cluster connected by 100Mb/s fast Ethernet with FreeBSD operating system and the implementation can be ported to SunOS. Performance measurements of 64 MB file access are also performed on our system. The measurements show that the network bandwidth is the system bottleneck when more than one data server are utilized.