Building Reliable and Cost-Effective Storage Systems for High-Performance Computing Datacenters

• Main Author: Qiao, Zhi
• Other Authors: Fu, Song
• Format: Others
• Language: English
• Published: University of North Texas 2020
• Subjects: Reliability; Storage Systems; High Performance Computing; Computer Science
• Online Access: https://digital.library.unt.edu/ark:/67531/metadc1707348/

In this dissertation, I first incorporate declustered redundant array of independent disks (RAID) technology in the existing system by maximizing the aggregated recovery I/O and accelerating post-failure remediation. Our analytical model affirms the accelerated data recovery stage significantly improves storage reliability. Then I present a proactive data protection framework that augments storage availability and reliability. It utilizes the failure prediction methods to efficiently rescue data on drives before failures occur, which significantly reduces the storage downtime and lowers the risk of nested failures. Finally, I investigate how an active storage system enables energy-efficient computing. I explore an emerging storage device named Ethernet drive to offload data-intensive workloads from the host to drives and process the data on drives. It not only minimizes data movement and power usage, but also enhances data availability and storage scalability. In summary, my dissertation research provides intelligence at the drive, storage node, and system levels to tackle the rising reliability challenge in modern HPC datacenters. The results indicate that this novel storage paradigm cost-effectively improves storage scalability, availability, and reliability.