ADuS: adaptive resource allocation in cluster systems under heavy-tailed and bursty workloads.

• Online Access: http://hdl.handle.net/2047/d20002422

A large-scale cluster computing system has been employed in the multiple areas by offering the pools of fundamental resources. How to effectively allocate the shared resources in a cluster system is a critical but challenging issue, which has been extensively studied in the past few years. Despite the fact that classic load balancing policies, such as Random, Join Shortest Queue and size-based polices, are widely implemented in actual systems due to their simplicity and efficiency, the performance benefts of these policies diminish when workloads are highly variable and heavily dependent. We propose a new load balancing policy ADuS, which attempts to partition jobs according to their sizes and to further rank the servers based on their loads. By dispatching jobs of similar size to the servers with the same ranking, ADuS can adaptively balance user traffic and system load in the system and thus achieve significant performance benefts. Extensive simulations show the effectiveness and the robustness of ADuS under many different environments.