DynDL: Scheduling Data-Locality-Aware Tasks with Dynamic Data Transfer Cost for Multicore-Server-Based Big Data Clusters

DynDL: Scheduling Data-Locality-Aware Tasks with Dynamic Data Transfer Cost for Multicore-Server-Based Big Data Clusters

Network bandwidth is a scarce resource in big data environments, so data locality is a fundamental problem for data-parallel frameworks such as Hadoop and Spark. This problem is exacerbated in multicore server-based clusters, where multiple tasks running on the same server compete for the server&...

Full description

Bibliographic Details
Main Authors: Jiahui Jin, Qi An, Wei Zhou, Jiakai Tang, Runqun Xiong
Format: Article
Language:English
Published: MDPI AG 2018-11-01
Series:Applied Sciences
Subjects:
data locality
task scheduling
multicore server
load balancing
bipartite graph
big data
cloud computing
Online Access:https://www.mdpi.com/2076-3417/8/11/2216

Internet

https://www.mdpi.com/2076-3417/8/11/2216

Similar Items