Performance Study of Map-Reduce over Hadoop of Large-scale Data Processing

• Main Authors: Yun-sun Yee, 俞詠善
• Other Authors: Jenq-Shiou Leu
• Format: Others
• Language: en_US
• Published: 2010
• Online Access: http://ndltd.ncl.edu.tw/handle/4tmm55

碩士 === 國立臺灣科技大學 === 電子工程系 === 98 === Popularity for the term ‘Cloud-Computing’ has been increasing in recent years. There are many great companies such as Yahoo, Google etc. tried to provide related services to business community, even through public users. In addition to the SQL technique, Map-Reduce, a programming model that realizes implementing large-scale data processing, has been a hot topic that is widely discussed through many studies. Many real-world tasks such as data processing for search engines can be parallel-implemented through a simple interface with two functions called Map and Reduce. In this paper, we focus on comparing the performance of the Hadoop implementation of Map-Reduce with SQL Server through simulations. In our studies, Hadoop can complete the same query faster than SQL Server. On the other hand, some concerned factors are also tested to see whether they would affect the performance for Hadoop or not. We also find that more machines included for data processing can make Hadoop achieve a better performance, especially for a large-scale data set.