Based on Hadoop Platform-Accelerating Query to Process Big Data Combined with Heterogeneous Databases and Hive

• Main Authors: Jia-Sheng Liang, 梁嘉勝
• Other Authors: Ruay-Shiung Chang
• Format: Others
• Published: 2013
• Online Access: http://ndltd.ncl.edu.tw/handle/222554

碩士 === 國立東華大學 === 資訊工程學系 === 101 === Cloud Computing has been a topic issue in the field of research in recent years. Nowadays, the environment of Internet has become more popular and powerful. Information can be gotten on any mobile devices (e.g., smartphone, laptop and tablet) through the web services easily. Parallelization in computing and data storage increases the performance of computing and capacity of fault-tolerant. These significant features speed up the development of Cloud Computing service. Besides, more and more computing power and storage provided by cloud platforms are required to process big data. Information system needs to be reconsidered to deal with the parallelization of Big Data analytics services. In the traditional framework, the Information system combines with front-end application and database. The system will cost significant time and resources while the RDBMS (Relational Database Management System) is used to analyze Big Data. In order to solve this problem of Big Data analysis, we propose the WSG (Web Service GUI) that takes the advantages of Hadoop and RDBMS to overcome the obstacle. In this thesis, WSG installed on the Hadoop platform acts as a Web Service combined with Hive and Sqoop to integrate Big Data between Heterogeneous Databases. Therefore, the technician can utilize Hive QL to analyze Big Data, and WSG can convey data from HDFS to RDBMS by using Sqoop. As a result, our proposed method will reduce the burden of reconstruction information system to accomplish the standard of Cloud Computing, and also be benefit to acceleration in querying data of small size.