Design and Implementation of a Multi-domain Big Data System with Private Data Regulation Capabilities

• Main Authors: Min-Kuan Chen, 陳敏寬
• Other Authors: Mon-Yen Luo
• Format: Others
• Language: zh-TW
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/z2bmmc

碩士 === 國立高雄應用科技大學 === 資訊工程系 === 104 === With the rapid growth of computer performance, pervasive Internet, and rapid development of storage technologies, the demands for big data processing technologies are on the rise. However, individuals and organizations face a series of striking challenges on building a big data system. The major challenge is a big data system usually requires huge amount of computing and storage resources. The major goal of this paper is to design a scalable architecture for big data system based on hybrid cloud model. Our system utilizes various virtualization technologies to build a private cloud system that can also seamlessly integrate resources of public cloud systems. Meanwhile, our design also addresses the data security and privacy risks on the hybrid cloud model. We revised Hadoop system to implement a data protection mechanism so that organizational sensitive information cannot be moved to public cloud without proper protection. The performance data show that our system can protect privacy/sensitive data in the hybrid cloud architecture with acceptable overhead.