Iterative and Incremental Data Quality Improvement Procedure for Assisting Big Data Project Risk Management

• Main Authors: CHARNG,WEI-JIA, 常瑋家
• Other Authors: LAI,SEN-TANG
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/6x5q9f

碩士 === 實踐大學 === 資訊科技與管理學系碩士班 === 106 === Big data brings many benefits to enterprises and organizations. However, there are many difficulties that must be overcome in big data applications. The risk of failure of big data projects remains high, which is still more serious than large software development projects. Data quality defects and changes in demand will directly affect the analysis efficiency and results of big data. The lack of communication and interaction among people often leads to crisis such as improper planning and wrong decisions and expectations. How to reduce the development risk of big data projects has been become an issue. Agile software development adopts Iterative and Incremental Development (IID) to reduce the risk of software development. This paper introduces IID into Data Preprocessing and proposes a set of Big Data Preprocessing Operation Quality Improvement (POQM) process to measure incrementally. Correct and improve the lack of data quality in time, and establish communication and interaction by the iterative review and inspection activities. Prior to data modeling and analysis operations, timely improve the quality of big data preprocessing operations and reduce the risk of big data project failures.