A Big Data Platform for Smart Manufacturing Based on Lambda Architecture and Optimization of Image Data Storage

• Main Authors: SHIU, SHR-WEN, 許釋文
• Other Authors: LIU, CHUAN-MING
• Format: Others
• Language: zh-TW
• Published: 2019
• Online Access: http://ndltd.ncl.edu.tw/handle/uyw7ku

碩士 === 國立臺北科技大學 === 資訊工程系 === 107 === The goal of Smart Manufacturing is to reduce the human resources requirements of the production line by applying the Big Data technology to the manufacturing business. The smart manufacturing and Industry 4.0 are used interchangeable, then the Big Data Platform takes an important position in this. The Big Data Platform is like the brain of the entire factory. It receive all data from the line sensor, processing and analyzing, and finally make the feedback decision. With the innovation of production technology, the data becomes more complex and larger day by day. Besides, many sophisticated manufacturing industries are also beginning to enter the field of Industry 4.0. In addition to the correctness and availability, it also requires the immediacy of data processing. After the sensor receives the data, the platform must provide feedback in a short time. This paper will present a Big Data Platform based on the Lambda architecture. The architecture includes both Stream Processing and Batch Processing, and meets the immediate feedback needs of high-precision manufacturing. Besides, this paper also optimizes the storage of image data generated by Automated Optical Inspection (AOI) technology commonly used in today's manufacturing industry, and optimizes queries for Hive data warehouse. In order to verify the effectiveness of the optimization, this paper generates a large amount of test data according to the actual data for testing, and finally confirms that the optimization of the paper for the storage of the paper does reduce the consumption of a large amount of memory as expected. For the Hive query also reduces the time spent.