Improving Fast Velocity and Large Volume Data Processing in ETSI M2M Architecture

• Main Authors: Peng, Po-Han, 彭柏翰
• Other Authors: Lin, Fuchun Joseph
• Format: Others
• Language: en_US
• Published: 2015
• Online Access: http://ndltd.ncl.edu.tw/handle/83558256305230384976

碩士 === 國立交通大學 === 網路工程研究所 === 104 === In the ETSI standard, each data collected by sensors would be stored in database before they are processed. In other words, those collecting data will not only occupy bandwidth of the internet, but also hold some storage resources in the platform. Furthermore, in ETSI the sensing data would not be stored like traditional format as SQL-like structure, but become more complicate one - resource tree which describes hierarchical attributes for better data management. Consequently, it takes more efforts to manage IoT data stored in the IoT platform. However, for those collected data, not all of them are really useful for the applications. Some of them, actually, are redundant and doesn’t need to be allocated too much resource. In this thesis research, we propose to process streaming data first then determine whether the data should be kept or not. The idea is to treat data differently based on their different nature. For example, we can filter redundant, useless and fallacious data out in the very beginning. Then for those not completely useful data, we can refine them carefully and take key values out for storage and further processing before they are sent to the server. As for important data, we can process them and take immediate action as soon as possible, then fully store them in the database if needed. By filtering and pre-processing those data, it can save lots of resources by reducing data transmission, data storage and data management and processing overhead for those data collected by IoT sensors, the goal is to make an IoT platform more efficient and highly utilized without wasting the resources on useless data. This research will focus on how to enable big velocity and large volume data processing in an ETSI M2M standard compliant IoT platform - OpenMTC. To compare the differences between our approach and traditional approach of handling data, a use case from factory management will be used to demonstrate the results in terms of cost and efficiency. For the cost analysis, we will measure the storage space and the data transmission volume required for each approach. For the efficiency analysis, we will observe the difference between these two approaches in terms of their cpu and memory usage. We are going to demonstrate our approach of handling data can largely improve big velocity and large volume data processing in an ETSI M2M architecture.