An efficient storage and service method for multi-source merging meteorological big data in cloud environment

• Main Authors: Ming Yang, Wenchun He, Zhiqiang Zhang, Yongjun Xu, Heping Yang, Yufeng Chen, Xiaolong Xu
• Format: Article
• Language: English
• Published: SpringerOpen 2019-10-01
• Series: EURASIP Journal on Wireless Communications and Networking
• Subjects: Multi-source merging sensors data; Meteorological data storage; Meteorological data service; Distributed NoSQL; Semi/unstructured data
• Online Access: http://link.springer.com/article/10.1186/s13638-019-1576-0

Abstract With the development of the meteorological IoT (Internet of Things) and meteorological sensing network, the collected multi-source meteorological data have the characteristics of large amount of information, multidimensional and high accuracy. Cloud computing technology has been applied to the storage and service of meteorological big data. Although the constant evolution of big data storage technology is improving the storage and access of meteorological data, storage and service efficiency is still far from meeting multi-source big data requirements. Traditional methods have been used for the storage and service of meteorological data, and a number of problems still persist, such as a lack of unified storage structure, poor scalability, and poor service performance. In this study, an efficient storage and service method for multidimensional meteorological data is designed based on NoSQL big data storage technology and the multidimensional characteristics of meteorological data. In the process of data storage, multidimensional block compression technology and data structures are applied to store and transmit meteorological data. In service, heterogeneous NoSQL common components are designed to improve the heterogeneity of the NoSQL database. The results show that the proposed method has good storage transmission efficiency and versatility, and can effectively improve the efficiency of meteorological data storage and service in meteorological applications.