Improving data quality for low-cost environmental sensors

• Main Author: Fang, Xinwei
• Other Authors: Bate, Iain
• Published: University of York 2018
• Subjects: 004
• Online Access: https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.752638

Using low-cost sensors to monitor the urban environment has become increasingly popular, as they can provide better data resolution than current practices. However, these low-cost sensors often produce poorer data quality, and so the data may not be utilised directly without processing. This thesis presents a two-phase solution for improving the data quality of low-cost environmental sensors. The solution consists of a novel method for anomaly detection and removal, and a process of sensor calibration. In the first phase, an anomaly model is utilised to identify the anomalies, which is constructed using a Bayesian-based approach. New contextual information is used to build the anomaly model, that is to the best of our knowledge the first time it has been used for such purpose. The result shows that this solution is more practical and robust than the existing approaches. In the second phase, a systematic comparison of the state-of-the-art calibration approaches is performed. The comparison aims to understand the difference between the methods, and the result shows a regression based method could provide a more predicable result and require much less computational resources. As a result, a regression based method is used for calibrating sensors in this work. In contrast to the existing approaches, the proposed method for calibration is able to systematically and automatically select the calibration parameters. The parameter selection ensures the best set of parameters are used in the model, which makes the calibration process less sensitive to different environmental conditions. The overall evaluations are performed using real datasets. The results show the data quality in terms of general accuracy against the reference instruments can be significantly improved, especially for sensors at roadside.