A Regularization-Based Big Data Framework for Winter Precipitation Forecasting on Streaming Data

• Main Authors: Andreas Kanavos, Maria Trigka, Elias Dritsas, Gerasimos Vonitsanos, Phivos Mylonas
• Format: Article
• Language: English
• Published: MDPI AG 2021-08-01
• Series: Electronics
• Subjects: Apache Cassandra; Apache Kafka; Apache Spark Streaming; big data; classification; knowledge discovery
• Online Access: https://www.mdpi.com/2079-9292/10/16/1872

In the current paper, we propose a machine learning forecasting model for the accurate prediction of qualitative weather information on winter precipitation types, utilized in Apache Spark Streaming distributed framework. The proposed model receives storage and processes data in real-time, in order to extract useful knowledge from different sensors related to weather data. In following, the numerical weather prediction model aims at forecasting the weather type given three precipitation classes namely rain, freezing rain, and snow as recorded in the Automated Surface Observing System (ASOS) network. For depicting the effectiveness of our proposed schema, a regularization technique for feature selection so as to avoid overfitting is implemented. Several classification models covering three different categorization methods namely the Bayesian, decision trees, and meta/ensemble methods, have been investigated in a real dataset. The experimental analysis illustrates that the utilization of the regularization technique could offer a significant boost in forecasting performance.