Proposal for Real-Time Pre-Processing of Anomalies on Data Collected by Meteorological Sensor Network

EasyChair Preprint no. 10616, version 2

Abstract

A weather station is a set of sensors that record and supply physical measurements and meteorological parameters related to climate variations in a locality. The station collects, processes and stores meteorological data for use in weather forecasting. Weather forecasting is the application of science and technology to predict atmospheric conditions for a specific location and period in the future. However, the weather forecast data produced often deviate considerably from the actual weather values. This large discrepancy can be explained by the anomaly pre-processing technique (missing values and outliers) used and sometimes by the absence of the preprocessing of anomalies in a meteorological observation system. The presence of an anomaly in one variable can compromise the prediction of the whole observation. This paper proposes an algorithm for preprocessing anomalies in a weather observation system. For the pre-processing of missing values, we have used seven imputation techniques, namely MICE, DecisionTreeRegression, BayesianRidge, LinearRegressor, ExtraTreesRegressor, KNeighborsRegressor and KNNImputer. The results showed that the MICE technique performed best, with an MSE of 0.019344.

Keyphrases: Automatic Weather Station, data preprocessing, machine learning, Weather forecasting

@Booklet{EasyChair:10616,
  author = {Salomon Mba Tene and Vivient Corneille Kamla},
  title = {Proposal for Real-Time Pre-Processing of Anomalies on Data Collected by Meteorological Sensor Network},
  howpublished = {EasyChair Preprint no. 10616},

  year = {EasyChair, 2023}}