CLIMIS4AVAL
Real-time cleansing of snow and weather data for operational avalanche forecasting
Abstract
Avalanche forecasting relies on snow, snow cover and weather data – for expert evaluation as well as for machine learning based support tools. The backbone of the Swiss avalanche forecast infrastructure are the data of the Intercantonal Measurement and Information System (IMIS), which currently consists of 187 automated snow and weather stations. They are distributed throughout the Swiss Alps and in most cases are situated in the high alpine regions above the tree line. The stations continuously record the snow and weather conditions and provide the national avalanche warning service of the SLF as well as local avalanche services responsible for public safety in settlements and on roads with the crucial information for danger assessment. Public products such as new snow or snow height maps also rely on these data.
Avalanche forecasting, physical snow models, hydrological predictions and many other Alpine scientific activities are largely data-driven; therefore, consistent and accurate data are fundamental for high-quality outputs. With increasing data volumes and the increased need for timely and accurate forecasts, preferably automated location-based forecasts, it becomes imperative to clean these essential data on the fly. The objective of this project is to develop algorithms that allow real-time detection of anomalies in the time series, but also to detect outliers, and impute missing data by applying state-of-the-art machine learning approaches. This real-time data cleansing will solve the long-standing issue of the IMIS data being contaminated with data anomalies, which has hindered automated processing.
People
Collaborators
Corinne joined the SDSC as a senior data scientist in December 2020. She graduated with a PhD in Statistics from the University of Washington in 2020. Her doctoral research focused on representation learning for partitioning problems. Prior to her PhD, she obtained bachelor's degrees in Math, Statistics, and Economics, along with a master's degree in Economics, from Penn State University. Her research interests include deep learning and kernel-based methods, with applications in fields ranging from computer vision to oceanography.
Michele received a Ph.D. in Environmental Sciences from the University of Lausanne (Switzerland) in 2013. He was then a visiting postdoc in the CALVIN group, Institute of Perception, Action and Behaviour of the School of Informatics at the University of Edinburgh, Scotland (2014-2016). He then joined the Multimodal Remote Sensing and the Geocomputation groups at the Geography department of the University of Zurich, Switzerland (2016-2017). His main research activities were at the interface of computer vision, machine and deep learning for the extraction of information from aerial photos, satellite optical images and geospatial data in general.
PI | Partners:
WSL, Institute for Snow and Avalanche Research SLF, Avalanches and Prevention
- Prof. Dr. Jürg Schweizer
- Dr. Jan Svoboda
- Marc Ruesch
- David Liechti
- Dr. Frank Techel
WSL, Swiss Federal Institute for Forest Snow and Landscape Research, Hydrological Forecasts
- Dr. Massimiliano Zappa
- Florian Lustenberger
description
Motivation
The IMIS network is widely used in many applications ranging from natural hazards to ecology. However, widespread and systematic use of data measured by IMIS stations is hindered by the many measurement anomalies and sensor outliers. In CLIMIS4AVAL we aim at performing application-agnostic anomaly and outlier detection, data imputation and downscaling and producing a maintenance model predicting whether a given measurement sensor is working as expected or not. In particular, the project focuses on the measurements of snow depth, wind speed and direction, air temperature, precipitation, and it will be validated by use of this data in routine application within WSL.
Proposed Approach / Solution
The SDSC is working with WSL and SLF to develop statistical and machine learning methods for anomaly detection in time series, outlier detection, and imputation for given IMIS parameters. We work on both traditional statistical methods and advanced deep learning forecasting methods.
As an example, we developed a machine learning tool to support parsing and exploitation of snow height measurements from Automated Weather Stations (AWS) in the IMIS network. Some stations rely on an ultrasonic or laser sensor to measure the height above the soil of material accumulating under the station. In winter season, in standard conditions, this is clearly related to snow height, while in summertime to vegetation growth. However, crucial measurements of snow height occur in spring and autumn, where due to ambiguous and fast changing environmental conditions it is very hard to assign the measurement to either snow or other factors using only, for instance, temperature thresholds. To this end, we trained a deep learning classifier taking as input multivariate environmental parameters to predict whether the station observes snow or not. It is currently used internally at SLF to flag measurements that are likely related to snow and can therefore be used in downstream tasks. Figure 1 summarizes the pipeline; more details can be found in the work of Svoboda et al. (2024).
Impact
Numerical avalanche prediction and other models used at SLF will be more accurate with the cleaned data. The cleaned data will also be made openly available in the data portal of SLF, and will therefore benefit the numerous downstream users of the data, ranging from hydrological modelers, avalanche forecasters, and alpine ecologists.
Presentation
Gallery
Annexe
Publications
- Svoboda, J., Ruesch, M., Liechti, D., Jones, C., Volpi, M., Zehnder, M. and Schweizer, J.: Towards deep learning solutions for classification of automated snow height measurements (CleanSnow v1.0.0). In Review. Preprint available at: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1752/
Additional resources
Bibliography
- Bavay, M. and Egger, T.: MeteoIO 2.4.2: a preprocessing library for meteorological data, Geosci. Model Dev., 7, 3135–3151, https://doi.org/10.5194/gmd-7-3135-2014, 2014.
- Pérez-Guillén, C., Techel, F., Hendrick, M., Volpi, M., van Herwijnen, A., Olevski, T., Obozinski, G., Pérez-Cruz, F., and Schweizer, J.: Data-driven automated predictions of the avalanche danger level for dry-snow conditions in Switzerland, Nat. Hazards Earth Syst. Sci., 22, 2031–2056, https://doi.org/10.5194/nhess-22-2031-2022, 2022.
Publications
Related Pages
- Code for the snow height model: https://zenodo.org/records/12698071
- Manually-annotated data for snow/no snow classification: https://envidat.ch/#/metadata/snow-height-classification
- Related previous project: DEAPSnow - Improving snow avalanche forecasting by data-driven automated predictions
More projects
ML-L3DNDT
BioDetect
News
Latest news
Smartair | An active learning algorithm for real-time acquisition and regression of flow field data
Smartair | An active learning algorithm for real-time acquisition and regression of flow field data
The Promise of AI in Pharmaceutical Manufacturing
The Promise of AI in Pharmaceutical Manufacturing
Efficient and scalable graph generation through iterative local expansion
Efficient and scalable graph generation through iterative local expansion
Contact us
Let’s talk Data Science
Do you need our services or expertise?
Contact us for your next Data Science project!