CLIMIS4AVAL

Real-time cleansing of snow and weather data for operational avalanche forecasting

Started
October 19, 2022
Status
In Progress
Share this project

Abstract

Avalanche forecasting relies on snow, snow cover and weather data – for expert evaluation as well as for machine learning based support tools. The backbone of the Swiss avalanche forecast infrastructure are the data of the Intercantonal Measurement and Information System (IMIS), which currently consists of 187 automated snow and weather stations. They are distributed throughout the Swiss Alps and in most cases are situated in the high alpine regions above the tree line. The stations continuously record the snow and weather conditions and provide the national avalanche warning service of the SLF as well as local avalanche services responsible for public safety in settlements and on roads with the crucial information for danger assessment. Public products such as new snow or snow height maps also rely on these data.

Avalanche forecasting, physical snow models, hydrological predictions and many other Alpine scientific activities are largely data-driven; therefore, consistent and accurate data are fundamental for high-quality outputs. With increasing data volumes and the increased need for timely and accurate forecasts, preferably automated location-based forecasts, it becomes imperative to clean these essential data on the fly. The objective of this project is to develop algorithms that allow real-time detection of anomalies in the time series, but also to detect outliers, and impute missing data by applying state-of-the-art machine learning approaches. This real-time data cleansing will solve the long-standing issue of the IMIS data being contaminated with data anomalies, which has hindered automated processing.

People

Collaborators

SDSC Team:
Corinne Jones
Michele Volpi

PI | Partners:

WSL, Institute for Snow and Avalanche Research SLF, Avalanches and Prevention

  • Prof. Dr. Jürg Schweizer
  • Dr. Jan Svoboda
  • Marc Ruesch
  • David Liechti
  • Dr. Frank Techel

More info

WSL, Swiss Federal Institute for Forest Snow and Landscape Research, Hydrological Forecasts

  • Dr. Massimiliano Zappa
  • Florian Lustenberger

More info

description

Motivation

The IMIS network is widely used in many applications ranging from natural hazards to ecology. However, widespread and systematic use of data measured by IMIS stations is hindered by the many measurement anomalies and sensor outliers. In CLIMIS4AVAL we aim at performing application-agnostic anomaly and outlier detection, data imputation and downscaling and producing a maintenance model predicting whether a given measurement sensor is working as expected or not. In particular, the project focuses on the measurements of snow depth, wind speed and direction, air temperature, precipitation, and it will be validated by use of this data in routine application within WSL.

Proposed Approach / Solution

The SDSC is working with WSL and SLF to develop statistical and machine learning methods for anomaly detection in time series, outlier detection, and imputation for given IMIS parameters. We work on both traditional statistical methods and advanced deep learning forecasting methods.  

As an example, we developed a machine learning tool to support parsing and exploitation of snow height measurements from Automated Weather Stations (AWS) in the IMIS network. Some stations rely on an ultrasonic or laser sensor to measure the height above the soil of material accumulating under the station. In winter season, in standard conditions, this is clearly related to snow height, while in summertime to vegetation growth. However, crucial measurements of snow height occur in spring and autumn, where due to ambiguous and fast changing environmental conditions it is very hard to assign the measurement to either snow or other factors using only, for instance, temperature thresholds. To this end, we trained a deep learning classifier taking as input multivariate environmental parameters to predict whether the station observes snow or not. It is currently used internally at SLF to flag measurements that are likely related to snow and can therefore be used in downstream tasks. Figure 1 summarizes the pipeline; more details can be found in  the work of Svoboda et al. (2024).

Impact

Numerical avalanche prediction and other models used at SLF will be more accurate with the cleaned data. The cleaned data will also be made openly available in the data portal of SLF, and will therefore benefit the numerous downstream users of the data, ranging from hydrological modelers, avalanche forecasters, and alpine ecologists.

Figure 1: Example of a processing pipeline to mask snow measurements at an automated weather station. The classifier takes as input several environmental parameters (RSWR: Reflected Short Wave Radiation, TSS: Temperature of Snow Surface, TA: Temperature of Air, HS: Height of Snow). Note that those measurements are done independently on whether snow is actually present under the sensor. Image from Svoboda et al., 2024.

Gallery

Annexe

Publications

Additional resources

Bibliography

  1. Bavay, M. and Egger, T.: MeteoIO 2.4.2: a preprocessing library for meteorological data, Geosci. Model Dev., 7, 3135–3151, https://doi.org/10.5194/gmd-7-3135-2014, 2014.
  2. Pérez-Guillén, C., Techel, F., Hendrick, M., Volpi, M., van Herwijnen, A., Olevski, T., Obozinski, G., Pérez-Cruz, F., and Schweizer, J.: Data-driven automated predictions of the avalanche danger level for dry-snow conditions in Switzerland, Nat. Hazards Earth Syst. Sci., 22, 2031–2056, https://doi.org/10.5194/nhess-22-2031-2022, 2022.

Publications

Related Pages

More projects

ML-L3DNDT

Completed
Robust and scalable Machine Learning algorithms for Laue 3-Dimensional Neutron Diffraction Tomography
Big Science Data

BioDetect

Completed
Deep Learning for Biodiversity Detection and Classification
Energy, Climate & Environment

IRMA

In Progress
Interpretable and Robust Machine Learning for Mobility Analysis
No items found.

FLBI

In Progress
Feature Learning for Bayesian Inference
No items found.

News

Latest news

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data
May 1, 2024

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

We’ve developed a smart solution for wind tunnel testing that learns as it works, providing accurate results faster. It provides an accurate mean flow field and turbulence field reconstruction while shortening the sampling time.
The Promise of AI in Pharmaceutical Manufacturing
April 22, 2024

The Promise of AI in Pharmaceutical Manufacturing

The Promise of AI in Pharmaceutical Manufacturing

Innovation in pharmaceutical manufacturing raises key questions: How will AI change our operations? What does this mean for the skills of our workforce? How will it reshape our collaborative efforts? And crucially, how can we fully leverage these changes?
Efficient and scalable graph generation through iterative local expansion
March 20, 2024

Efficient and scalable graph generation through iterative local expansion

Efficient and scalable graph generation through iterative local expansion

Have you ever considered the complexity of generating large-scale, intricate graphs akin to those that represent the vast relational structures of our world? Our research introduces a pioneering approach to graph generation that tackles the scalability and complexity of creating such expansive, real-world graphs.

Contact us

Let’s talk Data Science

Do you need our services or expertise?
Contact us for your next Data Science project!