SAITS: Self-Attention-based Imputation for Time Series

By Wenjie Du, David Cote, Y. Liu. Expert systems with applications, 2022

567 citations recorded by the source corpus.

Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism for missing value imputation in multivariate time series. Trained by a joint-optimization approach, SAITS learns missing values from a weighted combination of two diagonally-masked self-attention (DMSA) blocks. DMSA explicitly captures both the temporal dependencies and feature correlations between time steps, which improves imputation accuracy and training speed. Meanwhile, the weighted-combination design enables SAITS to dynamically assign weights to the learned representations from two DMSA blocks according to the attention map and the missingness information. Extensive experiments quantitatively and qualitatively demonstrate that SAITS outperforms the state-of-the-art methods on the time-series imputation task efficiently and reveal SAITS' potential to improve the learning performance of pattern recognition models on incomplete time-series data from the real world. The code is open source on GitHub at https://github.com/WenjieDu/SAITS.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Signal Processing

View Paper: https://www.semanticscholar.org/paper/41780e237ff6a6685eec7bf3b5316e1ff0cc18d1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nils Strodthoff
2. Juan Miguel Lopez Alcaraz

In Cognitive Neuroscience

1. Nils Strodthoff
2. Juan Miguel Lopez Alcaraz

In Signal Processing

1. Nils Strodthoff
2. Juan Miguel Lopez Alcaraz

Citations in the corpus

• Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models (2022, methodology): “However, these algorithms suffer of diverse training limitation, and as pointed out in recent works, many of them might only show sub-optimal performance across diverse missing scenarios and at different missing ratios [9] [12].”