Estimating Missing Data in Temporal Data Streams Using Multi-Directional Recurrent Neural Networks

By Jinsung Yoon, William R. Zame, Mihaela van der Schaar. IEEE Transactions on Biomedical Engineering, 2017

340 citations recorded by the source corpus.

Missing data is a ubiquitous problem. It is especially challenging in medical settings because many streams of measurements are collected at different—and often irregular—times. Accurate estimation of the missing measurements is critical for many reasons, including diagnosis, prognosis, and treatment. Existing methods address this estimation problem by interpolating within data streams or imputing across data streams (both of which ignore important information) or ignoring the temporal aspect of the data and imposing strong assumptions about the nature of the data-generating process and/or the pattern of missing data (both of which are especially problematic for medical data). We propose a new approach, based on a novel deep learning architecture that we call a Multi-directional Recurrent Neural Network that interpolates within data streams and imputes across data streams. We demonstrate the power of our approach by applying it to five real-world medical datasets. We show that it provides dramatically improved estimation of missing measurements in comparison to 11 state-of-the-art benchmarks (including Spline and Cubic Interpolations, MICE, MissForest, matrix completion, and several RNN methods); typical improvements in Root Mean Squared Error are between 35%–50%. Additional experiments based on the same five datasets demonstrate that the improvements provided by our method are extremely robust.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d21f137c0b26f13daa91319a5e23f430fec68ae9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation (2021, methodology): “A popular approach using deep learning is to use RNNs, including LSTMs and GRUs, for sequence modeling [17, 8, 7].”
SAITS: Self-Attention-based Imputation for Time Series (2022, methodology): “Among them, the methods [21, 30, 32, 33] are autoregressive models that are highly susceptible to compounding errors [42, 31].”
How artificial intelligence and machine learning can help healthcare systems respond to COVID-19 (2020, methodology): “Developments in the field of active sensing offer tools for optimally trading off the value of information against the cost of acquisition (Yoon et al. 2019, 2018).”
Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models (2022, methodology): “Deep-learning-based time series imputation methods can be broadly categorized based on the technology used: (1) RNN-based approaches such as BRITS [5], GRU-D [6], NAOMI [31], and M-RNN [50] use single or multi-directional RNNs to model the time series.”
Multimodal data fusion for cancer biomarker discovery with deep learning (2023, methodology): “…including mean/median substitution, regression, k -nearest neighbour and tree-based methods to more advanced algorithms such as multiple imputation, multivariate imputation by chained equations or neural networks such as RNNs, long short-term memory and generative adversarial networks 121–123 .”
Multi-Time Attention Networks for Irregularly Sampled Time Series (2020, background): “Yoon et al. (2019) and Yoon et al. (2018) presented an approach based on the multi-directional RNN (M-RNN) that can leverage observations from the relative past and future of a given time point.”