Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series

By Zachary Chase Lipton, David C. Kale, Randall C. Wetzel. 2016

273 citations recorded by the source corpus.

We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the intensive care unit (ICU) of a major urban medical center, our data consists of multivariate time series of observations. The data is irregularly sampled, leading to missingness patterns in re-sampled sequences. In this work, we show the remarkable ability of RNNs to make effective use of binary indicators to directly model missing data, improving AUC and F1 significantly. However, while RNNs can learn arbitrary functions of the missing data and observations, linear models can only learn substitution values. For linear models and MLPs, we show an alternative strategy to capture this signal. Additionally, we evaluate LSTMs, MLPs, and linear models trained on missingness patterns only, showing that for several diseases, what tests are run can be more predictive than the results themselves.

Subfields: Artificial Intelligence, Health Information Management, Model Architectures, Medical & Health AI, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Information Management

In Model Architectures

In Medical & Health AI

In Signal Processing

Citations in the corpus

Neural Ordinary Differential Equations (2018, neutral): “For example, a patient may be more likely to take a medical test if they are sick.”
$\mathtt {Deepr}$: A Convolutional Net for Medical Records (2016, methodology): “In [36] and [38], recurrent nets are used, and in [39], a convolutional net is employed.”
Learning Treatment-Response Models from Multivariate Longitudinal Data (2017, methodology): “Other ﬂexible methods such as recurrent neural networks have also been used for modeling multivariate longitudinal data (Lipton et al., 2016).”
GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series (2019, background): “To overcome this issue, the main approach consists in some form of data imputation and jointly feeding the observation mask and times of observations to the recurrent network (Che et al., 2018; Choi et al., 2016a; Lipton et al., 2016; Du et al., 2016; Choi et al., 2016b; Cao et al., 2018).”
Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Forecasting Network-wide Traffic State with Missing Values (2020, methodology): “Due to RNNs for times series with missing values have been explored and applied Che et al. (2018); Lipton et al. (2016), RNN-based models have the potential to combine imputation methods with prediction models.”
Scalable Joint Models for Reliable Uncertainty-Aware Event Prediction (2017, methodology): “Non-probabilistic methods based on recurrent neural networks have also been used for modeling irregularly sampled time series [61].”
Multi-Time Attention Networks for Irregularly Sampled Time Series (2020, background): “For example, Marlin et al. (2012) and Lipton et al. (2016) discretize continuous-time observations into hour-long bins.”
Estimating Missing Data in Temporal Data Streams Using Multi-Directional Recurrent Neural Networks (2017, methodology): “The benchmarks against which we compare are: the algorithms proposed in [23, 24, 25]; Spline and Cubic Interpolation [1] ; MICE [5]; MissForest [6], EM [4]; the matrix completion algorithm of [7];the Auto-Encoder algorithm proposed in [32]; and the Markov chain Monte Carlo (MCMC) method [33].”
Improving Factor-Based Quantitative Investing by Forecasting Company Fundamentals (2017, background): “Related Work Deep neural networks models have proven powerful for tasks as diverse as language translations [14, 1], video captioning [11, 16], video recognition [6, 15], and time series modeling [9, 10, 3].”
Latent ODEs for Irregularly-Sampled Time Series (2019, background): “Such preprocessing destroys information, particularly about the timing of measurements, which can be informative about latent variables [Lipton et al., 2016, Che et al., 2018].”
Time2Vec: Learning a Vector Representation of Time (2019, background): “It is possible to aggregate asynchronous events into time-bins and to use synchronous models over the bins [36, 2].”
BRITS: Bidirectional Recurrent Imputation for Time Series (2018, background): “Recently, some researchers attempted to impute the missing values with recurrent neural networks [7, 10, 21, 12, 33].”