Convolutional Sequence Modeling Revisited

By Shaojie Bai, J. Kolter, Vladlen Koltun. 2018

78 citations recorded by the source corpus.

Subfields: Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

1. Jack W. Rae
2. Anna Potapenko
3. Doina Bucur
4. Hui Xiong
5. Christin Seifert
6. Jianxin Li
7. Jieqi Peng
8. Meike Nauta

Citations in the corpus

• Compressive Transformers for Long-Range Sequence Modelling (2019, background): “This dataset has been a popular recent LM benchmark due to the potential to exploit longer-range dependencies (Grave et al., 2016; Rae et al., 2018; Bai et al., 2018b).”
• Relational recurrent neural networks (2018, neutral): “0.0 99.8 Table 2: Validation and test perplexities on WikiText-103, Project Gutenberg, and GigaWord v5. WikiText-103 Gutenberg GigaWord Valid. Test Valid Test Test LSTM [37] - 48.7 - - - Temporal CNN [38] - 45.2 - - - Gated CNN [39] - 37.2 - - - LSTM [29] 34.1 34.3 41.8 45.5 43.7 Quasi-RNN [40] 32 3”
• Mmdenselstm: An Efficient Combination of Convolutional and Recurrent Neural Networks for Audio Source Separation (2018, background): “Although it has been suggested that CNNs often work better than RNNs even for sequential data modeling [18, 20], RNNs can also benefit from multi-scale and multi-band modeling because they make it easier to capture long-term dependencies and can save parameters by omitting redundant connections…”
• Causal Discovery with Attention-Based Convolutional Neural Networks (2019, methodology): “Other issues with RNNs are the high memory usage to store partial results, their complex architecture making them hard to interpret and the impossibility of parallelism which hinders scaling [36].”
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2020, methodology): “CNN-based methods (Stoller et al. 2019; Bai, Kolter, and Koltun 2018) use the convolutional filter to capture the long term dependency, and their receptive fields grow exponentially with the stacking of layers, which hurts the sequence alignment.”
• Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection (2020, unknown): “CNNs have been proved to be competent for regression problems, and could also take long-time dependencies into account, which even outperform recurrent networks (Bai et al. 2018).”