Convolutional Neural Networks

By Sandro Skansi. 2018

1,538 citations recorded by the source corpus.

This chapter introduces the first deep learning architecture of the book, convolutional neural networks. It starts with redefining the way a logistic regression accepts data, and defines 1D and 2D convolutional layers as a natural extension of the logistic regression. The chapter also details on how to connect the layers and dimensionality problems. The local receptive field is introduced as a core concept of any convolutional architecture and the connections with the vanishing gradient problem is explored. Also the idea of padding is introduced in the visual setting, as well as the stride of the local receptive field. Pooling is also explored in the general setting and as max-pooling. A complete convolutional neural network for classifying MNIST is then presented in Keras code, and all the details of the code are presented as comments and illustrations. The final section of the chapter presents modifications needed to adapt convolutional networks, which are primarily visual classificators, to work with text and language.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022, methodology): “We compare our proposed ap-424 proach with the following nine baseline methods:425 • TextCNN (Kim, 2019) is a classic text classifi-426 cation model using CNN on top of word vectors.427 • TextRCNN (Lai et al., 2015) combines both428 RNN and CNN for text classification tasks.429 • GAT (Veličković e”
Graph Convolutional Encoders for Syntax-aware Neural Machine Translation (2017, methodology): “In order to produce syntax-aware feature representations of words, we exploit graph-convolutional networks (GCNs) (Duvenaud et al., 2015; Defferrard et al., 2016; Kearnes et al., 2016; Kipf and Welling, 2016).”
Ranking Sentences for Extractive Summarization with Reinforcement Learning (2018, positive): “In recent years, CNNs have proven useful for various NLP tasks (Collobert et al., 2011; Kim, 2014; Kalchbrenner et al., 2014; Zhang et al., 2015; Lei et al., 2015; Kim et al., 2016; Cheng and Lapata, 2016) because of their effectiveness in identifying salient patterns in the input (Xu et al.”
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “CNN-nonstatic/MC : Convolu-tional neural networks for sentence classiﬁcation (Kim, 2014).”
Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings (2017, neutral): “, 2011, 2012, 2013; İrsoy and Cardie, 2014), convolutional neural networks (Kalchbrenner et al., 2014; Kim, 2014; Hu et al., 2014), and recurrent neural networks using long short-term memory (Tai et al.”
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks (2019, positive): “We implement them as described in (Kim, 2014).”
Predicting Station-level Hourly Demands in a Large-scale Bike-sharing Network: A Graph Convolutional Neural Network Approach (2017, methodology): “A form of polynomial filters has been used in a few studies (Defferrard et al., 2016; Kipf and Welling, 2016; Shuman et al., 2013):”
Hate Speech Dataset from a White Supremacy Forum (2018, background): “• Convolutional Neural Networks (CNN), as described in (Kim, 2014).”
End-to-End Multi-View Networks for Text Classification (2017, methodology): “having views that weight arbitrary feature vectors, we augment our bag-of-words representation with vectors built by n-gram filters max-pooled over the entire text (Kim, 2014), with one feature vector for each n-gram order, n = 2 .”
A Deep Network with Visual Text Composition Behavior (2017, methodology): “We use the same splits for training, dev, and test data as in (Kim, 2014) to predict the ﬁne-grained 5-class sentiment categories of the sentences.”
Stock Market Prediction with Deep Learning: A Character-based Neural Language Model for Event-based Trading (2017, methodology): “…a range of di ↵ erent neural networks architectures have been proposed for text classi-ﬁcation, each one with strong results on di ↵ er-Leonardo ent benchmarks (e.g. Socher et al. (2013), Kim (2014) and Kumar et al. (2016)), and each one proposing di ↵ erent ways to encode the textual information.”