The Unstoppable Rise of Computational Linguistics in Deep Learning

By James Henderson. 2020

32 citations recorded by the source corpus.

In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of variable binding and its instantiation in attention-based models, and argue that Transformer is not a sequence model but an induced-structure model. This perspective leads to predictions of the challenges facing research in deep learning architectures for natural language understanding.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Benoît Sagot
2. Samson Tan
3. Chenglei Si
4. Manan Dey
5. Arun Raja
6. Wilson Y. Lee
7. Colin Raffel
8. Zaid Alyafeai

In Computer Vision and Pattern Recognition

1. Samson Tan
2. Chenglei Si
3. Manan Dey
4. Arun Raja
5. Wilson Y. Lee
6. Colin Raffel
7. Zaid Alyafeai
8. Sabrina J. Mielke

Citations in the corpus

• Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP (2021, background): “As Henderson (2020) puts it: “It remains to find effective neural architectures for learning the set of entities jointly with the rest of the neural model, and for generalising such meth-ods from the level of character strings to higher levels of representation”.”