Recent Trends in Deep Learning Based Natural Language Processing

By Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria. 2017

3,089 citations recorded by the source corpus.

Deep learning methods employ multiple processing layers to learn hierarchical representations of data and have produced state-of-the-art results in many domains. Recently, a variety of model designs and methods have blossomed in the context of natural language processing (NLP). In this paper, we review significant deep learning related models and methods that have been employed for numerous NLP tasks and provide a walk-through of their evolution. We also summarize, compare and contrast the various models and put forward a detailed understanding of the past, present and future of deep learning in NLP.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Overcoming Multi-Model Forgetting (2019, neutral): “Deep neural networks have been very successful for tasks such as visual recognition (Xie & Yuille, 2017) and nat-ural language processing (Young et al., 2017), and much recent work has addressed the training of models that can generalize across multiple tasks (Caruana, 1997).”
Artificial Intelligence for the Metaverse: A Survey (2022, background): “for some specific NLP tasks, such as text parsing, semantic labeling, context retrieval, language interpretation, and dialogue generation [32].”
An Attentive Survey of Attention Models (2019, background): “First, these models are now the state-of-the-art [Young et al., 2018] for multiple tasks such as Machine Translation, Question Answering, Sentiment Analysis, Part-of-Speech tagging, Constituency Parsing and Dialogue Systems.”
Transformer Network for Remaining Useful Life Prediction of Lithium-Ion Batteries (2022, neutral): “Recently, remarkable success has been achieved by deep learning in various ﬁelds, such as recommender systems [16]–[18], Computer Vision (CV) [19]–[21], and Natural Language Processing (NLP) [22]–[24].”
Why Gradient Clipping Accelerates Training: A Theoretical Justification for Adaptivity (2019, methodology): “In this work we experiment with LSTM LM (Sundermeyer et al., 2012), which has been an important building block for many popular NLP models (Young et al., 2017).”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, methodology): “Rather than looking at the input sequence in vanilla attention models, self-attention [136] in NLP is used to look at the surrounding words in a sequence to obtain more contextually sensitive word representations [152].”
Large-Scale Cross-Category Analysis of Consumer Review Content on Sales Conversion Leveraging Deep Learning (2019, positive): “We use deep learning NLP techniques (Goldberg 2016; LeCun, Bengio, and Hinton 2015; Young et al. 2018) as supervised learning to extract the six dimensions of quality and price information from the reviews scalably.”
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2019, background): “Language model (LM) has been a central module for natural language generation (NLG) tasks (Young et al., 2017) such as open-ended language generation (Radford et al., 2018; Brown et al., 2020), machine translation (Wu et al., 2017), dialogue response generation (Li et al., 2017), image captioning…”