BERT: A Review of Applications in Natural Language Processing and Understanding

By M. V. Koroteev. arXiv.org, 2021

533 citations recorded by the source corpus.

Koroteev M.V., Financial University under the government of the Russian Federation, Moscow, Russia mvkoroteev@fa.ru Abstract: In this review, we describe the application of one of the most popular deep learning-based language models BERT. The paper describes the mechanism of operation of this model, the main areas of its application to the tasks of text analytics, comparisons with similar models in each task, as well as a description of some proprietary models. In preparing this review, the data of several dozen original scientific articles published over the past few years, which attracted the most attention in the scientific community, were systematized. This survey will be useful to all students and researchers who want to get acquainted with the latest advances in the field of natural language text analysis.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/879eaab2275a364549809560b42f0fef357ebbce

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sicong Leng
2. Li Bing
3. Shijian Lu
4. Chunyan Miao
5. Guanzheng Chen
6. Hang Zhang
7. Xin Li

Citations in the corpus

• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, methodology): “The development of Vision-Language Models (VLMs) has transitioned from being rooted in BERT-based language decoders [13, 31, 47] for merging visual and textual data [34, 36, 59, 64], to a notable advancement ushered by the integration of Large Language Models (LLMs) [4, 7, 10, 11, 17, 53, 61–63].”
• Generative AI at Work (2023, unknown): “…for each conversation, we can compare the similarity of conversations by looking at the cosine similarities of their associated vectors; this common approach yields a score of 0 if two pieces of text are semantically orthogonal and a score of 1 if they have the same meaning (Koroteev, 2021).”
• Open-Source Embedding Models: A Comprehensive Survey of Techniques, Benchmarks, and Applications (2026, unknown): “BERT, developed by Google, is a prime example [37].”