BERTScore: Evaluating Text Generation with BERT

By Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, Yoav Artzi. 2019

9,100 citations recorded by the source corpus.

We propose BERTScore, an automatic evaluation metric for text generation. Analogously to common metrics, BERTScore computes a similarity score for each token in the candidate sentence with each token in the reference sentence. However, instead of exact matches, we compute token similarity using contextual embeddings. We evaluate using the outputs of 363 machine translation and image captioning systems. BERTScore correlates better with human judgments and provides stronger model selection performance than existing metrics. Finally, we use an adversarial paraphrase detection task to show that BERTScore is more robust to challenging examples when compared to existing metrics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Yejin Choi
  4. Luke S. Zettlemoyer
  5. Percy Liang
  6. Graham Neubig
  7. Noah A. Smith
  8. Caiming Xiong

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Mohit Bansal
  4. Trevor Darrell
  5. Ming-Hsuan Yang
  6. Xiaogang Wang
  7. Jianfeng Gao
  8. Cordelia Schmid

In Natural Language Processing

  1. Mohit Bansal
  2. Yejin Choi
  3. Luke S. Zettlemoyer
  4. Jianfeng Gao
  5. Graham Neubig
  6. Percy Liang
  7. Noah A. Smith
  8. Hannaneh Hajishirzi

In Evaluation & Benchmarks

  1. Mohit Bansal
  2. Yejin Choi
  3. Elizabeth Clark
  4. Jianfeng Gao
  5. Sebastian Gehrmann
  6. Luke S. Zettlemoyer
  7. Thibault Sellam
  8. Hannaneh Hajishirzi

Citations in the corpus