SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization

By Yixin Liu, Peng Liu. 2021

295 citations recorded by the source corpus.

In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i.e., quality estimation) assisted by contrastive learning. Experimental results show that, with minor modification over existing top-scoring systems, SimCLS can improve the performance of existing top-performing models by a large margin. Particularly, 2.51 absolute improvement against BART and 2.50 over PEGASUS w.r.t ROUGE-1 on the CNN/DailyMail dataset, driving the state-of-the-art performance to a new level. We have open-sourced our codes and results: https://github.com/yixinL7/SimCLS. Results of our proposed models have been deployed into ExplainaBoard platform, which allows researchers to understand our systems in a more fine-grained way.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Summarization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Pengfei Liu
  2. Graham Neubig
  3. Mohit Bansal
  4. Xiang Ren
  5. Xipeng Qiu
  6. Percy Liang
  7. Dragomir R. Radev
  8. Yixin Liu

In Representation & Self-Supervised Learning

  1. Graham Neubig
  2. Xiang Ren
  3. Mohit Bansal
  4. David Wan
  5. Lu Wang
  6. Shuyang Cao
  7. Pengfei Liu
  8. Bill Yuchen Lin

In Summarization

  1. Yixin Liu
  2. Pengfei Liu
  3. Graham Neubig
  4. Peng Liu
  5. Weizhe Yuan
  6. Dragomir R. Radev
  7. Mohit Bansal
  8. David Wan

Citations in the corpus