Learning by Semantic Similarity Makes Abstractive Summarization Better
By WonJin Yoon, Yoonsun Yeo, Minbyul Jeong, Bong-Jun Yi, Jaewoo Kang. 2020
18 citations recorded by the source corpus.
One of the obstacles of abstractive summarization is the presence of various potentially correct predictions. Widely used objective functions for supervised learning, such as cross-entropy loss, cannot handle alternative answers effectively. Rather, they act as a training noise. In this paper, we propose Semantic Similarity strategy that can consider semantic meanings of generated summaries while training. Our training objective includes maximizing semantic similarity score which is calculated by an additional layer that estimates semantic similarity between generated summary and reference summary. By leveraging pre-trained language models, our model achieves a new state-of-the-art performance, ROUGE-L score of 41.5 on CNN/DM dataset. To support automatic evaluation, we also conducted human evaluation and received higher scores relative to both baseline and reference summaries.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Pengfei Liu
- Graham Neubig
- Pierre Colombo
- Pablo Piantanida
- Guillaume Staerman
- Atabak Ashfaq
- Chloé Clavel
- Pranav Narayan Gour
In Natural Language Processing
- Pengfei Liu
- Pierre Colombo
- Pablo Piantanida
- Graham Neubig
- Guillaume Staerman
- Atabak Ashfaq
- Chloé Clavel
- Pranav Narayan Gour
Citations in the corpus
- ColdGANs: Taming Language GANs with Cautious Sampling Strategies (2020, neutral): “A similar result was reported by Yoon et al. [52] (refer to Table 2 in their paper).”
- How well do you know your summarization datasets? (2021, unknown): “, 2018) (7) SEMSIM (Yoon et al., 2020) (8) UNILM (Dong et al.”
- Re-evaluating Evaluation in Text Summarization (2020, background): “…et al., 2018), T5 (Raffel et al., 2019), Unilm-v1 (Dong et al., 2019), Unilm-v2 (Dong et al., 2019), twoStageRL (Zhang et al., 2019a), pre-SummAbs (Liu and Lapata, 2019b), preSummAbs-ext (Liu and Lapata, 2019b) BART (Lewis et al., 2019) and Semsim (Yoon et al., 2020) as abstrac-tive systems.”
- Automatic Text Evaluation through the Lens of Wasserstein Barycenters (2021, methodology): “Following the footstep of (Zhang et al., 2019), the Moverscore (Zhao et al., 2019b) applied optimal transport to the output of Contextualized Encoders (CE) such as BERT or ELMo.”