Simple and Effective Paraphrastic Similarity from Parallel Translations

By John Wieting, Kevin Gimpel, Graham Neubig, Taylor Berg-Kirkpatrick. 2019

52 citations recorded by the source corpus.

We present a model and methodology for learning paraphrastic sentence embeddings directly from bitext, removing the time-consuming intermediate step of creating para-phrase corpora. Further, we show that the resulting model can be applied to cross lingual tasks where it both outperforms and is orders of magnitude faster than more complex state-of-the-art baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Self-training Improves Pre-training for Natural Language Understanding (2020, methodology): “In Table 10, we compare our sentence embedding method to previous approaches including BERT (Mean) (Devlin et al., 2018), InferSent (Conneau et al., 2017), GenSen (Subramanian et al., 2018), USE (Cer et al., 2018), Sentence-BERT (Reimers and Gurevych, 2019), uSIF (Ethayarajh, 2018a), Charagram (Wiet”
Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing (2020, methodology): “…NMT model as a generative paraphraser.3 Semantic Similarity Parallel corpora in many language pairs have been used to produce fixed-size, multilingual sentence representations (Schwenk and Douze, 2017; Wieting et al., 2017; Artetxe and Schwenk, 2018; Wieting et al., 2019; Raganato et al., 2019).”
Beyond BLEU:Training Neural Machine Translation with Semantic Similarity (2019, background): “Recently, in (Wieting et al., 2019) it has been shown that strong performance on semantic similarity tasks can also be achieved using bitext directly without the need for backtranslation. there was nothing preventing the model from learning to generate long sentences, often at the expense of…”
A Bilingual Generative Transformer for Semantic Sentence Embedding (2019, methodology): “Training these approaches generally relies on discriminative objectives defined on paraphrase data (Ganitkevitch et al., 2013; Wieting and Gimpel, 2018) or bilingual data (Wieting et al., 2019b; Chidambaram et al., 2019; Yang et al., 2020).”
Explicit Alignment Objectives for Multilingual Bidirectional Encoders (2020, background): “Our sentence-level alignment objective encourages prediction of the correct translations within a minibatch for a given source sentence, which is inspired by work on learning multilingual sentence representations (Yang et al., 2019a; Wieting et al., 2019).”
Parallel Sentence Mining by Constrained Decoding (2020, background): “Wieting et al. (2019) obtain sentence embeddings from sub-word embeddings and train a simpler model to distinguish positive and negative examples.”
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval (2021, methodology): “There, pretrained language models (LMs) such as BERT [12] or RoBERTa [36] are fine-tuned via twin-network architectures on auxiliary tasks such as semantic textual similarity [45, 25], paraphrasing [55], response retrieval [57, 22, 21, 25], or translation ranking [10, 16].”