Learning Monolingual Compositional Representations via Bilingual Supervision

By Ahmed Elgohary, Marine Carpuat. 2016

Bilingual models that capture the semantics of sentences are typically only evaluated on cross-lingual transfer tasks such as cross-lingual document categorization or machine translation. In this work, we evaluate the quality of the monolingual representations learned with a variant of the bilingual compositional model of Hermann and Blunsom (2014), when viewing translations in a second language as a semantic annotation as the original language text. We show that compositional objectives based on phrase translation pairs outperform compositional objectives based on bilingual sentences and on monolingual paraphrases.

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Allyson Ettinger
2. Ahmed Elgohary
3. Philip Resnik
4. Samuel R. Bowman
5. Yonatan Belinkov
6. Alex Warstadt
7. Hinrich Schütze
8. Ellie Pavlick

In Representation & Self-Supervised Learning

1. Ahmed Elgohary
2. Philip Resnik
3. Allyson Ettinger
4. Samuel R. Bowman
5. Alex Warstadt
6. Ellie Pavlick

Citations in the corpus

• Probing for semantic evidence of composition by means of simple classification tasks (2016, background): “Elgohary and Carpuat (2016) meet this need by discarding all training sentences that include any observed bigrams in their evaluation sentences.”