Contrastive Self-Supervised Learning for Commonsense Reasoning

By Tassilo Johannes Klein, Moin Nabi. 2020

64 citations recorded by the source corpus.

We propose a self-supervised method to solve Pronoun Disambiguation and Winograd Schema Challenge problems. Our approach exploits the characteristic structure of training corpora related to so-called “trigger” words, which are responsible for flipping the answer in pronoun disambiguation. We achieve such commonsense reasoning by constructing pair-wise contrastive auxiliary predictions. To this end, we leverage a mutual exclusive loss regularized by a contrastive margin. Our architecture is based on the recently introduced transformer networks, BERT, that exhibits strong performance on many NLP benchmarks. Empirical results show that our method alleviates the limitation of current supervised approaches for commonsense reasoning. This study opens up avenues for exploiting inexpensive self-supervision to achieve performance gain in commonsense reasoning tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Elisa Ricci
2. Enrico Fini
3. Moin Nabi
4. Xavier Alameda-Pineda
5. Ming Ding
6. Dan Xu
7. Paolo Rota
8. Guanglei Yang

In Computer Vision and Pattern Recognition

1. Elisa Ricci
2. Enrico Fini
3. Xavier Alameda-Pineda
4. Dan Xu
5. Moin Nabi
6. Ming Ding
7. Paolo Rota
8. Guanglei Yang

Citations in the corpus

• Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, methodology): “This training scheme has been successfully employed in different fields such as image and video classification [25], [53], [54], as well as natural language processing [55], [56] and audio representation learning [57], [58], [59].”