WiC: 10, 000 Example Pairs for Evaluating Context-Sensitive Representations

By Mohammad Taher Pilehvar, José Camacho-Collados. 2018

50 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019, background): “…al., 2019]) • Coreference resolution (WNLI and WSC [Levesque et al., 2012]) • Sentence completion (COPA [Roemmele et al., 2011]) • Word sense disambiguation (WIC [Pilehvar and Camacho-Collados, 2018]) • Question answering (MultiRC [Khashabi et al., 2018], ReCoRD [Zhang et al., 2018], BoolQ [Clark…”
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts (2021, neutral): “…(Wang et al., 2019) BoolQ (Clark et al., 2019), CB (de Marneffe et al., 2019), COPA (Gordon et al., 2012), RTE (Dagan et al., 2006), WiC (Pilehvar & Camacho-Collados, 2018), WSC (Levesque et al., 2012), MultiRC (Khashabi et al., 2018), ReCoRD (Zhang et al., 2018) Natural Language…”
The Power of Scale for Parameter-Efficient Prompt Tuning (2021, background): “, 2009), WiC (Pilehvar and CamachoCollados, 2018), and WSC (Levesque et al.”
How many data points is a prompt worth? (2021, background): “Prompting enjoys a substantial advantage on every task, except for WiC as is reported in previous results (Schick and Schütze, 2020b).”
GPT Understands, Too (2021, background): “…& MultiRC (Khashabi et al., 2018)), textual entailment (CB (De Marneffe et al., 2019) & RTE (Dagan et al., 2005)), co-reference resolution (WiC (Pilehvar and Camacho-Collados, 2018)), causal reasoning (COPA (Roemmele et al., 2011)), and word sense disambiguation (WSC (Levesque et al., 2012)).”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, result): “This goes in line with a recent study analyzing how well sense and contextualized representations capture meaning in context (Pilehvar & Camacho-Collados, 2018).”
A Comprehensive Overview of Large Language Models (2023, unknown): “BloombergGPT FinPile [128], The Pile [290], C4 [11], Wikipedia [17] Financial Data, BIG-bench [236], MMLU [221], ARC, PiQA [216], CommonsenseQA [254], BoolQ [224], OpenBookQA [304], RACE [225], MultiRC [319], ReCoRD [320], ANLI [227], RTE [235], COPA [252], WIC [240], WinoGrad [220], WinoGrande [219”
SenseBERT: Driving Some Sense into BERT (2019, methodology): “…performance on a supersense-based variant of the SemEval WSD test sets standardized in Raganato et al. [2017], and on the Word in Context (WiC) task [Pilehvar and Camacho-Collados, 2018] (included in the recently introduced SuperGLUE benchmark [Wang et al., 2019]), both directly relying on the…”
OLMo: Accelerating the Science of Language Models (2024, background): “These tasks are headqa_en (Vilares and Gómez-Rodríguez, 2019), logiqa (Liu et al., 2020), mrpc (Dolan and Brock-ett, 2005), qnli (Wang et al., 2018), wic (Pilehvar and Camacho-Collados, 2018), and wnli (Wang et al., 2018).”
Towards a robust experimental framework and benchmark for lifelong language learning (2021, background): “Specifically, we find the following 264 10 datasets to meet this criterion: part-of-speech tagging (UDPOS) [46], named entity recognition 265 (PANNER) [25], few-shot relation extraction (FewRel) [13], Word-in-Context (WiC) [27], boolean 266 question answering (BoolQ) [7], DBPedia ontology classifica”
HyperGrid Transformers: Towards A Single Model for Multiple Tasks (2021, methodology): “The datasets in SuperGLUE (Wang et al., 2019) (Dagan et al., 2005; Bar-Haim et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009), Word-in-Context (WiC) (Pilehvar and os’e Camacho-Collados, 2018), and WSC (Winograd Schema Challenge) (Levesque et al., 2012).”