Towards a Seamless Integration of Word Senses into Downstream NLP Applications

By Mohammad Taher Pilehvar, José Camacho-Collados, Roberto Navigli, Nigel Collier. 2017

47 citations recorded by the source corpus.

Lexical ambiguity can impede NLP systems from accurate understanding of semantics. Despite its potential benefits, the integration of sense-level information into NLP systems has remained understudied. By incorporating a novel disambiguation algorithm into a state-of-the-art classification model, we create a pipeline to integrate sense-level information into downstream NLP applications. We show that a simple disambiguation of the input text can lead to consistent performance improvement on multiple topic categorization and polarity detection datasets, particularly when the fine granularity of the underlying sense inventory is reduced and the document is sufficiently large. Our results also point to the need for sense representation research to focus more on in vivo evaluations which target the performance in downstream NLP applications rather than artificial benchmarks.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations (2018, methodology): “This coarsening of the WordNet sense inventory has been shown particularly useful in downstream applications (R¨ud et al., 2011; Severyn et al., 2013; Flekova and Gurevych, 2016; Pilehvar et al., 2017).”
Dynamic Integration of Background Knowledge in Neural NLU Systems (2017, background): “2017) seamlessly incorporate information about word senses into their representations before solving the downstream NLU task, which is similar.”
WiC: 10, 000 Example Pairs for Evaluating Context-Sensitive Representations (2018, background): “been shown particularly useful in downstream applications (Rüd et al., 2011; Severyn et al., 2013; Flekova and Gurevych, 2016; Pilehvar et al., 2017).”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, methodology): “Pilehvar et al. (2017) proposed a method based on a shared space of word and knowledge-based sense embeddings, introducing a simple graph-based disambiguation step prior to their integration into a neural network architecture for text classiﬁcation.”
LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations (2019, background): “Recently, Şahin et al. (2019) proposed 16 multilingual probing tasks along with the corresponding datasets and found correlations between certain probing and downstream tasks and demonstrated their efficacy as diagnostic tools.”
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training (2016, background): “of words and senses in the same vector space proves essential for applying these knowledgebased sense embeddings in downstream applications, particularly for their integration into neural architectures (Pilehvar et al., 2017).”