Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization

By Ye Zhang, Matthew Lease, Byron C. Wallace. 2017

15 citations recorded by the source corpus.

A fundamental advantage of neural models for NLP is their ability to learn representations from scratch. However, in practice this often means ignoring existing external linguistic resources, e.g., WordNet or domain specific ontologies such as the Unified Medical Language System (UMLS). We propose a general, novel method for exploiting such resources via weight sharing. Prior work on weight sharing in neural networks has considered it largely as a means of model compression. In contrast, we treat weight sharing as a flexible mechanism for incorporating prior knowledge into neural models. We show that this approach consistently yields improved performance on classification tasks compared to baseline strategies that do not exploit weight sharing.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension (2018, background): “There is also some research on incorporating knowledge into NLP tasks such as reading comprehension (Lin et al., 2017; Yang and Mitchell, 2017) neural machine translation (Zhang et al., 2017a) and text classiﬁcation (Zhang et al., 2017b) etc.”
Neural information retrieval: at the end of the early years (2017, methodology): “2012), then later adapted to text-related tasks in NLP and IR (Collobert et al. 2011; Kalchbrenner et al. 2014; Kim 2014; Zhang and Wallace 2015; Zhang et al. 2017).”