Linguistic Structured Sparsity in Text Categorization

By Dani Yogatama, Noah A. Smith. 2014

55 citations recorded by the source corpus.

We introduce three linguistically motivated structured regularizers based on parse trees, topics, and hierarchical word clusters for text categorization. These regularizers impose linguistic bias in feature weights, enabling us to incorporate prior knowledge into conventional bagof-words models. We show that our structured regularizers consistently improve classification accuracies compared to standard regularizers that penalize features in isolation (such as lasso, ridge, and elastic net regularizers) on a range of datasets for various text prediction problems: topic classification, sentiment analysis, and forecasting.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Regularizing Text Categorization with Clusters of Words (2016, positive): “Our results are inline and confirm that of (Yogatama and Smith, 2014a) showing the advantages of using structured regularizers in the text categorization task.”
Sparse Overcomplete Word Vector Representations (2015, methodology): “Our work builds on recent explorations of sparsity as a useful form of inductive bias in NLP and machine learning more broadly (Kazama and Tsu-jii, 2003; Goodman, 2004; Friedman et al., 2008; Glorot et al., 2011; Yogatama and Smith, 2014, inter alia ).”
Understanding Neural Networks through Representation Erasure (2016, methodology): “, 2008), which has been widely employed in many NLP tasks, such as document classification (Yogatama and Smith, 2014) and providing rationales for neural model interpretation (Lei et al.”
Bridging CNNs, RNNs, and Weighted Finite-State Machines (2018, background): “Yogatama and Smith (2014) introduced structured sparsity, which encodes linguistic information into the regularization of a model, thus allowing to visualize the contribution of different bag-of-word features.”
Rotated Word Vector Representations and their Interpretability (2017, methodology): “The dataset has six types of questions including about person, location, etc. Topic Classiﬁcation (Topics: Sp.) Next, we obtain the 20 newsgroup dataset to classify Sports (baseball vs. hockey) topics (Yogatama and Smith, 2014; Faruqui et al., 2015).”
Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization (2017, background): “stream task (which uses word embeddings as inputs), as we do here. Past work on sparse linear models has shown the potential of exploiting linguistic knowledge in statistical NLP models. For example, Yogatama and Smith (2014) used external resources to inform structured, grouped regularization of lo”
Evaluation of Word Vector Representations by Subspace Alignment (2015, methodology): “8 Each task involves categorizing a document according to two related categories with training/dev/test split in accordance with Yogatama and Smith (2014).”