Small Statistical Models by Random Feature Mixing

By Kuzman Ganchev, Mark Dredze. 2008

83 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Dense Representations for Entity Retrieval (2019, positive): “, 2014), and we use 5M hash buckets for out-of-vocabulary unigrams and bigrams(Ganchev and Dredze, 2008).”
Compressing Neural Networks with the Hashing Trick (2015, background): “Let w 1 , . . . , w T denote the weight vectors of the respective learning tasks, then if t (cid:48) (cid:54) = t a classiﬁer for task t (cid:48) does not interfere with a hashed input for task t ; i.e. w t φ t (cid:48) ( x ) ≈ 0 .”
Modelling input texts: from Tree Kernels to Deep Learning (2015, methodology): “[49, 146], that aim at transforming high dimensional kernel spaces into the linear space, where fast learning methods can be applied.”
Natural Language Processing with Small Feed-Forward Networks (2017, methodology): “We use a distinct feature group g for each n - gram length N , and control the size V g directly by applying random feature mixing (Ganchev and Dredze, 2008).”
Hash Kernels for Structured Data (2009, unknown): “Random Feature Mixing Ganchev and Dredze (2008) provide empirical evidence that using hashing can eliminate alphabet storage and reduce the number of parameters without severely impacting model performance.”
Hash Kernels (2009, background): “Random Feature Mixing Ganchev and Dredze (2008) provide empirical evidence that using hashing can eliminate alphabet storage and reduce the number of parameters without severely impacting model performance. In addition, Langford et al. (2007) released the Vowpal Wabbit fast online learning software ”
Sparse Non-negative Matrix Language Modeling (2015, methodology): “To control memory usage, we employ a feature hashing technique (Langford et al., 2007; Ganchev and Dredze, 2008) where we store the meta-feature weights in a flat hash table θ of predefined size.”
Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models (2009, methodology): “For all text experiments, we used random feature mixing [9, 20] to control the size of the feature space.”
A Fast, Compact, Accurate Model for Language Identification of Codemixed Text (2018, unknown): “The feature id of an n -gram string x is given by H ( x )mod V g (Ganchev and Dredze, 2008), where H is a well-behaved hash function.”
Geolocation for Twitter: Timing Matters (2016, methodology): “Alternatively, we use feature hashing (to a 31-bit feature space) which can be a form of regularization as feature collisions mitigate overfitting (Ganchev and Dredze, 2008; Weinberger et al., 2009).”
Embedding Methods for Fine Grained Entity Type Classification (2015, methodology): “String features are randomly hashed to a value in 0 to 999,999, which simplifies feature extraction and adds some additional regularization (Ganchev and Dredze, 2008).”