Speed regularization and optimality in word classing

31 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Efficient softmax approximation for GPUs (2016, background): “We observe empirically that putting all the clusters in the leaves of the tree leads to a signiﬁcant drop of performance (around 5 − 10% performance drop, Mikolov et al., 2011c; Zweig & Makarychev, 2013).”
Joint Language and Translation Modeling with Recurrent Neural Networks (2013, methodology): “We obtain word-classes using a version of Brown-Clustering with an additional regularization term to optimize the runtime of the language model (Brown et al., 1992; Zweig and Makarychev, 2013).”
Chinese Poetry Generation with Recurrent Neural Networks (2014, methodology): “This strategy outperformed better known frequency-based classing methods (Zweig and Makarychev, 2013) on our task.”
From Feedforward to Recurrent LSTM Neural Networks for Language Modeling (2015, methodology): “Regarding the use of word classes in neural network LMs, these clustering algorithms were compared in [35], and it was found that perplexity-based word classes resulted in considerable improvements over the unigram frequency variant when used for the output layer factorization of a neural network LM”
An introduction to computational networks and the computational network toolkit (invited talk) (2014, background): “Word class can be derived using frequency based heuristics [33]or more sophisticated way[34].”