Structure compilation: trading structure for features

By Percy Liang, Hal Daumé Iii, Daniel Klein. 2008

88 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Model Architectures

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, neutral): “This result is in line with existing NLP studies comparing sentence-level and wordlevel likelihoods (Liang et al., 2008).”
Policy Distillation (2015, background): “The DQN algorithm requires long training times to train on a single task.”
Globally Normalized Transition-Based Neural Networks (2016, background): “For a detailed analysis of the trade-offs between structural features in CRFs and more powerful local classiﬁers without structural constraints, see Liang et al. (2008); in these experiments local classiﬁers are unable to reach the performance of CRFs on problems such as parsing and named entity…”
Understanding Knowledge Distillation in Non-autoregressive Machine Translation (2019, background): “2 SEQUENCE-LEVEL KNOWLEDGE DISTILLATION Knowledge distillation (Liang et al., 2008; Hinton et al., 2015) was originally proposed for training a weaker student classifier on the targets predicted from a stronger teacher model.”
Practical Very Large Scale CRFs (2010, unknown): “…of output labels), heuristically reduce the use of features, especially of features that test pairs of adjacent labels 1 , and/or propose heuristics to simulate contextual dependencies, via extended tests on the observations (see discussions in, eg., (Punyakanok et al., 2005; Liang et al., 2008)).”
Pointwise Prediction for Robust, Adaptable Japanese Morphological Analysis (2011, methodology): “However, while structure does provide valuable information, Liang et al. (2008) have shown that gains provided by structured prediction can be largely recovered by using a richer feature set.”
Access to Unlabeled Data can Speed up Prediction Time (2011, background): “Maybe most relevant to our setup of using unlabeled data to improve the runtime of the learned classifier is work of Liang et al. (2008).”
Fast and Accurate Entity Recognition with Iterated Dilated Convolutions (2017, background): “(Liang et al., 2008).”
Learning with Compositional Semantics as Structural Inference for Subsentential Sentiment Analysis (2008, background): “( Liang et al., 2008) Moilanen and Pulman (2007), on the other hand, handle the structural nature of the interactions more directly using the ideas from compositional semantics (e.g., Montague (1974), Dowty et al. (1981)).”
Fast and Accurate Sequence Labeling with Iterated Dilated Convolutions (2017, methodology): “CRF prediction explicitly reasons about interactions among neighboring output tags, whereas prediction in the first model compiles this reasoning into the feature extraction step (Liang et al., 2008).”
Towards Understanding Knowledge Distillation (2019, background): “Ideas underpinning distillation have a long history dating back to the work of Ba & Caruana (2014); Bucilua et al. (2006); Craven & Shavlik (1996); Li et al. (2014); Liang et al. (2008).”
Effect of Non-linear Deep Architecture in Sequence Labeling (2013, methodology): “Lastly, models like CRF actually already capture some of the non-linearity in the input space through the interactions of latent variables (Liang et al., 2008), and it is unclear how much additional gain we would get by explicitly modeling the non-linearity in local inputs.”
What’s Going On in Neural Constituency Parsers? An Analysis (2018, background): “1 Liang et al. (2008) provides theoretical results suggesting they may be useful for learning efﬁciently.”
Distillation-Based Training for Multi-Exit Architectures (2019, background): “Although similar techniques have been widely considered already in [2, 4, 5, 25, 28], distillation in its most well-known form was introduced only relatively recently in [15].”
An Introduction to Conditional Random Fields (2010, methodology): “has been exploited to approximate CRFs using per-position classifiers with an expanded feature set [71].”
Active Learning Literature Survey (2009, background): “Several others (Buciluǎ et al., 2006; Liang et al., 2008) have adapted this idea to “compress” large, computationally expensive model classes (such as complex ensembles or structured-output models) into smaller, more efficient model classes (such as neural networks or simple linear classifiers).”