Composition of Conditional Random Fields for Transfer Learning

By Charles Sutton, Andrew McCallum. 2005

59 citations recorded by the source corpus.

Many learning tasks have subtasks for which much training data exists. Therefore, we want to transfer learning from the old, general-purpose subtask to a more specific new task, for which there is often less data. While work in transfer learning often considers how the old task should affect learning on the new task, in this paper we show that it helps to take into account how the new task affects the old. Specifically, we perform joint decoding of separately-trained sequence models, preserving uncertainty between the tasks and allowing information from the new task to affect predictions on the old task. On two standard text data sets, we show that joint decoding outperforms cascaded decoding.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, methodology): “On the other hand, Sutton and McCallum (2005a) obtain results somewhat below the state-of-the-art using joint decoding for SRL and syntactic parsing.”
To transfer or not to transfer (2005, background): “Sutton and McCallum [7] demonstrated effective transfer by “cascading” a class of graphical models, with the prediction from one classifier serving as a feature for the next one in the cascade.”
Finding advertising keywords on web pages (2006, unknown): “, [5, 20]), named-entity recognition systems (e.”
Multitask Learning for Spoken Language Understanding (2006, methodology): “They have shown improvements using MTL for named entity extraction using ACE and CoNLL data sets [7].”
A unified architecture for natural language processing: deep neural networks with multitask learning (2008, background): “In ( Sutton & McCallum, 2005a ) the authors showed that one could learn the tasks independently, hence using dierent training sets, by only leveraging predictions jointly in a test time decoding step, and still obtain improved results.”
Encoding linear models as weighted finite-state transducers (2014, background): “Composition of sequence models and joint inference can often be preferable to a simple cascading of the models where the single-best output of one serves as the input to the other [9]; and also for model combination.”
Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data (2004, unknown): “, 2004) and transfer learning (Sutton and McCallum, 2005).”