A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data

By Rie Kubota Ando, Tong Zhang. 2005

1,708 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019, background): “Learning widely applicable representations of words has been an active area of research for decades, including non-neural (Brown et al., 1992; Ando and Zhang, 2005; Blitzer et al., 2006) and neural (Mikolov et al.”
Semi-supervised Sequence Learning (2015, background): “We believe that this semi-supervised approach [1] is superior to other unsupervised sequence learning methods, e.”
MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, background): “…to provide language representations and features to the downstream tasks, which includes word-level representations (Brown et al., 1992; Ando & Zhang, 2005; Blitzer et al., 2006; Collobert & Weston, 2008; Mikolov et al., 2013; Pennington et al., 2014) and sentence-level representations…”
An Overview of Multi-task Learning (2018, methodology): “Thenthereare manystudiestoanalyzegeneralizationboundsofdif-ferentMTLapproaches,includinge.g. [7,134]for thefeaturetransformapproach,[135]forthefea-tureselectionapproach,[24,135–138]forthelow-rankapproach,[136]forthetask-relationlearning approach,and[138]forthedirtyapproach.”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “Transfer learning is related to both multi-task learning [32, 4, 28, 10, 11, 45, 24, 2] and learning novel categories from few examples [49, 19, 21, 35, 51, 22, 5, 13, 50, 47, 31].”
Unsupervised Visual Representation Learning by Context Prediction (2015, background): “How can one write an objective function to encourage a representation to capture, for example, objects, if none of the objects are labeled? Interestingly, in the text domain, context has proven to be a powerful source of automatic supervisory signal for learning representations [3, 38, 9, 37].”
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition (2013, methodology): “Later works such as Argyriou et al. (2006) developed efﬁcient frameworks for optimizing representations from related tasks, and Ando & Zhang (2005) explored how to transfer parameter manifolds to new tasks.”
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction (2016, background): “The idea of learning representations from multisensory signals also shows up in structure learning [2], cotraining [3], and multi-view learning [46].”
From Frequency to Meaning: Vector Space Models of Semantics (2010, methodology): “In addition to unsupervised (clustering) and supervised (classification) machine learning, vectors from a VSM may also be used in semi-supervised learning (Ando & Zhang, 2005; Collobert & Weston, 2008).”
Deep learning via semi-supervised embedding (2008, methodology): “We note that an alternative multi-task learning scheme is presented in (Ando & Zhang, 2005) and applied to neural networks in (Ahmed et al., 2008) which instead constructs auxiliary supervised tasks from unlabeled * data by constructing tasks with labels y .”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, methodology): “In particular, Quattoni et al. (2007) use the multi-task learning framework of Ando and Zhang (2005) to learn a discriminative latent space from Web images and associated captions.”
Search-based structured prediction (2009, neutral): “The best reported results to date using the full data set are due to (Ando and Zhang 2005).”
A theory of learning from different domains (2010, background): “Although these bounds have a more limited scope than ours, they can sometimes yield useful results even when the optimal predictors for each task (or domain in the case of Daumé 2007) are quite different (Baxter 2000; Ando and Zhang 2005).”
Semi-Cyclic Stochastic Gradient Descent (2019, background): “For example, one can consider shared sparsity (e.g. Turlach et al., 2005), shared linear features or low-rank structure (Ando & Zhang, 2005), shared deep features (e.g. Caruana, 1997), shared kernel or low nuclearnorm structure (e.g. Argyriou et al., 2008; Amit et al., 2007), low-norm additive…”