The Teaching Dimension of Linear Learners
By Ji Liu, Xiaojin Zhu, H. Ohannessian. 2015
69 citations recorded by the source corpus.
Teaching dimension is a learning theoretic quantity that specifies the minimum training set size to teach a target model to a learner. Previous studies on teaching dimension focused on version-space learners which maintain all hypotheses consistent with the training data, and cannot be applied to modern machine learners which select a specific hypothesis via optimization. This paper presents the first known teaching dimension for ridge regression, support vector machines, and logistic regression. We also exhibit optimal training sets that match these teaching dimensions. Our approach generalizes to other linear learners.
Subfields: Education
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Education
Citations in the corpus
- Certified Defenses for Data Poisoning Attacks (2017, background): “A defender who trains naïvely on the full (clean + poisoned) data D c ∪ D p is doomed to failure, as even a single poisoned point can in some cases arbitrarily change the model (Liu and Zhu, 2016; Park et al., 2017).”
- Optimization Models for Machine Learning: A Survey (2019, background): “The teaching dimension of linear learners, such as Ridge regression, SVM, and logistic regression has been recently discussed in [153].”
- Learning to Teach (2018, background): “(Liu & Zhu, 2016)) define the teaching dimension of several learners.”
- Teacher Improves Learning by Selecting a Training Subset (2018, background): “This is interesting as it is known that when S is dense, one can select extremely sparse super teaching sets, as small as a few items, to teach effectively [28].”
- Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications (2018, methodology): “While machine teaching has been applied to regression and classification (Zhu 2015; Liu and Zhu 2016), little work has addressed machine teaching for sequential decision-making tasks such as learning from demonstration via IRL.”
- Learning to Reweight with Deep Interactions (2021, background): “…into two branches: (1) One is to transfer knowledge (e.g., image classification, machine translation) from the teacher to student (Zhu 2015; Liu and Zhu 2016) and the teacher model aims to teach the student model with minimal cost, like data selection (Liu et al. 2017, 2018) in supervised…”
- Learning to Teach with Dynamic Loss Functions (2018, background): “For example, the machine teaching [56, 34, 35] literature targets at building the smallest training set to obtain a pre-given optimal student model.”
- Towards Understanding Knowledge Distillation (2019, background): “Transferring knowledge via machine teaching techniques is extremely effective: perfect transfer is often possible from a small finite teaching set (Zhu, 2013; Liu & Zhu, 2016).”
- Teaching a black-box learner (2019, background): “…has been growing interest in settings where both dimensions are negative: for instance, the learner is a convex empirical risk minimizer (e.g., Liu & Zhu, 2016), or the teacher does not target a specific learner (Zhu et al., 2017), or the teacher does not know the learner’s hyper-parameters or…”
- An Overview of Machine Teaching (2018, background): “The teacher can still teach with a single labeled example, but must nudge the y value upward off the line in anticipation of learner shrinkage [28]: D2 = {(x1, θx1 + λθ ∗ x1 )}, x1 6 = 0.”