Learning from Sparse Data by Exploiting Monotonicity Constraints

By Eric Altendorf, Angelo C. Restificar, Thomas G. Dietterich. 2005

122 citations recorded by the source corpus.

When training data is sparse, more domain knowledge must be incorporated into the learning algorithm in order to reduce the effective size of the hypothesis space. This paper builds on previous work in which knowledge about qualitative monotonicities was formally represented and incorporated into learning algorithms (e.g., Clark & Matwin's work with the CN2 rule learning algorithm). We show how to interpret knowledge of qualitative influences, and in particular of monotonicities, as constraints on probability distributions, and to incorporate this knowledge into Bayesian network learning algorithms. We show that this yields improved accuracy, particularly with very small training sets (e.g. less than 10 examples).

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Learning Credible Models (2017, background): “In the first case, we only penalize |θ̂ |[1], effectively recovering the l∞ norm.”
An optimization approach to learning falling rule lists (2017, neutral): “Altendorf et al., 2005; Ben-David, 1995; Feelders and Pardoel, 2003), but they focus mostly on enforcing the monotonic relationship between certain attributes and ordinal class labels. In addition, our work also relates to those that underline the importance of the interpretability of models (Freita”
Falling Rule Lists (2014, neutral): “|{ y n } ; { x n } , H ).”
Human-Guided Learning for Probabilistic Logic Models (2018, background): “While there has been work on learning with qualitative constraints for Bayesian networks (Altendorf et al., 2005; Yang and Natarajan, 2013), they have not yet been applied to the relational case.”
Building Acceptable Classification Models (2010, background): “Although almost all research into this field is focused on the monotonicity constraint [2, 7, 21, 40, 47, 61], the taxonomy of possible domain constraints, shown in Table 3.”
Interacting meaningfully with machine learning systems: Three experiments (2009, background): “Since most learning algorithms involve an optimization problem either during training or during inference, this domain knowledge is added to the model by incorporating these constraints during the optimization (Altendorf et al. 2005; Fung et al. 2002).”
Comprehensible classification models: a position paper (2014, background): “For researchers who need benchmarking datasets for evaluating classification algorithms that respect monotonicity constraints, information on monotonic constraints for five UCI datasets is given in [2].”
Toward harnessing user feedback for machine learning (2007, unknown): “[1]).”
Domain knowledge integration in data mining using decision tables: case studies in churn prediction (2009, methodology): “This is for example the case for Bayesian network learning algorithms (Altendorf et al, 2005), AntMiner+ (Martens et al , 2006) and neural networks (Velikova et al , 2006).”
Nearest Neighbour Classification with Monotonicity Constraints (2008, background): “Examples are: classification trees [7,12,22], neural networks [3,26], and Bayesian networks [1,13].”
Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring (2008, methodology): “Several adaptions to existing classification techniques have been put forward to deal with monotonicity, such as for Bayesian learning [1], classification trees [7,21,24], classification rules [43] and neural networks [54,66]; e.”