Learning Bayesian Networks with Local Structure

By Nir Friedman, Moisés Goldszmidt. 1996

In this paper we examine a novel addition to the known methods for learning Bayesian networks from data that improves the quality of the learned networks. Our approach explicitly represents and learns the local structure in the conditional probability tables (CPTs), that quantify these networks. This increases the space of possible models, enabling the representation of CPTs with a variable number of parameters that depends on the learned local structures. The resulting learning procedure is capable of inducing models that better emulate the real complexity of the interactions present in the data. We describe the theoretical foundations and practical aspects of learning local structures, as well as an empirical evaluation of the proposed method. This evaluation indicates that learning curves characterizing the procedure that exploits the local structure converge faster than these of the standard procedure. Our results also show that networks learned with local structure tend to be more complex (in terms of arcs), yet require less parameters.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms (2019, methodology): “There, it has been common to use Minimum Description Length (MDL) principles to score and search over models Lam and Bacchus (1993); Friedman and Goldszmidt (1998), or the Bayesian Information Criterion (BIC) to search for models with high relative posterior probability Heckerman et al. (1995).”
Dependency Networks for Inference, Collaborative Filtering, and Data Visualization (2000, background): “An important di(cid:14)culty with this approach is that if either y or z are rare (a common occurrence when Y or Z contain many variables), many iterations are required for an accurate probability estimate.”
Learning Bayesian network parameters from small data sets: application of Noisy-OR gates (2001, methodology): “imate CPTs using fewer parameters and learning distributio ns with fewer parameters is in general more reliable [6].”
Inferring cellular networks – a review (2007, methodology): “The regression trees capture the local structure in the data, whereas the DAG describes the global structure [40,41].”
Modelling Gene Expression Data using Dynamic Bayesian Networks (2006, methodology): “Alternatively, we can use a decision tree [BFGK96], or a table of parent values along with their associated non-zero probabilities [FG96], to represent the CPD.”
An introduction and survey of estimation of distribution algorithms (2011, background): “Speciﬁcally, hBOA uses Bayesian networks with local structures (Chickering, Heckerman, & Meek, 1997; Friedman & Goldszmidt, 1999) to allow feasible learning and sampling of more complex networks than would be possible with convential Bayesian networks.”
Naive Bayes models for probability estimation (2005, background): “( Friedman & Goldszmidt, 1996; Chickering et al., 1997) have found that decision trees allow more complex models to be stored in less space.”
A Scoring Function for Learning Bayesian Networks based on Mutual Information and Conditional Independence Tests (2006, methodology): “The scoring functions are based on different principles, such as entropy and information (Chow and Liu, 1968; Herskovits and Cooper, 1990), the minimum description length (Bouckaert, 1993, 1995; Friedman and Goldszmidt, 1996; Lam and Bacchus, 1994; Suzuki, 1993), or Bayesian approaches (Buntine, 199”
Tree Induction vs. Logistic Regression: A Learning-Curve Analysis (2003, unknown): “For example, a Bayesian network learner has a different probability learning subtask at each node; manual parameter tuning for each is infeasible, so automatic (push-button) techniques typically are used (Friedman and Goldszmidt, 1996).”
Bayesian Network Classifiers (1997, methodology): “…from data is a rapidly growing ﬁeld of research that has seen a great deal of activity in recent years, including work by Buntine (1991, 1996), Cooper and Herskovits (1992), Friedman and Goldszmidt (1996c), Lam and Bacchus (1994), Hecker-man (1995), and Heckerman, Geiger, and Chickering (1995).”
Fast, frugal, and fit: Simple heuristics for paired comparison (2002, methodology): “In larger data sets (1000 objects or more) this type of Bayesian network becomes more robust (Friedman and Goldszmit, 1996).”
Tree Induction for Probability-Based Ranking (2003, background): “…and resear hers,for example in spee h re ognition (Jelinek, 1997), as node models in pet-mlj-final.tex; 5/06/2002; 18:04; p.2 3Bayesian networks (Friedman & Goldszmidt, 1996), in the re entlyintrodu ed dependen y-network representation and its appli ation to ollaborative ltering and other areas…”
Learning Graphical Models with Mercer Kernels (2002, methodology): “Although the performance is empirically robust to the setting of these parameters, learning those parameters from data would not only provide better and more consistent performance, but it would also provide a principled way to learn graphical models with local structure [ 15 ].”