A Tutorial on Learning with Bayesian Networks

By David E. Heckerman. 1999

A Bayesian network is a graphical model that encodes probabilistic relationships among variables of interest. When used in conjunction with statistical techniques, the graphical model has several advantages for data analysis. One, because the model encodes dependencies among all variables, it readily handles situations where some data entries are missing. Two, a Bayesian network can be used to learn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequences of intervention. Three, because the model has both a causal and probabilistic semantics, it is an ideal representation for combining prior knowledge (which often comes in causal form) and data. Four, Bayesian statistical methods in conjunction with Bayesian networks offer an efficient and principled approach for avoiding the overfitting of data. In this paper, we discuss methods for constructing Bayesian networks from prior knowledge and summarize Bayesian statistical methods for using data to improve these models. With regard to the latter task, we describe methods for learning both the parameters and structure of a Bayesian network, including techniques for learning with incomplete data. In addition, we relate Bayesian-network methods for learning to techniques for supervised and unsupervised learning. We illustrate the graphical-modeling approach using a real-world case study.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “Consequently, BNs are challenging to apply for solving practical problems [86].”
Venture: a higher-order probabilistic programming platform with programmable inference (2014, background): “, 2007; Rasmussen and Williams, 2006; Griffiths and Ghahramani, 2005), probabilistic models over worlds with a priori unknown numbers of objects, models for learning the structure of graphical models (Heckerman, 1998; Friedman and Koller, 2003; Mansinghka et al., 2006), models for inductive learning”
Model based Bayesian Exploration (1999, methodology): “We show that this is possible by adopting results from Bayesian learning of probabilistic models, such as Bayesian networks (Heckerman 1998).”
Advanced Lectures on Machine Learning (2004, methodology): “If the parameters and structure are unknown they can be learned from the data [49].”
Learning Bayesian networks from data: An information-theory based approach (2002, background): “This suggests that the best BN is the one that best fits the data, and leads to the scoringbased learning algorithms, which each seek a structure that maximizes the Bayesian, MDL or Kullback–Leibler (KL) entropy scoring function [17,28].”
Data Analysis with Bayesian Networks: A Bootstrap Approach (1999, background): “In the last decade there has been a great deal of research focused on learning Bayesian networks from data [2, 12].”
A Simple Approach for Finding the Globally Optimal Bayesian Network Structure (2006, background): “Bayesian network structures (see, e.g., (Heckerman, 1996)), or simply networks, for a variable set V = {V1, . . . , Vn} are directed acyclic graphs (DAGs) with exactly one node per variable.”
Applications of Deep Learning and Reinforcement Learning to Biological Data (2017, methodology): “Popular supervised methods include: Artificial Neural Network (ANN) [13] and its variants, Support Vector Machines [14] and linear classifiers [15], Bayesian Statistics [16], k-Nearest Neighbors [17], Hidden Markov Model [18], and Decision Trees [19].”
Being Bayesian about Network Structure (2000, background): “The simplest and therefore most common choice is a uniform prior over structures [8].”
Learning Bayesian Belief Network Classifiers: Algorithms and System (2001, background): “This suggests that the best BN is the one that best fits the data, and leads to the scoring-based learning algorithms, that seek a structure that maximizes the Bayesian, MDL or Kullback-Leibler (KL) entropy scoring function (Heckerman 1995; Cooper and Herskovits 1992).”
Bayesian models of cognition (2008, methodology): “For technical introductions to the methods and challenges of learning causal graphical models, N ( e see Heckerman (1998) and Glymour and Cooper (1999).”
A Practical Approach to Microarray Data Analysis (2003, methodology): “The Bayesian network (Heckerman, 1999; Jensen, 2001) is a kind of probabilistic graphical model, which represents the joint probability distribution over a set of variables of interest.”
Causal Feature Selection (2007, unknown): “[15]).”
Learning Conventions in Multiagent Stochastic Domains using Likelihood Estimates (1996, background): “resented by the Dirichlet parameters N j 1 ; N j n, capturing a density function (see [6]) over such strategies.”