The bnlearn Package. September 21, 2007

Transcription

1 Type Package Title Bayesian network structure learning Version 0.3 Date Depends R (>= 2.5.0) Suggests snow Author The bnlearn Package September 21, 2007 Maintainer <marco.scutari@gmail.com> Bayesian network structure learning via constraint-based (also known as conditional independence ) algorithms. This package implements the Grow-Shrink (GS) algorithm, the Incremental Association (IAMB) algorithm, the Interleaved-IAMB (Inter-IAMB) algorithm and the Fast-IAMB (Fast-AIMB) algoirthm. License GPL version 2 or newer LazyLoad yes LazyData yes R topics documented: bnlearn-package choose.direction compare fast.iamb gs iamb inter.iamb learning.test mb plot.bn Index 17 1

2 2 bnlearn-package bnlearn-package Bayesian network constraint-based structure learning. Details Bayesian network learning via constraint-based algorithms. Package: bnlearn Type: Package Version: 0.3 Date: License: GPLv2 or later This package implements some constraint-based algorithms for learning the structure of Bayesian networks. Also known as conditional independence learners, they are all optimized derivatives of the Inductive Causation algorithm (Verma and Pearl, 1991). These algorithms differ in the way they detect the Markov blankets of the variables, which in turn are used to compute the structure of the Bayesian network. Proofs of correctness are present in the respective papers. Available learning algorithms Grow-Shrink (gs): based on the Grow-Shrink Markov Blanket algorithm, a forward-selection technique for Markov blanket detection. Incremental Association (iamb): based on the Markov blanket detection algorithm of the same name, which uses stepwise selection. Fast Incremental Association (fast.iamb): a variant of IAMB which uses speculative forward-selection to reduce the number of conditional independence tests. Interleaved Incremental Association (inter.iamb): another variant of IAMB which interleaves the backward selection with the forward one to avid false positives (i.e. nodes erroneously included in the Markov blanket). This package includes three implementations of each algorithm: an optimized implementation, which uses backtracking to roughly halve the number of independence tests. This is the one used by default, when the optimized parameter is set to TRUE. an unoptimized implementation which is better at uncovering possible erratic behaviour of the statistical tests. To use this one the optimized parameter must be set to FALSE. a cluster-aware implementation, which requires a running cluster set up with the makecluster function from the snow package. To use this on pass the cluster object to the various functions via the cluster parameter; the optimized parameter will be ignored.

3 bnlearn-package 3 Their computational complexity is polynomial in the number of tests, usually O(N 2 ) (O(N 4 ) in the worst case scenario). The execution time also scales linearly with the size of the data set. Available (conditional) independence tests The conditional independence tests used in constraint-based algorithms in practice are statistical tests on the data set. Available tests (and the respective labels) are: categorical data (multinomial distribution) mutual information (mi): an information-theoretic distance measure. Is proportional to the log-likelihood ratio (they differ by a 2n factor) and is related to the deviance of the tested models. fast mutual information (fmi): a variant of the mutual information which is set to zero when there aren t at least five data per parameter. Cochran-Mantel-Haenszel (mh): a stratified independence test, included for testing purposes only. See mantelhaen.test in package stats. numerical data (multivariate normal distribution) linear correlation (cor): linear correlation. Fisher s Z (zf): a transformation of the linear correlation with asymptotic normal distribution. Used by commercial software (such as TETRAD II) for the PC algorithm (an R implementation is present in the pcalg) package on CRAN. Whitelist and blacklist support All algorithms support arc whitelisting and blacklisting: blacklisted arcs are never present in the graph. arcs whitelisted in one direction only (i.e. A B is whitelisted but B A is not) have the respective reverse arcs blacklisted, and are always present in the graph. arcs whitelisted in both directions (i.e. both A B and B A are whitelisted) are present in the graph, but their direction is set by the learning algorithm. Any arc whitelisted and blacklisted at the same time is assumed to be whitelisted, and is thus removed from the blacklist. Error detection and correction: the strict mode Optimized implementations of learning algorithms often hide learning errors, usually in the Markov blanket detection step, due to the use of backtracking. On the other hand in the unoptimized implementations the Markov blanket and neighbour detection of each node is completely independent from the rest of the learning process. Thus it may happen that the Markov blanket or the neighbourhoods are not symmetric (i.e. A is in the Markov blanket but not vice versa), or that some arc directions conflict with each other. The strict parameter enables some measure of error correction, which may help to retrieve a good model even when the learning process would fail: if strict is set to TRUE, every error stops the learning process and results in an error message.

4 4 bnlearn-package if strict is set to FALSE: The bn S3 class 1. v-structures are applied to the network structure in lowest-p.value order; if any arc is already oriented in the opposite direction, the v-structure is discarded. 2. nodes which causes asymmetries in any Markov blanket are removed from that Markov blanket. 3. nodes which causes asymmetries in any neighbourhood are removed from that neighbourhood. An object of class bn is a list containing at least the following components: nodes: a list. Each element is named after a node and contains the following elements: mb: the Markov blanket blanket of the node (an array of character strings). nbr: the neighbourhood of the node (an array of character strings). arcs: the arcs of the Bayesian network (a two-column matrix, whose column are labeled from and to) whitelist: a sanitized copy of the whitelist parameter (a two-column matrix, whose column are labeled from and to). blacklist: a sanitized copy of the blacklist parameter (a two-column matrix, whose column are labeled from and to). test: the label of the conditional independence test used by the learning algorithm (a character string). alpha: the target nominal type I error rate (a numerical value). ntests: the number of conditional independence tests used in the learning (an integer value). algo: the label of the learning algorithm used (a character string). discrete: is the network a discrete (multinomial) one (a boolean value)? Maintainer: marco.scutari@gmail.com References A. Agresti. Categorical Data Analysis. John Wiley & Sons, Inc., D. Margaritis. Learning Bayesian Network Model Structure from Data. PhD thesis, School of Computer Science, Carnegie-Mellon University, Pittsburgh, PA, May Available as Technical Report CMU-CS J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., I. Tsamardinos, C. F. Aliferis, and A. Statnikov. Algorithms for large scale markov blanket discovery. In Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference, pages AAAI Press, 2003.

5 bnlearn-package 5 S. Yaramakala, D. Margaritis. Speculative Markov Blanket Discovery for Optimal Feature Selection. In Proceedings of the Fifth IEEE International Conference on Data Mining, pages IEEE Computer Society, Examples library(bnlearn) data(learning.test) ## Simple learning # first try the Grow-Shrink algorithm res = gs(learning.test) # plot the network structure. plot(res) # now try the Incremental Association algorithm. res2 = iamb(learning.test) # plot the new network structure. plot(res2) # the network structures seem to be identical, don't they? compare(res, res2) # [1] TRUE # how many tests each of the two algorithms used? res$ntests # [1] 44 res2$ntests # [1] 51 # and the unoptimized implementation of these algorithms? ## Not run: gs(learning.test, optimized = FALSE)$ntests # [1] 90 ## Not run: iamb(learning.test, optimized = FALSE)$ntests # [1] 124 ## Blacklist and whitelist use # the arc B - F should not be there? blacklist = data.frame(from = c("b", "F"), to = c("f", "B")) blacklist # from to # 1 B F # 2 F B res3 = gs(learning.test, blacklist = blacklist) plot(res3) # force E - F direction (E -> F). whitelist = data.frame(from = c("e"), to = c("f")) whitelist # from to # 1 E F res4 = gs(learning.test, whitelist = whitelist) plot(res4) # use both blacklist and whitelist. res5 = gs(learning.test, whitelist = whitelist, blacklist = blacklist) plot(res5)

6 6 choose.direction ## Debugging # use the debugging mode to see the learning algorithm # in action. res = gs(learning.test, debug = TRUE) # log the learning process for future reference. sink(file = "learning-log.txt") res = gs(learning.test, debug = TRUE) sink() # if something seems wrong, try the unoptimized version # in strict mode (inconsistencies trigger errors): res = gs(learning.test, optimized = FALSE, strict = TRUE, debug = TRUE) # or disable strict mode to try to fix errors on the fly: res = gs(learning.test, optimized = FALSE, strict = FALSE, debug = TRUE) choose.direction Try to infer the direction of an undirected arc Check both possible directed arcs for existence, and choose the one with the lowest p-value. choose.direction(x, arc, data, debug = FALSE) Arguments x arc data debug an object of class "bn". a character string array of length 2, the labels of two nodes of the graph. a data frame, containing the variables in the model. a boolean value. If TRUE a lot of debugging output is printed; otherwise the function is completely silent. Value choose.direction returns invisibly an updated copy of x.

7 compare 7 Examples data(learning.test) res = gs(learning.test) # the arc between E and F has no direction plot(res) res = choose.direction(res, c("e", "F"), learning.test, debug = TRUE) # * testing E - F for direction. # > testing E -> F with conditioning set ' '. # > p-value is e-44. # > testing F -> E with conditioning set ' '. # > p-value is e-08. removing F -> E. plot(res, highlight = c("e", "F")) compare Compare two different Bayesian networks Compare two different Bayesian networks. compare(r1, r2, debug = FALSE) Arguments r1 r2 debug an object of class "bn". another object of class "bn". a boolean value. If TRUE a lot of debugging output is printed; otherwise the function is completely silent. Value A boolean value: TRUE if the objects describe the same network structure, FALSE otherwise.

8 8 fast.iamb Examples data(learning.test) res = gs(learning.test) # the arc between E and F has no direction plot(res, highlight = c("e", "F")) res2 = choose.direction(res, c("e", "F"), learning.test) plot(res, highlight = c("e", "F")) compare(res, res2, debug = TRUE) # * arcs in r1 not present in r2: # [1] "F -> E" # * arcs in r2 not present in r1: # character(0) # [1] FALSE fast.iamb Fast Incremental Association (Fast-IAMB) learning algorithm Estimate the equivalence class of a directed acyclic graph (DAG) from data using the Fast Incremental Association (FAST-IAMB) Constraint-based algorithm. fast.iamb(x, cluster = NULL, whitelist = NULL, blacklist = NULL, test = "mi", alpha = 0.05, debug = FALSE, optimized = TRUE, strict = TRUE, direction = FALSE) Arguments x cluster whitelist blacklist test alpha a data frame, containing the variables in the model. See bnlearn-package for details. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs to be included in the graph. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs not to be included in the graph. a character string, the label of the conditional independence test to be used in the algorithm. Possible values are mi (mutual information), mh (Cochran- Mantel-Haenszel), fmi (fast mutual information), cor (linear correlation), zf (Fisher s Z). See bnlearn-package for details. a numerical value, the target nominal type I error rate.

9 gs 9 debug optimized strict a boolean value. If TRUE a lot of debugging output is printed; otherwise the function is completely silent. a boolean value. See bnlearn-package for details. a boolean value. If TRUE conflicting results in the structure learning generate an error; otherwise they result in a warning. direction a boolean value. If TRUE each possible direction of each undirected arc is tested, and the one with the lowest p-value is accepted as the true direction for that arc. Value An object of class bn. See bnlearn-package for details. References S. Yaramakala, D. Margaritis. Speculative Markov Blanket Discovery for Optimal Feature Selection. In Proceedings of the Fifth IEEE International Conference on Data Mining, pages IEEE Computer Society, See Also gs, fast.iamb, iamb. gs Grow-Shrink (GS) learning algorithm Estimate the equivalence class of a directed acyclic graph (DAG) from data using the Grow-Shrink (GS) Constraint-based algorithm. gs(x, cluster = NULL, whitelist = NULL, blacklist = NULL, test = "mi", alpha = 0.05, debug = FALSE, optimized = TRUE, strict = TRUE, direction = FALSE)

10 10 gs Arguments x cluster whitelist blacklist test alpha debug optimized strict a data frame, containing the variables in the model. See bnlearn-package for details. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs to be included in the graph. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs not to be included in the graph. a character string, the label of the conditional independence test to be used in the algorithm. Possible values are mi (mutual information), mh (Cochran- Mantel-Haenszel), fmi (fast mutual information), cor (linear correlation), zf (Fisher s Z). See bnlearn-package for details. a numerical value, the target nominal type I error rate. a boolean value. If TRUE a lot of debugging output is printed; otherwise the function is completely silent. a boolean value. See bnlearn-package for details. a boolean value. If TRUE conflicting results in the structure learning generate an error; otherwise they result in a warning. direction a boolean value. If TRUE each possible direction of each undirected arc is tested, and the one with the lowest p-value is accepted as the true direction for that arc. Value An object of class bn. See bnlearn-package for details. References D. Margaritis. Learning Bayesian Network Model Structure from Data. PhD thesis, School of Computer Science, Carnegie-Mellon University, Pittsburgh, PA, May Available as Technical Report CMU-CS See Also iamb, fast.iamb, inter.iamb.

11 iamb 11 iamb Incremental Association (IAMB) learning algorithm Estimate the equivalence class of a directed acyclic graph (DAG) from data using the Incremental Association (IAMB) Constraint-based algorithm. iamb(x, cluster = NULL, whitelist = NULL, blacklist = NULL, test = "mi", alpha = 0.05, debug = FALSE, optimized = TRUE, strict = TRUE, direction = FALSE) Arguments x Value cluster whitelist blacklist test alpha debug optimized strict a data frame, containing the variables in the model. See bnlearn-package for details. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs to be included in the graph. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs not to be included in the graph. a character string, the label of the conditional independence test to be used in the algorithm. Possible values are mi (mutual information), mh (Cochran- Mantel-Haenszel), fmi (fast mutual information), cor (linear correlation), zf (Fisher s Z). See bnlearn-package for details. a numerical value, the target nominal type I error rate. a boolean value. If TRUE a lot of debugging output is printed; otherwise the function is completely silent. a boolean value. See bnlearn-package for details. a boolean value. If TRUE conflicting results in the structure learning generate an error; otherwise they result in a warning. direction a boolean value. If TRUE each possible direction of each undirected arc is tested, and the one with the lowest p-value is accepted as the true direction for that arc. An object of class bn. See bnlearn-package for details.

12 12 inter.iamb References I. Tsamardinos, C. F. Aliferis, and A. Statnikov. Algorithms for large scale markov blanket discovery. In Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference, pages AAAI Press, See Also gs, fast.iamb, inter.iamb. inter.iamb Interleaved Incremental Association (Inter-IAMB) learning algorithm Estimate the equivalence class of a directed acyclic graph (DAG) from data using the Interleaved Incremental Association (Inter-IAMB) Constraint-based algorithm. inter.iamb(x, cluster = NULL, whitelist = NULL, blacklist = NULL, test = "mi", alpha = 0.05, debug = FALSE, optimized = TRUE, strict = TRUE, direction = FALSE) Arguments x cluster whitelist blacklist test alpha debug optimized strict a data frame, containing the variables in the model. See bnlearn-package for details. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs to be included in the graph. a data frame with two columns (optionally labeled "from" and "to"), containing a set of arcs not to be included in the graph. a character string, the label of the conditional independence test to be used in the algorithm. Possible values are mi (mutual information), mh (Cochran- Mantel-Haenszel), fmi (fast mutual information), cor (linear correlation), zf (Fisher s Z). See bnlearn-package for details. a numerical value, the target nominal type I error rate. a boolean value. If TRUE a lot of debugging output is printed; otherwise the function is completely silent. a boolean value. See bnlearn-package for details. a boolean value. If TRUE conflicting results in the structure learning generate an error; otherwise they result in a warning. direction a boolean value. If TRUE each possible direction of each undirected arc is tested, and the one with the lowest p-value is accepted as the true direction for that arc.

13 learning.test 13 Value An object of class bn. See bnlearn-package for details. References S. Yaramakala, D. Margaritis. Speculative Markov Blanket Discovery for Optimal Feature Selection. In Proceedings of the Fifth IEEE International Conference on Data Mining, pages IEEE Computer Society, See Also gs, fast.iamb, iamb. learning.test Synthetic dataset to test learning algorithms This a synthetic dataset used as a test case in the bnlearn package. learning.test Examples data(learning.test) str(learning.test) # 'data.frame': 5000 obs. of 6 variables: # $ A: Factor w/ 3 levels "a","b","c": # $ B: Factor w/ 3 levels "a","b","c": # $ C: Factor w/ 3 levels "a","b","c": # $ D: Factor w/ 3 levels "a","b","c": # $ E: Factor w/ 3 levels "a","b","c": # $ F: Factor w/ 2 levels "a","b": gs(learning.test)

14 14 mb mb Utility functions to manipulate graphs Extract various quantities of interest from an object of class bn. mb(x, node) nbr(x, node) arcs(x) nodes(x) amat(x) parents(x, node) Arguments x node an object of class "bn". a character string, the label of a node. Value mb, nbr, nodes and parents return an array of character strings. arcs returns a matrix of two columns of character strings. amat returns a matrix of 0/1 numeric values. Examples data(learning.test) res = gs(learning.test) # the Markov blanket of A. mb(res, "A") # [1] "B" "D" "C" # the neighbourhood of F. nbr(res, "F") # [1] "B" "E" # the arcs in the graph. arcs(res) # from to # [1,] "A" "D"

15 plot.bn 15 # [2,] "B" "A" # [3,] "C" "D" # [4,] "E" "B" # [5,] "E" "F" # [6,] "F" "B" # [7,] "F" "E" # the nodes of the graph. nodes(res) # [1] "A" "B" "C" "D" "E" "F" # the adjacency matrix for the nodes of the graph. amat(res) # A # B # C # D # E # F # the parents of D. parents(res, "D") # [1] "A" "C" plot.bn Plot a Bayesian Network Plot the graph associated with a Bayesian network. ## S3 method for class 'bn': plot(x, ylim = c(0,600), xlim = ylim, radius = 250, arrow = 35, highlight = NULL, color = "red",...) Arguments x ylim xlim radius arrow highlight color an object of class bn. a numeric vector with two components containing the range on y-axis. a numeric vector with two components containing the range on x-axis. a numeric value containing the radius of the nodes. a numeric value containing the length of the arrow heads. an array of character strings, representing the labels of the nodes (and corresponding arcs) to be highlighted. an integer or character string (the highlight colour).... other parameters to be passed through to plotting functions.

16 16 plot.bn Note The following graphical parameters are always overridden: axes is set to FALSE. xlab is set to an empty string. ylab is set to an empty string. Examples data(learning.test) res = gs(learning.test) plot(res) # highlight node B and related arcs. plot(res, highlight = "B") # highlight B and its Markov blanket. plot(res, highlight = c("b", mb(res, "B"))) # a more compact plot. par(oma = rep(0, 4), mar = rep(0, 4), mai = rep(0, 4), plt = c(0.06, 0.94, 0.12, 0.88)) plot(res)

17 Index Topic datasets learning.test, 13 Topic graphs fast.iamb, 8 gs, 9 iamb, 10 inter.iamb, 12 Topic hplot plot.bn, 15 Topic htest choose.direction, 6 Topic models fast.iamb, 8 gs, 9 iamb, 10 inter.iamb, 12 Topic multivariate fast.iamb, 8 gs, 9 iamb, 10 inter.iamb, 12 Topic package bnlearn-package, 1 Topic utilities compare, 7 mb, 14 iamb, 2, 9, 10, 10, 13 inter.iamb, 2, 10, 11, 12 learning.test, 13 mantelhaen.test, 3 mb, 14 nbr (mb), 14 nodes (mb), 14 parents (mb), 14 plot.bn, 15 stats, 3 amat (mb), 14 arcs (mb), 14 bnlearn (bnlearn-package), 1 bnlearn-package, 8 12 bnlearn-package, 1 choose.direction, 6 compare, 7 fast.iamb, 2, 8, 9 11, 13 gs, 2, 9, 9, 11, 13 17