THE reconstruction of the AND-OR expression of a

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 37 Couplig Logical Aalysis of Data ad Shadow Clusterig for Partially Defied Positive Boolea Fuctio Recostructio Marco Muselli, Member, IEEE, ad Erico Ferrari Abstract The problem of recostructig the AND-OR expressio of a partially defied positive Boolea fuctio (pdpbf) is solved by adoptig a ovel algorithm, deoted by LSC, which combies the advatages of two efficiet techiques, Logical Aalysis of Data (LAD) ad Shadow Clusterig (SC). The kerel of the approach followed by LAD cosists i a breadth-first eumeratio of all the prime implicats whose degree is ot greater tha a fixed maximum d. I cotrast, SC adopts a effective heuristic procedure for retrievig the most promisig logical products to be icluded i the resultig AND-OR expressio. Sice the computatioal cost required by LAD prevets its applicatio eve for relatively small dimesios of the iput domai, LSC employs a depth-first approach, with asymptotically liear memory occupatio, to aalyze the prime implicats havig degree ot greater tha d. I additio, the theoretical aalysis proves that LSC presets almost the same asymptotic time complexity as LAD. Extesive simulatios o artificial bechmarks validate the good behavior of the computatioal cost exhibited by LSC, i agreemet with the theoretical aalysis. Furthermore, the pdpbf retrieved by LSC always shows a better performace, i terms of complexity ad accuracy, with respect to those obtaied by LAD. Idex Terms Positive Boolea fuctio, logic sythesis, Logical Aalysis of Data, Shadow Clusterig. Ç 1 INTRODUCTION THE recostructio of the AND-OR expressio of a Boolea fuctio startig from (a portio of) its truth table is a basic task i the realizatio of digital circuits. I geeral, the complexity of the resultig logical etwork is required to be as low as possible, which leads oe to prefer techiques possessig the ability of retrievig fuctios with miimal or ear-miimal expressios, accordig to some complexity measure. A wide variety of algorithms has bee proposed i the literature for this aim: from methods that perform a exhaustive search i the space of cosistet Boolea fuctios, such as the Quie-McCluskey procedure [1] or the cosesus method [2], to heuristic techiques that look for ear-optimal solutios while reducig the computatioal cost. The algorithms ESPRESSO [3] ad MINI [4] belog to this secod class. More recetly, Boolea fuctio recostructio has also bee cosidered for the solutio of patter recogitio problems with biary iputs. I these cases, oly a limited portio of the truth table, called traiig set, is available ad the goal amouts to fidig the Boolea fuctio that miimizes some proper criterio while behavig correctly i the examples at had. Possible criteria deped o simplicity, accordig to the Occam s razor priciple, or o some prior iformatio about the problem to be solved.. The authors are with the Istitute of Electroics, Computer ad Telecommuicatio Egieerig, Italia Natioal Research Coucil, Via De Marii 6, Geoa 16149, Italy. E-mail: {muselli, ferrari}@ieiit.cr.it. Mauscript received 8 Aug. 2008; revised 4 Apr. 2009; accepted 8 Aug. 2009; published olie 20 Nov. 2009. For iformatio o obtaiig reprits of this article, please sed e-mail to: tkde@computer.org, ad referece IEEECS Log Number TKDE-2008-08-0412. Digital Object Idetifier o. 10.1109/TKDE.2009.206. Eve machie learig problems with obiary iputs ca be solved through the recostructio of a Boolea fuctio. It is sufficiet to adopt suitable discretizatio techiques [5], [6], [7] ad biary codigs for mappig ordered ad omial variables (assumig values iside a fiite, oordered set, e.g., colors, geometrical shapes) ito Boolea strigs. For example, thermometer ad oly-oe codes have bee proved to preserve orderig ad distace whe used to trasform ordered ad omial attributes, respectively, [8]. However, the dimesio of the iput space turs out to be very high ( >100 or >1000), thus, prevetig the adoptio of exhaustive methods that retrieve the whole collectio of AND-OR expressios cosistet with the traiig set. O the other had, heuristic techiques for logic sythesis geerate Boolea fuctios with poor accuracy, sice they do ot take ito accout the geeralizatio ability of the resultig solutio. Two differet approaches have bee suggested to deal with this situatio. Followig the priciple of the Occam s razor, the Logical Aalysis of Data (LAD) [7] procedure geerates all the logical products (AND operatios), cosistet with the traiig set, which iclude a umber of operads ot greater tha a maximum d. The, if the disjuctio of these logical products fails to give the correct output for some examples of the traiig set, a simple heuristic is adopted to fid other AND operatios (with more tha d factors) that make the resultig Boolea fuctio cosistet with the whole traiig set. O the other had, the Hammig Clusterig (HC) algorithm [9], [10] employs more refied heuristics for retrievig oe at a time the logical products that cotribute to form the resultig Boolea fuctio. Differet criteria ca 1041-4347/11/$26.00 ß 2011 IEEE Published by the IEEE Computer Society

38 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 be adopted to pursue differet targets for the sythesis. For example, if the umber of operads is to be miimized, the Lowest Cube criterio must be used, whereas the Maximum coverig Cube criterio has the aim of icreasig the umber of examples i the traiig set correctly classified by each AND operatio. Now it ca be see that LAD caot treat problems with high, sice i its exhaustive part, LAD could have to examie Oð d Þ differet terms, which ca require a huge computatio time eve for d ¼ 4 whe 1000. I additio, sice LAD adopts a breadth-first eumerative techique to examie the logical products, eve the memory requiremet ca become excessive. Fially, if the value of d is too small (as is the case whe icreases), may examples of the traiig set remai misclassified at the ed of the exhaustive part, leavig to the simple heuristic method, the heavy work of retrievig most of the eeded AND operatios. The executio of extesive trials has show that HC is able to achieve good performace, i terms of accuracy, for differet values of, ot sufferig from computatioal limitatios, sice its executio time icreases at most quadratically with [9]. Nevertheless, wheever possible, the additio of a exhaustive part, as i LAD, ca improve sigificatly the quality of the resultig Boolea fuctio. Thus, i this paper, we propose a ew algorithm for recostructig a partially defied Boolea fuctio (pdbf), which combies the exhaustive part of LAD with the heuristic from HC. To avoid a excessive memory allocatio requiremet, a depth-first approach is adopted i the exhaustive search; however, a particular housekeepig method cotrols the icrease i the computatioal cost. It has bee verified [8] that a class of machie learig problems ca be solved through partially defied positive Boolea fuctio (pdpbf) recostructio. Therefore, i this paper, we describe the versio of the ew algorithm dealig with pdpbf sythesis. Sice the heuristic procedure (similar to HC) for derivig pdpbf is called Shadow Clusterig (SC) [11], [12], the proposed techique is amed LSC. Also stadard pdbf recostructio ca be treated by LSC through the adoptio of a proper mappig. It is worth otig that positive Boolea fuctios, whose Disjuctive Normal Form (DNF) does ot iclude egated variables, have recetly attracted the attetio of may researchers because of possible applicatios i the solutio of several practical problems, ragig from the implemetatios of stack filters [13], [14] to the choice of strategies i game theory [15]. I particular, the learability of positive Boolea fuctios has bee the subject of several papers [16], [17], [18], [19], [20], [21], [22]. I most of these works, the whole truth table is supposed to be available ad ca be queried through a oracle; i this case, the mootoe dualizatio algorithm ca be show to recostruct the desired fuctio i quasipolyomial time [23]. Uder this hypothesis, a efficiet method for positive Boolea fuctio sythesis has also bee proposed i [24]. The remaider of the paper is structured as follows: basic defiitios ad otatios itroduced i Sectio 2 are employed to aalyze the problem of recostructig a pdpbf (Sectio 3). The geeral framework of the LAD approach is Fig. 1. (a) Diagram of the poset f0; 1g 3, (b) lower shadow LðaÞ, ad (c) upper shadow UðaÞ for the elemet a ¼ 101. the preseted i Sectio 4, together with implemetatio details regardig its applicatio to pdpbf sythesis. Sectio 5 describes the proposed LSC techique, which combies a exhaustive part derived from a depth-first versio of LAD with a heuristic part usig SC. The computatioal cost of LAD ad LSC is theoretically examied i Sectio 6 through a worst-case aalysis that derives the asymptotic behavior of the executio time ad of the memory occupatio. A experimetal compariso of the performace exhibited by LAD ad LSC o pdpbf ad pdbf recostructio is reported i Sectio 7. Fially, some discussios ad coclusios are preseted i Sectio 8. 2 BASIC DEFINITIONS AND NOTATIONS Cosider the set f0; 1g of the biary vectors with legth ; to improve readability, its elemets will be deoted, heceforth, by strigs of bits. The usual operatios AND (logical product), OR (logical sum), ad NOT (complemetatio) ca be defied o it, makig f0; 1g a Boolea algebra. O the other had, a stadard partial orderig ca be imposed o f0; 1g by writig x y if ad oly if x þ y ¼ y; with this defiitio, f0; 1g becomes a partially ordered set (or a poset). If x i is the ith compoet of the biary vector x, it ca be easily see that we have x<yif ad oly if x j <y j for some idex j ad x i y i for every i 6¼ j. Alterative orderigs ca be cosidered, such as the well-kow lexicographic orderig, which sets x<yif ad oly if there exists a idex j such that x j <y j ad x i ¼ y i for every i ¼ 1;...;j 1. The followig defiitios will be used heceforth: Defiitio 1. Let ðx; Þ be a poset. The, a atichai is a subset A X such that for ay x; y 2 A with x 6¼ y, either x<yor y<xholds. Defiitio 2. The lower shadow of a 2 X is the poset ðlðaþ; Þ, beig LðaÞ ¼fx 2 X : x ag. Likewise, the upper shadow of a 2 X is the poset ðuðaþ; Þ, where UðaÞ ¼fx 2 X : x ag. It directly follows from Defiitio 2 that b 2 LðaÞ if ad oly if a 2 UðbÞ. The diagram of the poset f0; 1g ca be obtaied by drawig a ode for each of its elemets, placig x lower tha y wheever x<y. The, a edge is added to coect each pair x; y 2f0; 1g for which x<y ad o a 2f0; 1g exists such that x<a<y. Whe the stadard orderig is used, the diagram of f0; 1g 3 is show i Fig. 1a. A example of the lower ad upper shadows for a ¼ 101 is the represeted i Figs. 1b ad 1c, respectively.

MUSELLI AND FERRARI: COUPLING LOGICAL ANALYSIS OF DATA AND SHADOW CLUSTERING FOR PARTIALLY DEFINED POSITIVE... 39 Now cosider the class B of Boolea fuctios f : f0; 1g!f0; 1g. Each of them iduces a separatio of f0; 1g ito two subsets D 0 ðfþ ad D 1 ðfþ, called offset ad oset, respectively: D 0 ðfþ ¼fx 2f0; 1g : fðxþ ¼0g; D 1 ðfþ ¼fx 2f0; 1g : fðxþ ¼1g: A fuctio of B ca always be writte as a Boolea expressio, ivolvig literals, i.e., variables x i or their complemet x i, costat terms 0, 1, ad the operators AND, OR, NOT. Deote by I the set f1; 2;...;g of the first positive itegers; if J ad K are two subsets of I,a logical product tðxþ ¼ V j2j x V j k2k x k is called term.itcovers a poit x 2f0; 1g if tðxþ ¼1; the degree of the term t is give by the umber jjjþjkj of literals i it, havig deoted by jjj the cardiality of the set J. If f ad g are two Boolea fuctios i B, we write f g if ad oly if fðxþ gðxþ for every x 2f0; 1g.Aimplicat of f is a term t f; if this relatio is o more true whe ay literal is removed from t, the term t is called a prime implicat (or a miterm) off. It is kow that every Boolea fuctio ca be writte i DNF, i.e., as a logical sum of implicats! fðx 1 ;...;x Þ¼ _h ^ ^ ; ð1þ i¼1 x j x k j2j i k2k i where J i \ K i ¼;for i ¼ 1;...;h, havig set by defiitio V j2j x j ¼ 1 ad W j2j x j ¼ 0 if J ¼;. I geeral, the DNF of a Boolea fuctio f is ot uique, eve if oly prime implicats are icluded i it. If the complemet operator caot be employed i Boolea expressios, oly the class L of (mootoe) positive Boolea fuctios ca be realized. If f 2L, we have that x y implies fðxþ fðyþ, for every x; y 2f0; 1g. Cosider the mappig P : f0; 1g! 2 I defied as P ðaþ ¼fi 2 I : a i ¼ 1g; it produces the subset of idices of the compoets a i assumig value 1. The iverse of P will be deoted by p; for ay subset J I, it gives the elemet pðjþ 2f0; 1g whose ith compoet p i ðjþ has value 1 if ad oly if i 2 J. I a similar way, we ca defie the complemetary mappig SðaÞ ¼I PðaÞ, which gives the subset of idices of the compoets a i assumig value 0, ad its iverse sðjþ ¼pðI JÞ. It ca be show that for every positive Boolea fuctio f, there exists a uique atichai A of the poset f0; 1g such that f ca be writte through the followig expressio: fðx 1 ;...;x Þ¼ _ ^ x j ; ð2þ a2a j2pðaþ which is called irredudat Positive Disjuctive Normal Form (PDNF) of the fuctio f. Every elemet a 2 A is called a miimum true poit of f ad is associated with the prime implicat V j2pðaþ x j, through the mappig P. Note that the oset of this prime implicat is give by the poits x a, which belog to the upper shadow UðaÞ. They ca be obtaied by chagig some of the compoets of a from 0 to 1. 3 PARTIALLY DEFINED POSITIVE BOOLEAN FUNCTION RECONSTRUCTION The problem of recostructig a Boolea fuctio f : f0; 1g!f0; 1g startig from its truth table (logic sythesis) has received great attetio i the sciece literature, sice it is the fudametal task i the desig of digital circuits. Several techiques have bee proposed to deal with this problem [1], [2], [3], [4], [24], [25]; i geeral, they aim to fid the prime implicats to be icluded i the DNF expressio for the ukow fuctio f. However, sice the DNF is ot uique, the choice of the prime implicats to be cosidered is performed accordig to some specific goal, such as the miimizatio of the complexity of the Boolea expressio. Possible measures of complexity are the umber of implicats or the total umber of literals i the DNF. This optimizatio process ca require a excessive computatioal burde, at least whe the umber of iputs icreases; hece, suboptimal algorithms [3], [4] have bee proposed, which fid a good DNF expressio withi a reasoable executio time. Oe of the most used methods of this kid is ESPRESSO [3], curretly employed i the sythesis of custom digital circuits. The same approach ca be adopted whe oly a reduced portio of the truth table for the ukow Boolea fuctio f is available. I this case, we speak of pdbf ad some of the above-cited optimal algorithms, e.g., spectral techiques [25], ca still be used oly if a prelimiary extesio of the available iformatio to the whole iput space is performed. However, it should be observed that the miimizatio of the DNF expressio heavily depeds o the method adopted for the prelimiary extesio. A good compromise betwee optimal ad suboptimal algorithms is offered by the LAD approach [7]: through a exhaustive search, it geerates all the prime implicats havig degree ot greater tha a maximum d. The, LAD adopts a simple heuristic to complete the collectio of miterms for the recostructio of the desired DNF. By properly choosig the value of d, a good trade-off betwee solutio optimality ad computatioal cost ca be achieved. Nevertheless, whe the dimesio of the iput domai is large ( >100 or >1000), eve a small d ca icrease excessively the executio time ad the memory occupatio required by LAD. To overcome this problem, the approach offered by HC [9] ca be employed. It has o exhaustive part, but the prime implicats are geerated oe at a time, clusterig poits aroud the kow examples of the truth table through the adoptio of clever heuristics. Here, a ew techique for pdbf sythesis is proposed; it is obtaied by combiig the approaches of LAD ad HC. I particular, the versio devoted to the recostructio of pdpbfs is described. It is called LSC, sice SC is the algorithm similar to HC for the sythesis of positive Boolea fuctios. The extesio to pdbf recostructio ca be readily obtaied, although it is show that a simple iput mappig allows LSC to be used eve for the geeral Boolea fuctio sythesis without ay loss i computatioal performace. The pdpbf recostructio problem ca be formalized as follows: pdpbf recostructio problem. Give two subsets T;F of poits i f0; 1g, with T \ F ¼;, we wat to fid (if it

40 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 exists) the positive Boolea fuctio f that satisfies T D 1 ðfþ ad F D 0 ðfþ ad miimizes some selected complexity measure. The followig otios of coverig ad separatio will help to clarify this goal: Defiitio 3. Let A; B be two subsets of f0; 1g. A covers B iff, for every b 2 B, there is a a 2 A such that a b. A is lower (resp., upper) separated from B iff, for every x 2 A, there is o y 2 B such that x y (resp., x y). A ad B are separated from each other iff A is both lower ad upper separated from B. Accordig to these defiitios, the goal of a algorithm for pdpbf recostructio is to fid the atichai A of the poset f0; 1g, which miimizes a selected complexity measure while covers T ad is separated from F. Startig from A, the irredudat PDNF of the desired Boolea fuctio f is readily writte by usig (2). Note that if there are x 2 T ad y 2 F such that x y, a positive Boolea fuctio f satisfyig T D 1 ðfþ ad F D 0 ðfþ caot exist. Thus, the prerequisite for pdpbf sythesis is that T must be lower separated from F. However, i a real-world problem, the presece of oise ca lead to a violatio of this requiremet. I such cases, the approaches described i this paper ca still be applied by acceptig a small amout of error whe costructig prime implicats to be icluded i the atichai A. The complexity of a PDNF is usually evaluated by takig ito P accout the umber jaj of miterms ad the umber a2a jp ðaþj of literals icluded i it. These measures are importat also whe a patter recogitio problem is to be solved, due to the Occam s razor priciple, which asserts that a simpler fuctio usually presets a better geeralizatio ability. However, aother importat quality measure i machie learig is give by the umber of poits i T covered by each prime implicat; i fact, each miterm ca be viewed as a itelligible rule, whose relevace depeds o how may cases it explais. Whe T [ F ¼f0; 1g, the positive Boolea fuctio f to be retrieved is completely defied ad the uiqueess of the irredudat PDNF esures that there is oly oe solutio to the problem at had. This solutio ca be foud by determiig all the miimum true poits of f to be icluded i the atichai A of expressio (2). I the opposite case, whe T [ F is a proper subset of f0; 1g ad T is lower separated from F, we ca solve the pdpbf recostructio problem by searchig for promisig biary strigs of f0; 1g, which will become the miimum true poits of the resultig positive Boolea fuctio f. The followig defiitio provides the characterizatio of these elemets: Defiitio 4. A biary strig a 2f0; 1g will be called a bottom poit for the pair ðt;fþ iff the followig three coditios hold: 1. at least a poit of T is i the upper shadow of a, 2. a is ot i the lower shadow of ay poit i F, ad 3. for every b<a, there is y 2 F such that b y. With a slight abuse of otatio, the expressio degree of a bottom poit a will idicate the degree of the term V j2p ðaþ x j, which is equal to the cardiality of P ðaþ. Fig. 2. Geeral procedure followed by LAD for recostructig positive Boolea fuctios. It ca be easily see that the collectio of the bottom poits for the pair ðt;fþ forms a atichai of the poset f0; 1g. Hece, it follows that the prime implicats of f could be obtaied by examiig all the bottom poits a for the pair ðt;fþ. However, if >20, a thorough aalysis ca require a excessive computatioal cost ad approximatios are eeded to produce a satisfactory solutio for the problem at had. 4 THE LAD APPROACH A good compromise betwee solutio optimality ad computatioal cost is offered by the LAD approach, whose geeral procedure is show i Fig. 2. At the ed of Step 1, the fuctio Exhaustive returs the set A icludig all the bottom poits with degree ot greater tha a chose maximum d. The, the collectio S of patters i T, ot covered by ay prime implicat i A, is cosidered at Step 2. Startig from each x 2 S (Step 4), a ew bottom poit a, lyig i the lower shadow of x, is geerated by followig a proper heuristic, performed by the fuctio Heuristic at Step 5. The poits covered by the ew prime implicat a are subsequetly removed from S (Step 7) ad the procedure Heuristic is repeated util S becomes empty. Note that it is always possible to fid a prime implicat a lyig i the lower shadow of ay x 2 T provided that T is lower separated from F. This geeral scheme allows us to select differet implemetatios for the exhaustive ad the heuristic parts. I particular, the origial LAD procedure proposes to adopt for the fuctio Exhaustive a breadth-first eumerative techique, which ca be easily modified for the case of pdpbf recostructio, as show i Fig. 3. The kerel of the fuctio is the loop at Step 2 that aalyzes durig each iteratio all the bottom poits havig degree l. It employs two sets of patters B ad C: they iclude all the biary strigs, which cover some poits both i T ad F ad have value 1 i l 1 ad l bits, respectively. Cosequetly, the set C for the curret iteratio will become the set B for the ext oe, as performed at Step 2c. The bottom poits foud durig each iteratio are subsequetly added to the set A (Step 2bbcaa).

MUSELLI AND FERRARI: COUPLING LOGICAL ANALYSIS OF DATA AND SHADOW CLUSTERING FOR PARTIALLY DEFINED POSITIVE... 41 Fig. 4. The heuristic procedure followed by LAD for geeratig a prime implicat coverig a give poit x 2 T i the recostructio of a pdpbf. Whe the exhaustive part is completed, the origial implemetatio of LAD uses a simple heuristic procedure to retrieve the remaiig bottom poits to be iserted i A (if eeded). Agai, it ca be directly modified to deal with pdpbf recostructio as reported i Fig. 4. The bits of x havig value 1 are chaged oe at a time (Step 2a), verifyig that the resultig patter does ot cover ay poit of F. The set J cotais the idices of the compoets that caot be switched. It should be oted that the fial bottom poit pðjþ depeds o the order followed whe selectig the bits of x to be chaged. A radom choice is ormally performed, sice the exhaustive part is supposed to have foud all the relevat prime implicats. However, this is ot true whe the dimesio of the iput domai is high, so that oly small values of d ca be cosidered without icreasig excessively the computatioal burde. Fig. 3. The breadth-first eumerative techique followed by LAD for geeratig the cosistet prime implicats for a pdpbf. Iitially, B cotais oly the patter pð;þ havig all the bits with value 0 (Step 1). The, durig each iteratio, the biary strigs of B are examied, ad for each of them, the last ru of 0s is cosidered (Step 2ba). The bits of this ru, whose idex lies betwee k þ 1 ad, are set to 1 oe at a time, thus, obtaiig ew patters havig l bits with value 1 (Step 2bbb). However, if l>1, each of these patters ca be a valid bottom poit oly if the set B icludes all the biary strigs obtaied by it after chagig a bit from 1 to 0. This is verified through the loop at Steps 2bbaa-2bbac. The if-the statemets at Steps 2bbc ad 2bbca have the aim of checkig if each ew valid patter covers poits i T ad F, respectively. Accordig to the results of these cotrol statemets, the patter is iserted ito A (if it covers some poits i T ad o elemets of F ), ito C (if it covers poits both i T ad F), or discarded (if it does ot cover ay poit of T). The loop at Step 2bb esures that each patter with l bits to 1 is cosidered oly oce. As a matter of fact, sice all the biary strigs i B are differet amog them, the additio of value 1 i the last ru of 0s caot give rise to the same patter. 5 THE PROPOSED LSC TECHNIQUE As previously oted, the methods adopted by the origial versio of LAD for implemetig the fuctios Exhaustive ad Heuristic preset some drawbacks that ca make the performace of the approach worse, especially whe the legth of the biary strigs is high. I particular, the breadth-first techique adopted i Fig. 3 may lead to a excessive icrease i memory occupatio, sice the umber of patters havig l bits with value 1 is Oð l Þ. To recover this problem, the proposed LSC techique carries out the exhaustive part of Fig. 2 (Step 1) by adoptig a depth-first approach, which sigificatly reduces the memory request. The, the SC procedure is employed i the heuristic part of LSC to retrieve the ear-optimal prime implicats startig from a patter x 2 T ot covered by the bottom poits icluded i the curret atichai A. 5.1 LSC: Exhaustive Part The exhaustive part of LSC simply cosists i a call to the procedure Depth show i Fig. 5 with parameters I ¼ I, J ¼;, ad A ¼;. All the parameters i the argumet to the call are passed by referece (or ca be cosidered as global variables). I particular, the two sets I ad J cotai the idices of the bits with value 0 i the curretly examied patter sði [ JÞ. By costructio (Step 2a), the elemets of J are always greater tha those of I; it should be oted that Steps 1 ad 3

42 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 Fig. 5. The recursive depth-first eumerative techique followed by LSC for geeratig the cosistet prime implicats for a pdpbf. allow us to preserve the cosistecy of these two sets after a call to Depth. The sigle loop at Step 2 forms the mai body of the procedure: the greatest elemet of I is removed from it (i.e., the correspodig bit is set), checkig if the resultig biary strig sði [ JÞ covers some poits of T. The, if it is also ot cotaied i the lower shadow of ay y 2 F ad is actually a bottom poit for the pair ðt;fþ (i.e., a elemet a 2 A with a sði [ JÞ caot be foud), the patter sði [ JÞ is added to the atichai A (Step 2baa). I the opposite case, further bits are set through a ew call to the procedure Depth (Step 2baA) provided that the umber of 0s i the examied patter does ot fall below the required miimum d. For a better uderstadig of the behavior of the procedure Depth, let us aalyze the followig pdpbf recostructio problem: Example 1. Cosider the sets 00111; 01101; 01110; 10011 T ¼ ; 11000; 11011; 11100; 11111 00001; 00010; 00101; 01001 F ¼ 01011; 10010; 10100; 10101 cotaiig biary strigs with ¼ 5 bits, obtaied through a radom samplig of the positive Boolea fuctio ; fðx 1 ;...;x 5 Þ¼x 1 x 2 þ x 1 x 4 x 5 þ x 3 x 4 þ x 2 x 3 x 5 ; ad suppose to perform the exhaustive part of LSC by callig the procedure Depth with A ¼;, I ¼f1; 2; 3; 4; 5g, J ¼;, ad d ¼ 2. ð3þ ð4þ After havig set k ¼ 6 at Step 1, the first executio of Step 2a removes the idex 5 from I, thus, producig the patter sði [ JÞ ¼00001. It ca be easily see that this biary strig lies i the lower shadow of 00111 2 T ad 00001 2 F; therefore, sice I is ot empty ad ji [ Jj ¼4 > 3 ¼ d, a ew call to Depth is doe at Step 2baA. Agai the local variable k is set to 6 sice J ¼;; the, at Step 2a, the elemet i ¼ 4 is deleted from I. It follows that sði [ JÞ ¼ 00011, which covers both 00111 2 T ad 01011 2 F ; cosequetly, it caot be a prime implicat. Now, sice ji [ Jj ¼ d ¼ 3, a ew call to Depth is ot permitted ad Step 2c is executed, puttig i ¼ 4 ito J. A ew iteratio of the loop starts by removig the idex 3 from I, leadig to the curret patter sði [ JÞ ¼00101. Sice it lies i the lower shadow of 00111 2 T ad 00101 2 F, it caot be icluded i A. The subsequet iteratios of the loop examie the biary strigs 01001 ad 10001, cocludig that either of them ca be implicats sice they cover poits i F. The executio of the secod call to Depth is thus fiished ad the cotrol returs to Step 2c of the callig procedure, where the idex i ¼ 5 is added as the first elemet to the set J. The, a ew iteratio is performed by removig the value 4 from I; the patter sði [ JÞ ¼00010 is determied i this way ad a ew call to Depth is doe at Step 2baA with I ¼f1; 2; 3g ad J ¼f5g. After havig set k ¼ 5 at Step 1, the mai loop begis by deletig the idex i ¼ 3 from I. The resultig biary strig sði [ JÞ ¼00110 covers oly poits i T, ad therefore, ca be icluded i the set A. The executio cotiues alog the same lies ad returs at the ed of the first call to Depth the atichai A ¼f00110; 01100; 11000g. Note that oly the biary strig 10011 2 T is ot covered by prime implicats of A. 5.2 LSC: Heuristic Part The simple heuristic method adopted by LAD ad show i Fig. 4 is essetially a greedy techique that starts from a specific poit x 2 T ad subsequetly switches oe at a time some of its bits from 1 to 0, while esurig that the lower shadow of ay elemet of F is ot reached. However, the way adopted to select the idex i to be removed from I at Step 2a is crucial for the quality of the bottom poits produced by this approach. The SC procedure offers a systematic way for performig the selectio of the idex i whe the goal is to miimize a desired quality factor measurig the complexity of the fial irredudat PDNF. I particular, the size of the atichai A ad the umber of literals P a2a P ðaþ i expressio (2) are two importat quatities that must be miimized to make the resultig positive Boolea fuctio simpler. To pursue this aim, SC adopts two possible criteria: the first oe, called Maximum coverig Shadow Clusterig (MSC), tries to icrease the umber of poits i S ad T covered by every geerated bottom poit; the secod oe, called Deepest Shadow Clusterig (DSC), maages to reduce the degree of the prime implicat produced. Other quality factors ca be miimized through SC by employig differet ad hoc criteria; a aalysis of some iterestig choices will be the subject of a forthcomig paper. It should be oted that the goal of MSC is very importat whe the algorithm for pdpbf recostructio is used i the solutio of

MUSELLI AND FERRARI: COUPLING LOGICAL ANALYSIS OF DATA AND SHADOW CLUSTERING FOR PARTIALLY DEFINED POSITIVE... 43 machie learig problems. I fact, ay bottom poit with high coverig represets a relevat rule iferred from data. The approach followed by MSC cosists i switchig from 1 to 0 the compoet x i of the selected poit x 2 S, which does ot lead to a coflict with the patters of F ad maximizes a measure of the potetial coverig associated with the idex i. This is give by the cardiality of the set S 0 i, havig defied for every Z f0; 1g, Z 0 i ¼fz 2 Z : z i ¼ 0g; Z 1 i ¼fz 2 Z : z i ¼ 1g: It ca be easily see that at every executio of Step 2a of the algorithm i Fig. 4, the poit pði [ JÞ does ot cover ay biary strig i Si 0 for i 2 I [ J. The, by switchig the ith bit associated with a high value of jsi 0 j, we icrease the probability of havig may elemets of S i the upper shadow of the ew patter reached by the algorithm. However, sice the size of S decreases durig the executio of SC, bottom poits that cover few poits of T ca be geerated whe jsj becomes small, due to the presece of may biary strigs whose upper shadows iclude the same umber of elemets i S. To perform a better choice, the selectio of the idex i 2 I ca also be drive by the size jti 0 j, which takes ito accout the potetial coverig of the whole set T. The alterative approach followed by the DSC criterio tries to miimize the degree of the resultig bottom poit by choosig the idices i 2 I that keep as large as possible the distace of the curret patter from the lower shadows of the poits i F. A proper measure of distace is offered by the followig defiitio: Defiitio 5. Let x; y be two poits of the poset f0; 1g. The lower distace d l ðx; yþ betwee x ad y is defied as where d l ðx; yþ ¼ X jzj þ ¼ i¼1 jx i y i j þ ; z; if z 0; 0; otherwise: I the same way, the upper distace d u ðx; yþ betwee x ad y is give by d u ðx; yþ ¼ X i¼1 jy i x i j þ : I few words, d l ðx; yþ is equal to the umber of idices i for which x i ¼ 1 ad y i ¼ 0, whereas d u ðx; yþ accouts for the idices i that give x i ¼ 0 ad y i ¼ 1. Note that d l ðx; yþ ¼0 if ad oly if x y, whereas d u ðx; yþ ¼0 if ad oly if x y.it should also be observed that either d l or d u provides a (pseudo)metric for f0; 1g, sice they are ot symmetric, but d l ðx; yþ ¼d u ðy; xþ. By employig the usual defiitio of distace betwee a poit x ad a set Z (give by the miimum distace betwee x ad the elemets of Z), we obtai that the lower distace d l ðx; ZÞ is defied as d l ðx; ZÞ ¼mi d lðx; yþ: y2z Similarly, d u ðx; ZÞ ¼mi y2z d u ðx; yþ. Fig. 6. The heuristic procedure followed by LSC for geeratig a prime implicat coverig a give poit x 2 T i the recostructio of a pdpbf. It follows that every bottom poit a for the pair ðt;fþ satisfies the coditio d l ða; F Þ¼1. Thus, the approach adopted by DSC for selectig the idex i 2 I cosists i maximizig the distace d l ðpði [ JÞ;Fi 0 Þ, beig pði [ JÞ the curretly cosidered patter. Sice the goals of MSC ad DSC are ot i coflict, both of them ca be adopted i the selectio of the idex i 2 I simply by imposig a order i the optimizatio of the differet measures. I this way, MSC maximizes the vector ðjsi 0j; jt i 0j;d lðpði [ JÞ;Fi 0 ÞÞ, where the first compoet jsi 0 j is cosidered more relevat tha the secod oe j, ad this, i tur, is more importat tha the distace d l ðpði [ JÞ;Fi 0 Þ. O the other had, the DSC criterio adopts the vector ðd l ðpði [ JÞ;Fi 0Þ; js0 0 i j; jti jþ with the jt 0 i associated preferece order. Fig. 6 shows i detail the algorithm adopted by MSC ad DSC for performig the heuristic part of the LSC techique. This algorithm ca be properly modified to take ito accout differet quality measures. At Step 2, the values of jsi 0j ad jt i 0 j for every i 2 I are computed; these values remai uchaged durig the executio of the procedure. The, at Step 3, the basic iteratio of the procedure takes place util the set I becomes empty. The distaces d l ðpði [ JÞ;Fi 0 Þ betwee the curret patter pði [ JÞ ad the sets Fi 0 are evaluated (Step 3a) ad the idices i 2 I with d l ðpði [ JÞ;Fi 0 Þ¼1 are moved to the set J (Step 3b), sice they caot be switched from 1 to 0 without creatig a coflict with the elemets of F. Now, if I becomes empty, the iteratio is iterrupted ad the fuctio returs pðjþ at Step 4. Otherwise, the idex i that maximizes the suitable vector is removed from I at Step 3c; this is equivalet to clear the correspodig bit i the curret patter pði [ JÞ. A example may help to illustrate the fuctioig of the algorithm ad the differece betwee MSC ad DSC. Example 2. Cosider the problem of recostructig the positive Boolea fuctio

44 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 from the sets fðx 1 ;...;x 6 Þ¼x 1 x 3 x 4 þ x 5 T ¼f000111; 100110; 101010; 101100; 110010g; F ¼f010101; 011100; 100101; 110100; 111000g; cotaiig biary strigs with ¼ 6 bits. Suppose that the exhaustive part of LSC has ot bee performed by settig d ¼ 0; it follows that A ¼; ad S ¼ T at Step 3 of the geeral procedure i Fig. 2. If at Step 4 the poit x ¼ 000111 is selected, the procedure Heuristic i Fig. 6 sets at Step 1 I ¼f4; 5; 6g ad J ¼;. The, at Step 2, we obtai S4 0 ¼ T 4 0 ¼f101010; 110010g; S5 0 ¼ T 5 0 ¼f101100g; S6 0 ¼ T 6 0 ¼f100110; 101010; 101100; 110010g; which gives js4 0j¼jT 4 0j¼2, js0 5 j¼jt 5 0 j¼1, ad js6 0j¼jT 6 0j¼4. Now, the mai loop starts by computig at Step 3a F 0 4 ¼f111000g; F0 5 ¼ F; F 0 6 ¼f011100; 110100; 111000g; which leads to the distaces d l ðpði [ JÞ;F4 0Þ¼3, d l ðpði [ JÞ;F5 0Þ¼1, ad d lðpði [ JÞ;F6 0Þ¼2. The executio of Steps 3b yields J ¼f5g ad I ¼f4; 6g; cosequetly, a specific criterio is to be employed at Step 3c to select the idex i to be removed from I. It is straightforward to observe that the maximizatio of the vector ðjsi 0j; jti 0j;d lðpði [ JÞ;Fi 0 ÞÞ, performed by MSC, leads to the choice i ¼ 6, whereas the applicatio of ðd l ðpði [ JÞ;Fi 0Þ; js0 i j; jt i 0jÞ, followed by DSC, selects i ¼ 4. Now, i this simple example, both approaches produce the same bottom poit, sice i the ext iteratio, the biary strig pðjþ ¼000010 is geerated both by MSC ad DSC ad retured to the callig program. Nevertheless, i more complex situatios, the atichai A obtaied with the two methods may differ sigificatly. The set S is the simplified at Step 7 of Fig. 2 by removig from it the poits covered by the patter 000010 just iserted ito A. After this step, S cotais the sigle elemet 101100 ad a subsequet executio of the fuctio Heuristic shows that this biary strig caot be further simplified ad is to be added to A. The fial atichai is therefore A ¼f000010; 101100g, which leads to the irredudat PDNF (5). Itesive trials have show that MSC achieves a higher level of accuracy with respect to DSC without icreasig the computatioal burde, ad therefore, will be employed i the tests preseted i Sectio 7. 5.3 LSC: Atichai Simplificatio The set A, geerated accordig to the procedure described i the previous sectios, cotais cosistet bottom poits for the pdpbf recostructio problem at had. I fact, every patter i A covers some poits of T ad o elemets of F. However, there ca be several bottom poits i A that cover the same poit x of T ad, i geeral, differet subsets of A ca be foud, which cover the whole set T. Thus, the fial phase of the pdpbf sythesis cosists i selectig the subset of A that miimizes the desired quality ð5þ measure. I particular, if complexity miimizatio is oe of the goals, the subset A with the lowest size is to be chose. If c ¼jAj is the umber of biary strigs i the atichai A, the subset A ca be retrieved by solvig the followig set coverig problem [7]: X c mi k k¼1 subj to Xc jk k 1 for j ¼ 1;...; jtj ð6þ k¼1 k 2f0; 1g; where k ¼ 1 if the kth elemet of A is icluded i the fial subset A ad 0 otherwise, whereas jk ¼ 1 if the jth patter of T is covered by the kth biary strig i A ad 0 otherwise. With this defiitio, the costrait P c k¼1 jk k 1 esures that every poit i T is covered by at least oe patter of A. By actig o the objective fuctio of problem (6) or o its costraits, the optimizatio of other quality measures, such as the degree of the bottom poits associated with the elemets of A or their average coverig, ca be pursued. For istace, the cost fuctio ca become P c k¼1 k k, beig k a positive coefficiet that measures the quality of the kth patter. The greater is k, the poorer is the correspodig kth bottom poit accordig to the selected quality measure. Sice the set coverig problem (6) is kow to be NPcomplete, approximated approaches are employed to fid a good solutio i a reasoable time. Oe of these methods is the greedy techique that subsequetly adds patters oe by oe util all the poits i T are covered. At each additio, it selects the patter that covers the highest umber of poits i T ot yet covered by the biary strigs already preset i A. This procedure is employed i all the tests preseted i Sectio 7. 6 COMPUTATIONAL ISSUES I this sectio, a theoretical aalysis of the computatioal cost of the algorithms LAD ad LSC is performed. I particular, the asymptotic behavior of the executio time ad memory occupatio is derived by cosiderig a worstcase approach. The characteristic variables of this aalysis are: the legth of the biary strigs, the size m of the portio T [ F of the truth table at had (m ¼jT [ F j), ad the maximum degree d of terms retrieved i the exhaustive part of the algorithm. Sice the computatioal cost of LAD ad LSC is essetially due to the compariso betwee biary strigs, a uitary amout of executio time is covetioally assiged to the compariso betwee two bits. 6.1 Executio Time Cosider the geeral procedure i Fig. 2; it ca be observed that the time required to extract the patter x (Steps 2-4) ad to remove from S the elemets z covered by the curret prime implicat a (Step 7) ca be eglected with respect to the executio time of Steps 1 ad 5. Cosequetly, the computatioal cost of LAD (resp., LSC) is essetially give by the two procedures, Exhaustive (resp., Depth) ad Heuristic. 6.1.1 LAD The worst-case aalysis for the exhaustive part of LAD leads to the followig result:

MUSELLI AND FERRARI: COUPLING LOGICAL ANALYSIS OF DATA AND SHADOW CLUSTERING FOR PARTIALLY DEFINED POSITIVE... 45 Theorem 1. If 1 d<=2, the executio time of the procedure Exhaustive i LAD is at most Oð 2d þ m dþ1 Þ. Proof. The computatioally relevat part of the algorithm i Fig. 3 cosists i the repeated executio of Steps 2bba ad 2bbc. The former is executed oly if l 2: it cosiders a ew cadidate bottom poit pðp ðaþ[figþ ad checks if the set B icludes all the l 1 biary strigs (differet from a) obtaied by pðp ðaþ[figþ after chagig i a a bit from 1 to 0. Sice B cotais oly terms of degree l 1, its cardiality jbj is at most, l 1 ad cosequetly, the executio of Step 2bba requires ðl 1Þð l 1Þ elemetary operatios. O the other had, Step 2bbc checks all the patters i T ad F: i the worst case, such aalysis eeds ðjtjþ jf jþ ¼ m comparisos. Now ote that both Steps 2bba ad 2bbc are repeated k times (Step 2bb), where k is the highest iteger i P ðaþ, for each possible elemet a 2 B (Step 2b). If l ¼ 1, oly the value k ¼ 0 is cosidered, which leads to executios of the Step 2bbc. If l 2, the maximum umber of strigs i B characterized by a certai k is equal to the umber ð k 1 l 2Þ of possible choices of l 2 idices from a total set of cardiality k 1. Hece, a upper boud for the executio time t 1 of the procedure Exhaustive ca be writte as t 1 m 2 þ Xd ðl 1Þ X 1 l¼2 k¼l 1 l 1 k 1 ð kþ l 2 þ m ; where the first summatio o the right-had side takes ito accout that the algorithm is repeated for each value of l ot greater tha the fixed maximum d. The case l ¼ 1 is ot icluded i the summatio sice it gives rise to the term m 2. By employig the idetities k k 1 ¼ h k ; h 1 h X k¼h k h ð7þ ¼ þ 1 ; ð8þ h þ 1 which hold for every itegers h; k;, we obtai X 1 k 1 ð kþ l 2 k¼l 1 ¼ X 2 k X 2 k ðk þ 1Þ l 2 l 2 k¼l 2 k¼l 2 ¼ 1 X 2 ðl 1Þ k þ 1 l 1 l 1 k¼l 2 ¼ l ðl 1Þ ¼ : l l l This result ca be substituted ito (7), thus, obtaiig t 1 Xd ðl 1Þ þ Xd l l 1 l¼2 ¼ 2 Xd 1 þ m Xd l 2 l l¼2 l¼1 l¼1 m l ; l where the first idetity of (8) has bee used agai. Now a upper boud for the first term o the righthad side ca be foud by observig that 1 ðl 1Þð l þ 1Þ 2 2: ¼ l 2 l l l 1 l 1 I additio, sice d<=2, we have for every l d d l d d ; ð9þ from which it follows for d 2 t 1 2 Xd l¼2 2 X d þ m l 1 l l¼1 2 ðd 1Þ 2d 2 d þ md 2 ðd 1Þ d 2d þ m dþ1 : This iequality for t 1 is also valid for d ¼ 1, sice i this case, we have from (7) t 1 m 2. tu A simpler aalysis allows us to retrieve a upper boud for the computatioal cost of the heuristic part of LAD. Theorem 2. The executio time of the heuristic part of LAD is at most Oðm 2 2 Þ. Proof. The heaviest task of the algorithm i Fig. 4 cocers the aalysis of the biary strigs i F, performed at Step 2b. Sice this check is repeated for each i 2 I (Step 2), a sigle executio of Heuristic requires at most jikfj m 2 elemetary operatios. Now, sice the procedure Heuristic is called for each patter x 2 T ot yet covered by the implicats i A, the heuristic part of LAD eeds a time t 2 m 2 jtj m 2 2 : From Theorems 1 ad 2, the behavior for the computatioal cost of LAD is readily derived. Corollary 1. If 1 d<=2, the executio time eeded for LAD is at most Oð 2d þ m dþ1 þ m 2 2 Þ. Hece, the computatioal burde of LAD is i the worst case a rapidly icreasig fuctio of, which ca lead to a excessive executio time eve for moderate values of ad d. 6.1.2 LSC Before retrievig similar upper bouds for the computatioal cost of LSC, let us prove a basic result about the completeess of its exhaustive part. Theorem 3. The exhaustive part of LSC aalyzes at most oce all the bottom poits for the pair ðt;fþ with degree l d. Proof. I the exhaustive part of LSC, bottom poits are icremetally built by recursively callig the procedure Depth show i Fig. 5. Iitially, the cadidate implicat sði Þ icludig oly 0s is cosidered ad each subsequet call to the procedure Depth chages a bit from 0 to 1 by removig a idex i the set I (Step 2a). Hece, geeratig a strig with l 1s eeds l recursive calls to Depth. Furthermore, it ca be easily see that every call to Depth with actual parameters I ad J aalyzes oly ut

46 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 poits belogig to the upper shadow of sði [ JÞ. Thus, if a implicat sði [ JÞ is added to the atichai A, o further calls to Depth are performed sice they could ot reach other bottom poits for the pair ðt;fþ. I every case, if the biary strig sði [ JÞ is aalyzed at Step 2b of Fig. 5, we ca directly fid the exact sequece of calls that produced it. It is sufficiet to observe that ay call to Depth acts oly o the first ru of 0s at the left (whose idices are icluded ito the set I), leavig uchaged the rest of the strig. I particular, Step 2a selects oe at a time the bits of this ru, startig from the right; i this way, the ru of 0s at the left is preserved. As a example, suppose that the strig 00010110 is aalyzed at Step 2b of Fig. 5. It ca be readily foud that the curret call of Depth is the third oe i the recursio order (equal to the umber of 1s i the strig). At the preset step, we have I ¼f1; 2; 3g (first ru of 0s) ad J ¼f5; 8g (remaiig 0s); we ca also ifer the cotet of I ad J for the caller ad for the first istace of Depth. They are I ¼f1; 2; 3; 4; 5g, J ¼f8gad I ¼f1; 2; 3; 4; 5; 6g, J ¼f8g, respectively. Sice every call of Depth examies oly the bits of the first ru of 0s, it is ot possible to obtai twice the same biary strig, ad cosequetly, every bottom poit is cosidered oly oce. O the other had, by costructio, all the bits of the ru are aalyzed through the loop at Step 2 of Fig. 5. Hece, all the bottom poits with degree l d are retrieved. tu Theorem 3 allows oe to readily obtai the computatioal cost of the exhaustive part of LSC. Corollary 2. If 1 d<=2, the executio time eeded for the exhaustive part of LSC is at most Oð 2dþ1 þ m dþ1 Þ. Proof. Sice the fial atichai A ca iclude at most d bottom poits, the executio of Steps 2b ad 2ba requires o more tha m þ ð dþ elemetary operatios. Hece, sice P the umber of biary strigs examied is at most d l¼1 ð l Þ, the total time t 3 eeded by the recursive executio of Depth verifies the iequality t 3 Xd m þ l d l¼1 d 2 þ md 2dþ1 þ m dþ1 ; d d havig used agai the iequalities (9). tu Fially, the computatioal cost of the heuristic part of LSC is provided by the followig result: Theorem 4. The executio time eeded for the heuristic part of LSC is at most Oðm 2 2 Þ. Proof. The algorithm i Fig. 6 subsequetly removes bits from the set I, util I is empty. The computatioally relevat part of the procedure cosists i Step 3a, which determies the lower distaces betwee the curret implicat pði [ JÞ ad some patters of F. This calculatio eeds at most jf j elemetary operatios ad is repeated for each i 2 I, with I I. O the other had, Step 2 requires jtj calculatios to retrieve the sets jsi 0 0 j ad jti j. Sice the procedure i Fig. 6 is repeated for each patter belogig to T, the executio time t 4 of the heuristic part of LSC is at most t 4 jtjðjtjþ 2 jf jþ m 2 þ m 2 2 2m 2 2 ; which proves the theorem. tu The global behavior of LSC directly follows from Theorems 3 ad 4. Corollary 3. If 1 d<=2, the computatioal cost of LSC is at most Oð 2dþ1 þ m dþ1 þ m 2 2 Þ. Note that, accordig to Corollaries 1 ad 3, i the worst case, the computatioal burde of LAD is slightly lower tha that of LSC. However, it should be oted that the executio time heavily depeds o the fuctio to be recostructed, ad i particular, o the possible bottom poits for the pair ðt;fþ. I fact, the umber of elemets i the atichai A to be checked at Step 2b of the procedure Depth caot be established a priori ad sigificatly affects the performace of LSC. This is clearly evidet i the simulatios, where LSC shows a lower computatioal burde with respect to LAD, eve if the former adopts a depth-first approach. 6.2 Memory Occupatio Whe calculatig the amout of memory eeded for LAD ad LSC, the storage required for T ad F must be take ito accout. It is OðmÞ ad does ot deped o the specific procedure adopted for sythesizig the positive Boolea fuctio. The, sice the umber of bottom poits for the pair ðt;fþ is supposed to be low (or at most comparable) with respect to m, the memory for the atichai A is eglected. Likewise, we do ot cosider the storage eeded for the idex sets I ad J or for the distaces d l ðpði [ JÞ;Fi 0Þ used i the heuristic part of LSC. 6.2.1 LAD Cocerig the exhaustive part of LAD, memory has to be reserved for the sets B ad C, which cotai promisig biary strigs of degree l 1 ad l, respectively. O the other had, othig computatioally relevat is stored durig the heuristic part of LAD. This leads to the followig result: Theorem 5. The amout of memory required for LAD is at most Oðm þ d Þ. Proof. Apart from the cotributio OðmÞ due to the storage of the sets T ad F, we must cosider for l<d the memory eeded for the auxiliary sets B ad C, which iclude at most þ 2 2 d l 1 l d 1 bits. 6.2.2 LSC Sice LSC does ot require ay sigificat additioal storage other tha that eeded for the sets T ad F,we ca readily obtai the followig behavior: Theorem 6. The memory required by LSC is at most OðmÞ. As expected, the computatioal demad of LAD, i terms of memory occupatio, is cosiderably higher tha that of LSC, sice the former adopts a breadth-first approach, which geerally sacrifices storage for speed. However, the results of this sectio show that the depth-first procedure adopted tu

MUSELLI AND FERRARI: COUPLING LOGICAL ANALYSIS OF DATA AND SHADOW CLUSTERING FOR PARTIALLY DEFINED POSITIVE... 47 by LSC dramatically reduces the memory demad while maitaiig almost uchaged the executio time. Simulatios reported i the ext sectio poit out this fact. 7 TESTS AND RESULTS To measure the performace achieved by LSC i recostructig a pdpbf, two differet experimets have bee carried out. The first oe cocers the ability of retrievig the optimal expressio of a positive Boolea fuctio f, whose PDNF is radomly geerated. I particular, we wat to measure the differece betwee the cosidered algorithms LAD ad LSC, i terms of complexity of the geerated solutio ad computatioal cost required to produce the irredudat PDNF. The secod experimet evaluates the possibility of usig LSC i the sythesis of geeral Boolea fuctios. I fact, every mappig g : f0; 1g!f0; 1g ca be realized through a positive Boolea fuctio f : f0; 1g 2!f0; 1g by applyig the trasformatio : f0; 1g!f0; 1g 2 defied as 2i 1 ðxþ ¼1 x i ; 2i ðxþ ¼x i ; ð10þ for i ¼ 1;...;, where j ðxþ is the jth bit of the biary strig ðxþ havig legth 2. Oce f has bee geerated, the associated Boolea fuctio g is simply obtaied by settig gðxþ ¼fððxÞÞ for every x 2f0; 1g, where is give by (10). I this way, every prime implicat for f ca be directly trasformed ito a miterm for g. Thus, LSC ca be used for recostructig a Boolea fuctio g from two sets of biary strigs T 0 ad F 0, with T 0 \ F 0 ¼;, where we wat to have gðxþ ¼1 for every x 2 T 0 ad gðxþ ¼0 for every x 2 F 0. The first actio to be performed is traslatig T 0 ad F 0, which iclude biary strigs with legth, ito two sets T ad F, cotaiig elemets with 2 bits, by applyig the mappig. It ca be show that the resultig set T is lower separated from F. Target Boolea fuctios are geerated by choosig, at radom, the miimum true poits to be icluded ito the atichai A of the irredudat PDNF (2). By adoptig proper costraits, the complexity of the resultig expressio is cotrolled so as to avoid too simple or too specialized fuctios. This kid of fuctios allows a fair evaluatio of the quality of the results produced by LSC ad LAD. I additio, they lead to recostructio problems similar to those arisig from real-world applicatios, particularly i the machie learig field. The performace of LSC i recostructig geeral Boolea fuctios is compared with those of the stadard LAD algorithm, as described i [7]. All the experimets have bee carried out o a persoal computer with a Petium 4 (CPU 2.80 GHz, RAM 480 MB) ruig uder the Widows XP operatig system. 7.1 pdpbf Recostructio To aalyze the performace of LAD ad LSC i recostructig a pdpbf, several trials have bee performed for differet values of the iput dimesio. I particular, to obtai a reliable value for the measured parameters, i.e., complexity ad executio time, 50 differet positive Boolea fuctios have bee radomly costructed for assumig iteger values i the set f5; 10; 20; 50; 100; 200; 500; 1000g. The truth table for each fuctio was geerated by buildig, at radom, miðb=2c; 10Þ logical products, which form the PDNF of the target positive Boolea fuctio f. Fig. 7. Quality of the pdpbf recostructed by LAD ad LSC: (a) umber of implicats ad (b) classificatio error. Each logical product cotais a umber of terms radomly selected i the rage ½3; mið; 10ÞŠ so as to refuse too specialized fuctios icludig AND operatios (with >10) that assume value 1 oly for a very small set of iput strigs. Note that the presece of logical products with oe or two terms would lead to too simple fuctios, easily recogized by every trivial method; therefore, we have imposed the costrait 3. The idices of the terms icluded i each logical product are radomly chose with uiform probability. A subset of r ¼ mið2 1 ; 1024Þ iput-output pairs is the radomly extracted without replacemet to form the sets T ad F, which are employed i the recostructio task. Experimets have bee carried out with the LAD ad LSC algorithms, chagig the value of the maximum degree d i their exhaustive part. For each method, the average complexity, i.e., the umber of implicats i the PDNF, the classificatio accuracy, i.e., the percetage of errors i recogizig ew samples, ad the computatioal burde, i.e., executio time ad memory occupatio, have bee measured. Some plots comparig the performace of the differet methods are reported i Figs. 7 ad 8. The odd values 3, 5, ad 7 have bee cosidered for the maximum degree d. The labels LAD-d (resp., LSC-d) i the legeds refer to the applicatio of algorithm LAD (resp., LSC) usig the value d i its exhaustive part. The direct applicatio of the SC procedure is also cosidered ad deoted by LSC-0 i the legeds.

48 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 TABLE 1 Maximum Number of Iputs i the pdpbf Recostructio, Which Ca Be Treated by LAD ad LSC for Differet Values of d Fig. 8. Computatioal cost of pdpbf recostructio by LAD ad LSC: (a) executio time ad (b) memory occupatio. As expected, the plot i Fig. 7b shows that the accuracy of the PDNF icreases whe higher values of the maximum degree d are used. O the other had, eve for low values of d, both LAD ad LSC require a excessive computatioal cost whe the umber of iputs icreases, as show i Fig. 8. I particular, some trials are ot reported sice their executio exceeds the available time (about oe day for each pdpbf recostructio) or the physical memory of the computer. Table 1 shows the maximum dimesio of the iput space for which a fuctio f : f0; 1g!f0; 1g ca be recostructed by a specific method. Whe >200, the applicatio of LAD is ot possible sice its exhaustive part requires the memorizatio of a huge amout of data also for d ¼ 3. I cotrast, the depthfirst approach followed by LSC permits a more geeral applicability; i this case, the costrait o the executio time is the most relevat. As a example, the pdpbf recostructio with LSC i the case ¼ 1000 ad d ¼ 3 requires few hours of CPU ad less tha 1 MB of memory. Plots i Figs. 7 ad 8 show that LSC with the choice d ¼ 3 i its exhaustive part is always advatageous, sice it improves complexity ad accuracy while keepig the computatioal cost uchaged with respect to the simple SC procedure (LSC-0). However, the complexity of the solutio (Fig. 7a) does ot grow idefiitely, but starts decreasig whe >200, sice the portio of the truth table icluded i the sets T ad F becomes almost egligible (less tha 10 60 of the total). Cosequetly, the umber of bottom poits for the pair ðt;fþ icreases accordigly, thus makig it more probable to fid prime implicats with low degree. Lookig at Figs. 7a ad 7b, it ca be oticed that the LSC approach obtais better values of complexity ad accuracy with respect to LAD. This is maily due to the adoptio of SC i the heuristic part, which employs a smart strategy for selectig the bits to be chaged for retrievig a good bottom poit coverig the patter at had. The Mootoe Dualizatio (MD) algorithm [24] has also bee tested o some problems with low iput dimesio. MD performs the recostructio of a positive Boolea fuctio f uder the assumptio that the value of fðxþ is kow for ay x 2f0; 1g. I particular, MD groups the biary strigs of f0; 1g to form totally ordered sequeces C i, called Hasel chais. Each C i is the aalyzed to determie the first elemet x of the chai havig correspodig output fðxþ ¼1. Sice all the subsequet poits of C i lie i the upper shadow of x, their aalysis is ot eeded. By examiig i this way all the Hasel chais, the collectio of miimum true poits for f is readily obtaied. Note that the geeratio of the Hasel chais requires a memory occupatio Oð2 Þ, sice all the biary strigs of legth must be stored simultaeously. For this reaso, the MD method caot be employed for problems with high iput dimesio ( >30). The performed tests show that the MD algorithm is able to retrieve the correct PDNF for a positive Boolea fuctio f by examiig oly about 10 percet of the total -dimesioal Boolea strigs. However, the executio time has the same order of magitude as that required by LAD ad LSC whe d ¼, i.e., whe the exhaustive part of the algorithms cosiders all the possible implicats. As a example, for ¼ 20, MD retrieves the correct PDNF withi 15 miutes, whereas LAD ad LSC eed 20 miutes to perform the same task. 7.2 pdbf Recostructio As i the previous case, the truth tables of 50 target Boolea fuctios have bee radomly geerated for each iput dimesio i the set f5; 10; 20; 50; 100; 200; 500; 1000g, by costructig the correspodig DNF expressios. Each DNF cotais mið; 10Þ logical products, havig a umber of operads chose agai, at radom, i the iterval ½3; mið; 10ÞŠ. As i the geeratio of positive Boolea fuctios, the idices of the terms icluded i each logical product are radomly selected with uiform probability. The quality of the recostructed fuctio, i terms of complexity ad accuracy, is show i Fig. 9, while the computatioal effort eeded for its geeratio is reported i Fig. 10. Note that i this set of trials, the stadard LAD

MUSELLI AND FERRARI: COUPLING LOGICAL ANALYSIS OF DATA AND SHADOW CLUSTERING FOR PARTIALLY DEFINED POSITIVE... 49 Fig. 9. Quality of the pdbf recostructed by LAD ad LSC: (a) umber of implicats ad (b) classificatio error. approach (described i [7]) has bee employed, sice it is expressly devoted to pdbf sythesis. Notwithstadig this choice, the results reported i Figs. 9 ad 10 reflect the same behavior poited out for pdpbf recostructio. I particular, the quality of fuctios built by LSC is sigificatly better tha that exhibited by LAD, both i terms of complexity ad accuracy. This superiority is ot achieved at the expese of a icrease i computatioal burde. I cotrast, the executio time ad the memory occupatio of LSC are agai lower tha those of LAD if the values of ad d are equal. It follows that the itroductio of the mappig (10) does ot make the performace of LSC worse, either i terms of quality of the recostructed Boolea fuctio or i terms of computatioal efficiecy. I particular, due to its excessive memory occupatio, agai LAD is ot able to treat problems with >200, whereas LSC achieves a good accuracy eve whe ¼ 1000. The executio time for LSC rages from few secods ( <100) to about oe day ( ¼ 1000). 8 CONCLUSIONS A ovel algorithm, called LSC, for the recostructio of a pdpbf has bee itroduced. It combies two existig techiques, LAD ad SC, to produce the irredudat PDNF of the resultig fuctio. I particular, LAD adopts a exhaustive breadth-first approach for examiig all the Fig. 10. Computatioal cost of pdbf recostructio by LAD ad LSC: (a) executio time ad (b) memory occupatio. prime implicats with degree ot greater tha a maximum d, whereas SC employs a smart heuristic to determie a good miterm (accordig to some measure of quality) coverig a give patter x. As with LAD, the LSC techique is composed by a exhaustive part, which follows a depth-first approach to save memory, ad by a heuristic part that adopts SC to complete the irredudat PDNF of the desired positive Boolea fuctio. Whereas [7] proves the correctess of the approach followed by LAD, the theoretical aalysis of Sectio 6 (Theorem 3) shows that LSC is able to solve ay cosistet pdpbf recostructio problem, sice the correctess of SC ca be directly established. I additio, LSC requires oly the memory eeded for storig the portio of the truth table at had. I cotrast, LAD must also maitai (i the worst case) all the biary strigs with degree l for l<d, which leads to a icrease Oð d Þ i the memory occupatio. A compariso betwee the performace of LSC ad LAD, i terms of complexity ad accuracy of the retrieved solutio, has bee carried out by performig extesive tests o radomly geerated Boolea fuctios. As expected from theory, the reduced memory request allows LSC to treat problems with higher iput dimesio. However, due to a efficiet housekeepig method, eve the executio time of LSC is sigificatly lower tha that of LAD. This decrease i computatioal cost is ot achieved at the expese of a worse fial solutio. I cotrast, positive Boolea fuctios geerated by LSC exhibit, i geeral, a

50 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 1, JANUARY 2011 lower complexity ad a higher accuracy tha those built with LAD. This improvemet is essetially due to the employmet of SC i the heuristic part of the procedure ad degrades whe d icreases. Note that i most applicatios, the high iput dimesio forces the choice of low values for d; i these cases, the adoptio of SC leads to resultig fuctios with better quality. However, the adoptio of a exhaustive part improves the accuracy of the solutio, as poited out by the compariso betwee LSC ad SC (deoted by LSC-0 i the plots of Sectio 7). The behavior of LSC is advatageous also whe dealig with pdbf recostructio; as a matter of fact, experimetal trials have show that a simple mappig allows LSC to treat successfully classic Boolea fuctio sythesis. The compariso with stadard LAD, as described i [7], poits out the superiority of LSC i terms of both complexity ad accuracy. I the presece of oisy data, the portio of the truth table to be employed for pdpbf recostructio may be o more cosistet, i.e., o positive Boolea fuctio satisfies all the give iput-output relatios. I this case, the LSC algorithm, as described i this paper, caot be directly applied. However, it is still possible to retrieve the irredudat PDNF of a fuctio that behaves well for most iput patters. It is sufficiet to relax the cotrol statemets at Step 2ba of Fig. 5 (exhaustive part) ad Step 3b of Fig. 6 (heuristic part) so as to accept that the resultig positive Boolea fuctio provides a wrog output i correspodece with a small amout of iput poits. The modificatio of LSC for permittig the treatmet of oisy ad missig data will be the subject of a forthcomig paper. Curretly, LSC is adopted for the geeratio of Switchig Neural Networks (SNNs) [12] solvig the real-world classificatio problems. I most applicatios, the devices produced by LSC show a better accuracy with respect to SNN traied by SC. The possibility of extractig a itelligible set of rules uderlyig the classificatio problem at had icreases the practical iterest of this approach. REFERENCES [1] H.W. Gschwid ad E.J. McCluskey, Desig of Digital Computers. Spriger-Verlag, 1975. [2] T. Dows ad M.F. Schultz, Logic Desig with Pascal. Va Nostrad Reihold, 1988. [3] R.K. Brayto, G.D. Hachtel, C.T. McMulle, ad A.L. Sagiovai- Vicetelli, Logic Miimizatio Algorithms for VLSI Sythesis. Kluwer Academic Publishers, 1984. [4] S.J. Hog, R.G. Cai, ad D.L. Ostapko, MINI: A Heuristic Approach for Logic Miimizatio, IBM J. Research ad Developmet, vol. 18, pp. 443-458, 1974. [5] H. Liu ad R. Setioo, Feature Selectio via Discretizatio, IEEE Tras. Kowledge ad Data Eg., vol. 9, o. 4, pp. 642-645, July 1997. [6] R. Kohavi ad M. Sahami, Error-Based ad Etropy-Based Discretizatio of Cotiuous Features, Proc. Secod It l Cof. Kowledge Discovery ad Data Miig, pp. 114-119, 1996. [7] E. Boros, P.L. Hammer, T. Ibaraki, A. Koga, E. Mayoraz, ad I. Muchik, A Implemetatio of Logical Aalysis of Data, IEEE Tras. Kowledge ad Data Eg., vol. 12, o. 2, pp. 292-306, Mar./ Apr. 2000. [8] M. Muselli, Approximatio Properties of Positive Boolea Fuctios, WIRN/NAIS 2005, Lecture Notes i Computer Sciece, B. Apolloi, M. Mariaro, G. Nicosia, ad R. Tagliaferri, eds., vol. 3931, pp. 18-22, Spriger-Verlag, 2006. [9] M. Muselli ad D. Liberati, Traiig Digital Circuits with Hammig Clusterig, IEEE Tras. Circuit ad Systems I: Fudametal Theory ad Applicatios, vol. 47, o. 4, pp. 513-527, Apr. 2000. [10] M. Muselli ad D. Liberati, Biary Rule Geeratio via Hammig Clusterig, IEEE Tras. Kowledge ad Data Eg., vol. 14, o. 6, pp. 1258-1268, Nov. 2002. [11] M. Muselli ad A. Quarati, Recostructig Positive Boolea Fuctios with Shadow Clusterig, Proc. 17th Europea Cof. Circuit Theory ad Desig (ECCTD 05), 2005. [12] M. Muselli, Switchig Neural Networks: A New Coectioist Model for Classificatio, WIRN/NAIS 2005, Lecture Notes i Computer Sciece, B. Apolloi, M. Mariaro, G. Nicosia, ad R. Tagliaferri, eds., vol. 3931, pp. 23-30, Spriger-Verlag, 2006. [13] J. Astola, D. Akopia, O. Vaiio, ad S. Agaia, New Digit-Serial Implemetatio of Stack Filters, Discrete Applied Math., vol. 61, pp. 181-197, Aug. 1997. [14] P.D. Wedt, E.J. Coyle, ad N.C. Gallagher, Stack Filters, IEEE Tras. Acoustic, Speech ad Sigal Processig, vol. 34, pp. 898-911, 1986. [15] E. Zermelo, Über eie Awedug der Megelehre auf die Theorie des Schachspiels, Proc. Fifth Cogress of Mathematicias, pp. 501-504, 1912. [16] N. Bshouty, Exact Learig Boolea Fuctios via the Mootoe Theory, Iformatio ad Computatio, vol. 123, pp. 146-153, 1995. [17] A. Blum, C. Burch, ad J. Lagford, O Learig Mootoe Boolea Fuctios, Proc. 39th A. Symp. Foudatios of Computer Sciece, pp. 408-415, 1998. [18] V.I. Torvik ad E. Triataphyllou, Miimizig the Average Query Complexity of Learig Mootoe Boolea Fuctios, INFORMS J. Computig, vol. 14, pp. 142-172, 2002. [19] V. Gurvich ad L. Khachiya, O Geeratig the Irredudat Cojuctive ad Disjuctive Normal Forms of Mootoe Boolea Fuctios, Discrete Applied Math., vols. 96/97, pp. 363-373, 1999. [20] E. Boros, P.L. Hammer, T. Ibaraki, ad K. Kawakami, Polyomial Time Recogitio of 2-Mootoic Positive Boolea Fuctios Give by a Oracle, SIAM J. Computig, vol. 26, pp. 93-109, 1997. [21] E. Boros, T. Ibaraki, ad K. Makio, Error-Free ad Best Fit Extesios of Partially Defied Boolea Fuctios, Iformatio ad Computatio, vol. 140, pp. 254-283, 1998. [22] I. Shmulevich, A. Korshuov, ad J. Astola, Almost All Mootoe Boolea Fuctios Are Polyomially Learable Usig Membership Queries, Iformatio Processig Letters, vol. 79, pp. 211-213, 2001. [23] M. Fredma ad L. Khachiya, O the Complexity of Dualizatio of Mootoe Disjuctive Normal Forms, J. Algorithms, vol. 21, pp. 618-628, 1996. [24] B. Kovalerchuk, E. Triataphyllou, A.S. Deshpade, ad E. Vityaev, Iteractive Learig of Mootoe Boolea Fuctios, Iformatio Scieces, vol. 94, pp. 87-118, 1996. [25] S. Agaia, J. Astola, ad K. Egiazaria, Biary Polyomial Trasforms ad Noliear Digital Filters. Marcel Dekker, 1995. Marco Muselli received the MSc degree i electroic egieerig from the Uiversity of Geoa, Italy, i 1985. He joied the Istitute of Electroic Circuits i 1988, ow part of the Istitute of Electroics, Computer ad Telecommuicatio Egieerig, Italia Natioal Research Coucil, where he is curretly a seior researcher. His research iterests iclude machie learig, bioiformatics, eural etworks, global optimizatio, mathematical statistics, ad probability theory. His research activity maily focuses o the developmet of ew efficiet rule geeratio methods ad their applicatios i several fields, especially i the solutio of biomedical problems. He is the author or coauthor of more tha 100 scietific publicatios, most of which o iteratioal jourals. He is a member of the IEEE. Erico Ferrari received the MSc ad PhD degrees i physics from the Uiversity of Geoa, Italy, i 2005 ad 2009, respectively. He is curretly a postdoctoral fellow i the Istitute of Electroics, Computer ad Telecommuicatio Egieerig, Italia Natioal Research Coucil. His research iterests are i the field of probability theory, statistics, ad machie learig. His research activity focuses o the developmet of ovel learig algorithms ad their applicatio i several real-world problems, especially i the aalysis of biomedical data.