Clusterng on antmatrods and convex geometres YULIA KEMPNER 1, ILYA MUCNIK 2 1 Department of Computer cence olon Academc Insttute of Technology 52 Golomb tr., P.O. Box 305, olon 58102 IRAEL 2 Department of Computer cence Rutgers Unversty, NJ, DIMAC 96 Frelnghuysen Road, Pscataway, NJ 08854-8018 U Abstract: - The clusterng problem as a problem of set functon optmzaton wth constrants s consdered. The behavor of quas-concave functons on antmatrods and on convex geometres s nvestgated. The dualty of these two set functon optmzatons s proved. The greedy type Chan algorthm, whch allows to fnd an optmal cluster, both as the most dstant group on antmatrods and as a dense cluster on convex geometres, s descrbed. Key-Words: - Quas-concave functon, antmatrod, convex geometry, cluster, greedy algorthm 1 Introducton In ths paper we consder the problem of clusterng on antmatrods and on convex geometres. There are two approaches to clusterng. In the frst one we try to fnd extreme dense clusters (ether as a partton of the consdered set of obects nto homogeneous groups, or as a sngle tght set of obects). The second approach fnds a cluster whch s the most dstant from ts complementary subset n the whole set (as n the sngle lnkage clusterng method) wthout a control how close are obects wthn a cluster. The problem of densty (n the frst approach) or solaton (n the second approach) of a subset may be formalzed n terms of a set functon - the functon that ts values score the subsets accordng to ther tghtness or to dstant relatons wth ther complementary subsets. Usually, n cluster analyss each subset may be referred as a cluster. owever, n last years the problem to fnd clusters as optmzaton problem wth constrants became actual, partcularly n scentfc database mnng, where obects have complex structure. In those cases ther smlarty measurement requres to consder the problem from mult-crtera perspectves. For nstance, f one consders transportaton problems n a large cty and wants to apply clusterng technque, she or he has to take nto account at least two crtera: a populaton flow densty along transportaton lnes and a capacty of all types of transports along the lnes. In ths paper we also consder restrcted stuatons for clusterng. We found that such stuatons can be characterzed as constructvely defned famles of subsets, such that only elements from these famles can be consdered as feasble clusters. For example, let us consder a boolean matrx of data n whch varables (columns of matrx) are dvded nto several groups. Varables, belongng to one group, are ordered. In ths case every group of varables can be nterpreted as a complex varable whch represent an ordnal scale for some property: value 1 of the frst (accordng to the order) boolean varable means that an obect represents the correspondng property on very general level, 1 for the second varable means that the obect represents the property on more specfc level, etc. Thus the value 1 of a boolean varable wth rank k n ts group means that all prevous varables of the group also have ths value 1. In other words, t s an ordnal data matrx. The problem s to fnd an extreme n ths ordnal data whch represents a group of ponts, that are smaller (accordng to the ordnal vector comparson) than the choosen extreme. It s obvously that ths case can be generalzed on a common ordnal data. If one nterprets ordnal scales as crtera for a partcular multcrtera evaluaton then he can nterpret the above cluster as the best choce soluton. Thus the cluster problem can be formulated as follows: for a gven set system over E,.e. for a par (E,) where 2 E s a famly of feasble subsets of fnte E, and for a gven functon F : R, fnd
elements of for whch the value of the functon F s maxmal. In general, ths optmzaton problem s NP-hard, but for some specfc functon and for some specfc set systems polynomal algorthms are known. The well-known examples are modular cost functons that can be optmzed over matrods by usng polynomal greedy algorthm [2] and bottleneck functon that can be maxmzed over greedods [5]. The other example s set functon defned as the mnmum value of monotone lnkage functons. uch set functon can be maxmzed by a greedy type algorthm over a famly of all subsets of E [4,7] and over antmatrods [3]. In ths paper we refne our results obtaned for antmatrods and extend them to convex geometres. In ecton 2 some basc nformaton about convex geometres and antmatrods are lsted. ecton 3 gves a bref ntroducton to the optmzaton of quas-concave functons and nvestgates the optmzaton problem on antmatrods. In ecton 4 the optmzaton problem s consdered on convex geometres and dualty of two problems are establshed. 2 Prelmnares Let E be a fnte set. A set system over E s a par (E,) where 2 E s a famly of feasble subsets of E, called feasble sets. We wll use X x for X {x} and X x for X {x}. Defnton 2.1 A closure operator, τ : 2 E 2 E, s a map satsfyng the closure axoms: C1: X τ(x) C2: X Y τ(x) τ(y) C3: τ(τ(x)) = τ(x) A set X s sad to be closed f τ(x) = X. The par (E,τ) s called a closure space. To obtan an equvalent defnton consder a set system (E,) such that () E () X,Y X Y Then the operator (1) τ(a) = {X : A X and X } s a closure operator. A set system ( E, ) satsfyng () and () wth closure operator τ defned n (1) s an equvalent axomatzaton of closure space [5]. Defnton 2.2 A closure space s called a convex geometry f the followng ant-exchange property s satsfed: f y,z τ(x) then z τ(x y) mples y τ(x z) We call an element x A an extreme pont of A f x τ(a x). For a convex set X,.e. for a set from a convex geometry (E,), ths s equvalent to X x. The set of extreme ponts of A s denoted ex(a). Consder another set system. Defnton 2.3 A nonempty set system (E,) s an antmatrod f (A1) for each nonempty X there s an x X such that X x (A2) for all X,Y, and X Y, there exsts an x X Y such that Y x Any set system satsfyng (A1) s called accessble. Another antmatrod defnton s based on the followng property: Defnton 2.4 A set system (E,) has the nterval property wthout upper bounds f for all X,Y wth X Y and for all x E Y, X x mples Y x. Theorem 2.1 [1,5] For an accessble set system (E,) the followng statements are equvalent: () (E,) s an antmatrod () s closed under unon () (E,) satsfes the nterval property wthout upper bounds A maxmal feasble subset of set X E s called a bass of X, and s denoted by B X. Clearly, (by ()), there s only one bass for each set. Now consder the Theorem that establshes dualty between two structures. Theorem 2.2 [5] A set system (E,) s an antmatrod f and only f (E,) s a convex geometry, where = { E X : X }. As an mmedate consequence we have (2) τ(e X) = E B X For a set X, let Γ(X) ={x E X : X x } be the set of feasble contnuatons of X, then (3) Γ(X) = ex(e
3 Quas-concave functons and antmatrods In ths secton we consder set functons defned as the mnmum value of monotone lnkage functons. uch set functons can be maxmzed by a greedy type algorthm over a famly of all subsets of E [10] and over antmatrods [3]. ere we detal the behavor of these functons defned on antmatrods n order to use them for clusterng. We consder the problem to fnd a cluster whch s most dstant from ts complementary subset n the whole consdered set. To defne a dstance between the subset and rest obects, the concept of an element-to-set lnkage functon can be used. A lnkage functon π(x,x) measures a dstance between subset X and element x. The well-known example s the sngle lnkage functon π ( x, = mn d y X whch s defned n terms of par-wse dstances d xy. As another example consder a data table A = (x k ) where x k s a value of varable k K for entty E. A lnkage functon can be defned as π (, = mn xk x k X k K An mportant feature of these examples s the monotoncty of lnkage functons. The monotone lnkage functons were ntroduced by Mullat [9]. Defnton 3.1 A functon π : E 2 E R s a monotone lnkage functon f for all X,Y E and x E (4) X Y mples π(x,x) π(x,y) Consder F : 2 E R defned for each X E F( = mn π ( x, x E X Ths functon may be consdered as the functon that measures solaton of the subset X,.e. t measures how ths subset s dstant from rest obects. ence a subset maxmzng ths set functon can be referred as a cluster. It was shown [6], that the functon F satsfes the next condton: for each X,Y E, F(X Y) mn {F(X), F(Y)} uch functons are called quas-concave set functons. These functons were studed n [10,4]. In partcular, the smple polynomal algorthm whch fnds the mnmal set X E such that F(X) = max{f(y) : Y E} was developed and used for ncomplete clusterng constructor [10]. In ths secton we extend our results to antmatrods. For ths purpose we defne a new functon specfed for antmatrods: xy F ( = mn π ( x, ( and f Γ(X) =, then F (X) = -. It should be mentoned, that an antmatrod (E,) has one and only one maxmal feasble set, E = X X. Thus, for each feasble set X - E the contnuaton Γ(X) s not-empty,.e., F s well defned on - E for any antmatrod (E,). Addtonal notaton s that the feasble sets of an antmatrod (E,), ordered by ncluson, form a lattce L, wth lattce operatons: X Y = X Y, X Y = B X Y Theorem 3.1. If a set system (E,) s an antmatrod, then the functon F s a quas-concave functon on L,.e., for each X,Y, F (X Y) mn {F (X), F (Y)} Proof. The case X Y = E s trval, then assume that X Y - E,.e., F ( X Y) = mn π ( x, X Y ). ( X Y ) Let F (X Y) = π(x 0, X Y), where x 0 Γ(X Y). ence, x 0 E (X Y), snce Γ (B E X. Indeed, suppose that x Γ (B and x X, then B X x X and B X x. Thus B X x s also a bass, a contradcton. Assume, wthout loss of generalty, that x 0 E X. ence, we have x 0 E X, and X Y X, and x 0 Γ (X Y ), so from the nterval property wthout upper bounds follows that x 0 Γ (. Then 0 0 F ( X Y) = π ( x, X Y) π ( x, mn π ( x, = F ( mn{ F (, F ( Y )} ( Consder the followng optmzaton problem - gven a monotone lnkage functon π and a set system (E,), fnd the feasble set X such that F (X) = max{f (Y) : Y }. From Theorem 3.1 t follows that the set of the optmzaton problem solutons s a meet-semlattce wth unque mnmal element. To fnd ths mnmal element we use the followng algorthm [3]: The Chan Algorthm ( E,, π ) 1. X := 2. X 0 := 3. Whle Γ (X) do 3.1 f F (X) > F (X 0 ), X 0 :=X 3.2 choose (X) such that π (x, π (y, for all y Γ (X) 3.3 X := X x 4. Return X 0
Thus, the algorthm generates a chan of sets = X 0 X 1 X k, where X = X -1 x and x Γ (X -1 ), and returns the mnmal set X 0 of the chan on whch the value F (X 0 ) s maxmal. Theorem 3.2 Let a set system (E,) be an antmatrod, then for all monotone lnkage functon π the Chan Algorthm fnds a mnmal feasble set that maxmzes the functon F. Proof. Let X 0 be a set obtaned by the Chan Algorthm. To prove that X 0 s a mnmal feasble set that maxmzes F, we have to prove two statements: () F ( < F ( X 0 ) for each X and X 0 X Let = X 0 X 1 X k be the chan generated by the Chan Algorthm. Let be the least nteger for whch X X. Then X -1 X, x X and X -1 x, that mples (from the nterval property wthout upper bounds) x Γ (. ence (5) F ( π(x, π(x, X -1 ) = F ( X -1 ) nce X 0 X we have X -1 X 0, so F (X -1 )<F ( X 0 ) () F ( F ( X 0 ) for each X and X 0 X If X = E or X = X k the statement s obvously, otherwse, by analogy wth prevous case, we obtan (5), but now F (X -1 ) F (X 0 ), because t s possble that X 0 X -1. The Chan Algorthm s a greedy type algorthm snce t based on the best choce prncple: t chooses on each step the extreme element (n sense of lnkage functon) and thus approaches the optmal soluton. Let P s the maxmum complexty of π computaton, then the Chan Algorthm fnds the mnmal set that maxmzes the functon F n O(P E 2 ) tme. Notce, that a quas-concave functon determnes on an antmatrod (E,) the specal structure. It has been already noted that the famly of feasble sets maxmzng the functon F s a meet-semlattce wth unque mnmal element. Denote ths famly by T 0 and let a 0 wll be the value of the functon F on the sets from T 0. The famly of sets, that maxmze the functon F over - T 0, we wll denote by T 1 and by a 1 wll be denoted the value of the functon F on these sets. Contnung ths process we have = T. It s easy to see that U L = T s a U t =0 =0 subsemlattce of, where L = { X : F ( a }. We wll call these subsemlattces by level semlattces. Denote by K the mnmal element (null) of level semlattce L. nce L L +1, we obtan K 0... K t. Theorem 3.3. Let = A 0... A p be a chan of sets that were stored as local optmal subsets X 0 n the Chan Algorthm runnng on an antmatrod (E,), then ths chan concdes wth the chan of level semlattce nulls K 0... K t. Proof. From the algorthm constructon follows that f = X 0 X 1 X k s a sequence of sets generated by the Chan Algorthm, then for each X such that Al 1 X Al, we have F ( X ) F 1) < F ). ence, (6) X Al F ( X ) F 1) To prove the Theorem, we frst prove the followng clam: Clam. For each l = 0,1,...,p and for each X, f A l X, then F ( F 1) < F ) By analogy wth the proof of Theorem 3.2 (case ()) we obtan that there exst the set X belongng to the chan generated by the Chan Algorthm, such that X A l and F (X) F (X ), then the Clam follows from (6). Now prove that for each l = 0,1,...,p, A l s a null of some level semlattce L,.e., A l = K. Let a =F (A l ). From the Clam mmedately follows that A l s a null of L. It remans to show that for each null K exsts A l = K. nce F (A p ) s a maxmal value of the functon F, for each null K exsts 1 l p, such that F 1) < F ( K ) F ) or F ( K ) F ) where A l =. how that K A l. In the second case (A l = ) t s obvously. uppose, that n the frst case A l K, then from the Clam we have F ( K ) F 1), a contradcton. On the other hand, A l L,.e., K A l because K s a null of L. Then A l = K. The obtaned chan-nested structure, referred n [7,8] as a layered cluster, can be consdered an abstract mplementaton of the dea of multresolutonal vew on a system of nterrelated obects.
4 Quas-concave functons and convex geometry In ths secton we consder convex geometres. As set functons, defned on convex geometres, we nvestgate functons whch are dual to quasconcave functons ntroduced n the prevous sectons. Let (E,) be a convex geometry and let F ( = F ( E for each X. From Theorem 2.2 follows that the set system = {E X : X } s an antmatrod, then the defnton s correct. From the defnton follows that F ( = mn π ( x, E. Then, by usng (3) we F ( = ( E mn π ( x, E = x ex( obtan * mn π ( x, x ex( It s easy to see that the functon π * s a monotone ncreasng lnkage functon where f X,Y and X Y then π * (x,x) π * (x,y). It should be menton, that we not assume that the orgnal gven functon s ust a monotone lnkage functon π. If a gven functon s a monotone ncreasng lnkage functon π *, then we straghtforwardly obtan the set functon F defned on the convex geometry. Consder, for example, * π ( x, = max s, defned n terms of par-wse y X xy smlarty s xy. Obvously, thus defned π * s a monotone ncreasng functon. Then the functon F may be referred as the tghtness functon [8], and a subset maxmzng ths functon can be consdered as a dense cluster. Notce that the feasble sets of a convex geometry ordered by ncluson form a lattce L, wth lattce operatons: X Y = τ ( X Y), X Y = X Y then we obtan: Lemma 4.1 If a set system (E,) s a convex geometry, then for two lattces L and L we have F ( X Y ) = F ( X Y ) The proof s based on the Theorem 2.2 and on (2). Theorem 4.1 If a set system (E,) s a convex geometry, then the functon F s a quas-concave functon on L,.e., for each X,Y, F (X Y ) mn {F (X), F (Y)} Proof. From the Lemma 4.1 and from the Theorem 3.1 we have F ( X Y ) = F ( X Y ) mn{ F = mn{ F (, F ( Y )} (, F ( Y )} = Consder the followng optmzaton problem - gven a monotone lnkage functon π *, and a convex geometry (E,), fnd the feasble set X such that F (X) = max{f (Y) : Y }. From Theorem 4.1 t follows that the set of the optmzaton problem solutons s a on-semlattce wth unque maxmal element. To fnd ths maxmal element we use the algorthm that s a mrror verson of the Chan Algorthm: The Mrror Chan Algorthm ( E,, π * ) 1. X := E 2. X 0 := E 3. Whle ex (X) do 3.1 f F (X) > F (X 0 ), X 0 :=X 3.2 choose x ex (X) such that π * (x, π * (y, for all y ex (X) 3.3 X := X - x 4. Return X 0 Thus, the algorthm generates a chan of sets E = X 0 X 1 X k, where X = X -1 - x and x ex(x -1 ), and returns the maxmal set X 0 of the chan on whch the value F (X 0 ) s maxmal. Theorem 3.2 Let a set system (E,) be a convex geometry, then for all monotone lnkage functon π * the Mrror Chan Algorthm fnds a maxmal feasble set that maxmzes the functon F. The proof of ths Theorem mmedately follows from Theorem 3.2, snce each step X X x of the Mrror Chan algorthm correspondences to the step X X x of the orgnal Chan Algorthm, and the result of the Mrror Chan Algorthm s a complement of the mnmal set obtaned by the Chan Algorthm. By analogy wth a quas-concave functon F a quas-concave functon F determnes on a convex geometry (E,) the chan-nested structure of onsemlattces L 0 L t, where 0 t L = { X : F ( a } and a >... > a are the all values of functon F on a convex geometry (E,). We also call these subsemlattces by level semlattces. Denote by I the maxmal element (one) of level semlattce L. nce L L +1, we obtan 0 1 t I I... I.
Theorem 4.3 Let E = A 0 A 1 A p be a chan of sets that were stored as a local maxmal subsets X 0 n the Mrror Chan Algorthm runnng on a convex geometry (E,), then ths chan concdes wth the chan of level semlattce ones 0 1 t I I... I, such that the maxmal optmal subset A p =I 0 and E=A 0 =I t. [9] J.Mullat, Extremal subsystems of monotone systems: I, II, Automaton and Remote Control 37, (1976) 758-766; 1286-1294 [10] Y.Zaks (Kempner), and I.Muchnk, Incomplete classfcatons of a fnte set of obects usng monotone systems, Automaton and Remote Control 50, (1989), 553-560 The proof of the Theorem 4.3 s based on the dualty of antmatrods and convex geometres, and on Theorem 3.3 5 Concluson In ths paper we have nvestgated the problem of clusterng wth constrants. The clusterng problem was consdered as optmzaton problem on antmatrods and on convex geometres wth quasconcave goal functon. The greedy type Chan algorthm, whch allows to fnd an optmal cluster, also as the most dstant group on antmatrods and as dense cluster on convex geometres, was descrbed. References: [1] A. Börner and G.M. Zegler, Introducton to greedods, n Matrod applcatons, ed. N. Whte, Cambrdge Unv.Press, Cambrdge, UK,1992 [2] J.Edmonds, Matrod and the greedy algorthm, Mathematcal Programmng 1, (1971),127-136 [3] Y. Kempner, and V.Levt, Algorthmc characterzatons of antmatrods, Thrd afa Workshop on Interdscplnary Applcatons of Graph Theory, Combnatorcs and Algorthms, afa, Israel,2003 [4] Y.Kempner, B.Mrkn, and I.Muchnk, Monotone lnkage clusterng and quas-concave functons, Appl.Math.Lett. 10,No.4 (1997) 19-24 [5] B.Korte, L.Lovász, and R.chrader, ''Greedods'', prnger-verlag, New York/Berln, 1991 [6] A.Malshevsk, Propertes of ordnal set functons, n A.Malshevsk,''Qualtatve Models n the Theory of Complex ystems'', Nauka, Moscow,1998 (n Russan) [7] B.Mrkn, and I.Muchnk, Layered clusters of tghtness set functons, Appl.Math.Lett,15 (2002),147-151 [8] B. Mrkn, and I.Muchnk, Induced layered clusters, heredtary mappngs and convex geometres, Appl.Math.Lett,15 (2002),293-298