Integrating Fuzzy c-means Clustering with PostgreSQL *

Transcription

1 SQL alle pgfcm. Seton 0 brefly susses relate work. Seton 0 ontans onluson remarks an retons for future work. Integratng Fuzzy -Means Clusterng wth PostgreSQL * R. M. Mnakhmetov taven@gmal.om South Ural State Unversty Abstrat. Many ata sets to be lustere are store n relatonal atabases. Havng a lusterzaton algorthm mplemente n SQL proves easer lusterzaton nse a relatonal DBMS than outse wth some alternatve tools. In ths paper we propose Fuzzy -Means lusterng algorthm aapte for PostgreSQL open-soure relatonal DBMS. Keywors: fuzzy -means; fm; fuzzy lusterng; postgresql; ntegratng lusterng; relatonal bms. 1. Introuton Integratng lusterng algorthms s a top ssue for atabase programmers [1]. Suh an approah, on the one han, enapsulates DBMS nternal etals from applaton programmer. On the other han, t allows to avo overhea onnete wth export ata outse a relatonal DBMS. The Fuzzy -Means (FCM) [], [3], [4] lusterng algorthm proves a fuzzy lusterng of ata. Currently ths algorthm have many mplementatons on a hgh-level programmng languages [5], [6]. For mplementaton the FCM algorthm n SQL we hoose an open-soure PostgreSQL DBMS [7]. The paper s organze as follows. Seton ntroues bas efntons an an overvew of the FCM algorthm. Seton 3 proposes mplementaton of the FCM n * Ths paper s supporte by the Russan Founaton for Bas Researh (grant No a). 63. The Fuzzy -Means Algorthm The K-Means [8] s one of the most popular lusterng algorthms, t s a smple an farly fast [9]. The FCM algorthm generalzes K-Means to prove fuzzy lusterng, where ata vetors an belong to several parttons (lusters) at the same tme wth a gven weght (membershp egree). To esrbe FCM we use the followng notaton: N mensonalty of a ata vetors (or ata tems) spae; l N : 1 l subsrpt of the vetor's oornate; 64 n N arnal number of tranng set; X R tranng set for ata vetors; N : 1 n vetor subsrpt n a tranng set; x X the -th vetor n the sample; k N number of lusters; N : 1 k luster number; k C R matrx wth lusters' enters (entros); R enter of luster, -mensonal vetor; R U x, l -s oornates of vetors l l x an respetvely; nk R matrx wth membershp egrees, where R 0 u 1 s a membershp egree between vetor x an luster ; u : x ) stane funton, efnes a membershp egree between vetor ( x an luster ; m R : m > 1 the fuzzyfaton egree of obetve funton;

2 J FCM obetve funton. The FCM s base on mnmzaton of the obetve funton J FCM N k J FCM : m ( X, k, m) = u ( x ) (1) =1 =1 Fuzzy lusterzaton s arre out through an teratve optmzaton of the obetve funton (1). Membershp matrx U an entros are upate usng the followng formulas: Input: X, k, m, Ouput: U (0) 1. s := 0, U := ( ) {ntalzaton} u. repeat {omputaton of new entros' oornates} ( s) 3. Compute C := ( ) usng formula (3) where u U (s) u = ( x k t= 1 ( x t ) ) 1m () n m u xl =1, l l = n (3) m u =1 (s) ( 1) Let s s a number of teraton, u an u s are elements of matrx U on steps s an s 1 respetvely, an (0,1) R s a termnaton rteron. Then the termnaton onton an be wrtten as follows: s1) max{ u u ( ( s) } < Obetve funton (1) onverges to a loal mnmum (or a sale pont) [10]. (4) {upate matrxes values} 4. Compute 5. s := s 1 (s) ( s1) U an U usng formula () ( s) ( s1) 6. untl max{ u u } Fg. 1. The Fuzzy -Means Algorthm. Algorthm showe on Fg. 1 presents the bas FCM. The nput of algorthm reeves a set of ata vetors X = ( x1, x,, xn), number of lusters k, fuzzyfaton egree m, an termnaton rteron. The output s a matrx of membershp egrees U. 3. Implementaton of Fuzzy -Means Algorthm usng PostgreSQL In ths seton we suggest pgfcm algorthm as a way to ntegrate FCM algorthm wth PostgreSQL DBMS. 3.1 General Defntons To ntegrate FCM algorthm wth a relatonal DBMS t s neessary to perform matrxes U an X as relatonal tables. Subsrpts for entfaton elements of 65 66

3 n k, relatonal tables are presente n Table 1 (numbers, are efne above n a seton ). Table 1. Data Elements Subsrpts. Subsrpt Range Semants l 1, n vetor subsrpt 1, k luster subsrpt 1, vetor's oornate subsrpt As a funton of stane ( x ), wthout loss of generalty, we use the Eulean metr: ( x ) = ( xl l ) (5) l=1 To ompute the termnaton rteron (4) we ntroue the funton as follows: ( s1) ( s) = max{ u u } (6) 3. Database Sheme Table summarzes atabase sheme of pgfcm algorthm (unerlne olumns are prmary keys). Table. Relatonal Tables of pgfcm Algorthm No. Table Semants Columns 1 SH SV tranng set for ata vetors (horzontal form) tranng set for ata vetors (vertal form) Number of rows, x1, x,, x n, l, val n 67 3 C entros' oornates, l, val k 68 4 SD stanes between x an 5 U 6 UT 7 P egree of membershp vetor x to a luster on step s egree of membershp vetor x to a luster on step s 1 result of omputaton funton (6) on step s,, st n k,, val n k,, val n k, k, n, s, elta s In orer to store sample of a ata vetors from set X t s neessary to efne table SH (, x1, x,, x). Eah row of sample stores vetor of ata wth menson an subsrpt. Table SH has n rows an olumn as a prmary key. FCM steps eman aggregaton of vetor oornates (sum, maxmum, et.) from set X. However, beause of ts efnton, table SH oes not allow usng SQL aggregaton funtons. To avo ths obstale we efne a table SV (, l,, whh ontans n rows an have a omposte prmary key (, l). Table SV represents a ata sample from table SH ans supports SQL aggregaton funtons max an sum. Due to store oornates of luster entros temporary table C (, l, s efne. Table C has k rows an the omposte prmary key (, l). Lke the table SV, struture of table C allows to use aggregaton funtons. In orer to store stanes ( x ) table SD (,, st) s use. Ths table has n k rows an the omposte prmary key (, ). Table U (,, stores membershp egreesalulate on s-th step. To store membershp egrees on s 1 step smlar table UT (,, s use. Both tables have n k rows an the omposte prmary key (, ). Fnally, table P (, k, n, s, elta) stores teraton number s an the result of omputaton funton (6) elta for ths teraton number. Number of rows n table P epens on the number of teratons.

4 3.3 The pgfcm Algorthm The pgfcm algorthm s mplemente by means of a store funton n PL/pgSQL language. Fg. shows the man steps of the pgfcm algorthm. Input: Ouput: U SH, k, m, eps {ntalzaton} 1. Create an ntalze temporary tables ( U, P, SV, et.). repeat {omputatons} 3. Compute entros oornates. Upate table C. 4. Compute stanes x ). Upate table SD. ( 5. Compute membershp egrees UT ( ut ). {upate} 6. Upate tables P an U. {hek for termnaton} 7. untl P. elta eps = Fg.. The pgfcm Algorthm. The nput set of ata vetors X store n table SH. Fuzzyfaton egree m, termnaton rteron, an number of lusters k are funton parameters. The table U ontans a result of pgfcm work. 3.4 Intalzaton Intalzaton of tables SV, U, an P prove by SQL-oe I1, I, an I3 respetvely. Table SV s forme by samplng reors from the table SH. I1: INSERT INTO SV SELECT SH., 1, x1 FROM SH;... INSERT INTO SV SELECT SH.,, x FROM SH; For table U a membershp egree between ata vetor for all =. I: INSERT INTO U (,, VALUES (1, 1, 0);... INSERT INTO U (,, VALUES (,, 1);... INSERT INTO U (,, VALUES (n, k, 0); x an luster takes 1 In other wors, as a start oornates of entros, frst ata vetors from sample X are use. = u = 1 = x When ntalzng the table P, the number of ponts k s taken as a parameter of the funton pgfcm. A ata vetors spae mensonalty an a arnal number of the tranng set n also prove by funton pgfcm parameters. The teraton number s an elta ntalzes as zeros. I3: INSERT INTO P(, k, n, s, elta) VALUES (, k, n, 0, 0.0);

5 3.5 Computatons Aorng to Fg., the omputaton stage s spltte to the followng three substeps: omputaton oornates of entrosomputaton of stanes, an omputaton membershp egrees, marke as C1, C, an C3 respetvely. C1: INSERT INTO C SELECT R1., R1.l, R1.s1 / R.s AS val FROM (SELECT, l, sum(u.val^m * SV. AS s1 FROM U, SV WHERE U. = SV. GROUP BY, l) AS R1, (SELECT, sum(u.val^m) AS s FROM U GROUP BY ) AS R WHERE R1. = R.; C: INSERT INTO SD SELECT,, sqrt(sum((sv.val - C.^))) AS st FROM SV, C WHERE SV.l = C.l GROUP BY, ; Through the FCMomputatons of the stanes prove by formula (). In formula (3) the fraton's numerator oes not epen on t, then we an rewrte ths formula as follows: u 1 k 1m 1 = (, ) m x (, ) x t (7) t=1 val en 3.6 Upate SELECT,, SD.st^(.0^(1.0-m)) * SD1.en AS FROM (SELECT, 1.0 / sum(st^(.0^(m-1.0))) AS FROM SD GROUP BY ) AS SD1, SD WHERE SD. = SD1.; Upate stage of the pgfcm mofes P an U tables as shown below n U1 an U respetvely. U1: INSERT INTO P SELECT L., L.k, L.n, L.s + 1 AS s, E.elta FROM (SELECT,, max(abs(ut.`val - U.) AS elta FROM U, UT GROUP BY, ) AS E, (SELECT, k, n, max(s) AS s FROM P GROUP BY, k, n) AS L) AS R Table UT stores temporary membershp egrees to be nserte nto table U. To prove the rap removal all the table U rows, obtane at the prevous teraton, we use the trunate operator. U: TRUNCATE U; INSERT INTO U SELECT * FROM UT; 3.7 Chek Thus, the omputaton of membershp egrees an be wrtten as follows: C3: INSERT INTO UT 71 Ths stage s the fnal for the algorthm pgfcm. On eah teraton the termnaton onton (4) must be heke. To mplement the hek, the result elta of the funton (6) from table P s store n the temporary varable tmp. 7

6 CH1: SELECT elta INTO tmp FROM P,(SELECT, k, n, max(s) AS max_s FROM P GROUP BY, k, n) AS L WHERE P.s = L.max_s AND P. = L. AND P.k = L.k AND P.n =L.n; < After seletng the elta, we nee to hek the onton onton s true we shoul stop, otherwse, work wll be ontnue. CH: IF (tmp < eps) THEN RETURN; END IF; The fnal result of the algorthm pgfcm wll be store n table U. 4. Relate Work. Then f ths Researh on ntegratng ata mnng algorthms wth relatonal DBMS nlues the followng. Assoaton rules mnng s explore n [11]. General ata mnng prmtves are suggeste n [1]. Prmtves for eson trees mnng are propose n [13]. Our researh was nspre by papers [1], [14], where ntegratng K-Means lusterng algorthm wth relatonal DBMS, was arre out. The way we explot s smlar to mentone above. The man ontrbuton of the paper s an extenson of results presente n [1], [14] for the ase where ata vetors may belong to several lusters. Suh a ase s very mportant n problems onnete wth mene ata analyss [15], [16]. To the best of our knowlege there are no papers evote to mplementng fuzzy lusterng wth relatonal DBMS. 5. Conluson In ths paper we have propose the pgfcm algorthm. pgfcm mplements Fuzzy - Means lusterng algorthm an proesses ata store n relatonal tables usng PostgreSQL open-soure DBMS. There are followng ssues to ontnue our researh. Frstly, we plan to nvestgate pgfcm salablty usng both synthetal 73 an real ata sets. The seon reton of our researh s evelopng a parallel verson of pgfcm for strbuton memory multproessors. Referenes 74 [1] C. Oronez. Programmng the K-means lusterng algorthm n SQL. Proeengs of the 10th ACM SIGKDD Internatonal Conferene on Knowlege Dsovery an Data Mnng, 004, pp [] A. K. Jan, M. N. Murty, an P. J. Flynn. Data lusterng: a revew. ACM Computng Surveys, 1999, Vol. 31, Iss. 3, pp [3] J. C. Dunn. A Fuzzy Relatve of the ISODATA Proess an Its Use n Detetng Compat Well-Separate Clusters. Journal of Cybernets, 1973, Vol. 3, Iss. 3, pp [4] J. C. Bezek. Pattern Reognton wth Fuzzy Obetve Funton Algorthms. Kluwer Aaem Publshers, Norwell, USA, 1981, p. 56. [5] E. Dmtraou, K. Hornk, F. Lesh, D. Meyer, an Wengessel A. Mahne Learnng Open-Soure Pakage r-ran-e1071, Referene ate: [6] I. Drost, T. Dunnng, J. Eastman, O. Gosponet, G. Ingersoll, J. Mannx, S. Owen, an K. Wettn. Apahe Mahout, Referene ate: [7] M. Stonebraker, L. A. Rowe, an M. Hrohama. The Implementaton of POSTGRES. IEEE Transatons on Knowlege an Data Engneerng, Marh 1990, Vol., Iss. 1, pp [8] J. B. MaQueen. Some Methos for Classfaton an Analyss of MultVarate Observatons. Proeengs of 5th Berkeley Symposum on Mathematal Statsts an Probablty, 1967, Vol. 1, pp [9] P. S. Braley, U. M. Fayya, an C. Rena. Salng Clusterng Algorthms to Large Databases. Proeengs of the 4th Internatonal Conferene on Knowlege Dsovery an Data Mnng, 1998, pp [10] J. Bezek, R. Hathaway, M. Sobn, an W. Tuker. Convergene Theory for Fuzzy -Means: Counterexamples an Repars. IEEE Transatons on Systems, Man an Cybernets, 1987, Vol. 17, Iss. 5, pp [11] S. Sarawag, S. Thomas, an R. Agrawal. Integratng assoaton rule mnng wth relatonal atabase systems: alternatves an mplatons. Proeengs of the 1998 ACM SIGMOD Internatonal Conferene on Management of Data, 1998, pp [1] J. Clear, D. Dunn, B. Harvey, M. Heytens, P. Lohman, A. Mehta, M. Melton, L. Rohrberg, A. Savasere, R. Wehrmester, an M. Xu. NonStop SQL/MX prmtves for knowlege sovery. Proeengs of the 5th ACM SIGKDD nternatonal onferene on Knowlege sovery an ata mnng, 1999, pp [13] G. Graefe, U. M. Fayya, an S. Chauhur. On the Effent Gatherng of Suffent Statsts for Classfaton from Large SQL Databases. Proeengs of the 4th Internatonal Conferene on Knowlege Dsovery an Data Mnng, 1998, pp

7 [14] C. Oronez. Integratng K-Means Clusterng wth a Relatonal DBMS Usng SQL IEEE Transatons on Knowlege an Data Engneerng, 006, Vol. 18, Iss., pp [15] A. I. Shhab. Fuzzy Clusterng Algorthms an ther Applatons to Meal Image Analyss. PhD thess, Unversty of Lonon, 000. [16] D. Zhang an S. Chen. A Novel Kernelze Fuzzy -Means Algorthm wth Applaton n Meal Image Segmentaton. Artfal Intellgene n Mene, 004, Vol. 3, pp