Comparison of Performance in Text Mining using Categorization of Unstructured Data

Transcription

1 Indan Journal of Scence and Technology, Vol 9(4), DOI: /jst/06/v94/9648, June 06 ISSN (Prnt) : ISSN (Onlne) : Comparson of Performance n Text Mnng usng Categorzaton of Unstructured Data Lee Junyeon *, Shn Seungsoo and Km Jungju 3 Department of Meda Engneerng, Tongmyung Unversty, Korea; jylee@tu.ac.kr Department of Informaton Securty, Tongmyung Unversty, Korea; shnss@tu.ac.kr 3 Choonhae College of Health, Korea; jj79005@hanmal.net Abstract Background/Objectves: The text mnng would help fndng nformaton to the users n the enormous documents. The text mnng has been actvely developed and utlzed n varous felds, manly Englsh-based document, but Study on the Korean text mnng has been relatvely lmted. The mportance of the Korean text mnng has emerged wth ncreasng bg data ncludng Korean text data, the needs for the ntensve study and applcaton of Bg Data are ncreasng. Methods/ Statstcal Analyss: In ths study, we compared the performance of these classfcatons by applyng the method of Bayesan methods, k-nn, decson trees, SVM, and as a neural network n classfcaton of unstructured newspaper artcle nto gven categores. Fndngs: In the experment result, the SVM model has a hgh F-measure value relatve to other models, and has shown stable results n the classfcaton nformaton and recall rate. Also, ths model showed a hgh F-measure value n the classfcaton of a more granular lst. Applcaton/Improvements: The methods of k-nn and decson tree show slghtly lower performance than SVM, they are turned out to be approprate models usng classfcaton problem cause of havng advantages to easy nterpretaton and short learnng tme. Keywords: Categorzaton, Decson Tree, k-nn, Nave Bayes, Text Mnng. Introducton The goal of nformaton access s to help users fnd documents that satsfy ther nformaton needs. The standard procedure s akn to lookng for needles n a needlestack - the problem sn t so much that the desred nformaton s not known, but rather that the desred nformaton coexsts wth many other vald peces of nformaton. The goal of data mnng s to dscover or derve new nformaton from data, fndng patterns across datasets, and/or separatng sgnal from nose. The fact that an nformaton retreval system can return a document that contans the nformaton a user requested does not mply that a new dscovery has been made: the nformaton had to have already been known to the author of the text; otherwse the author could not have wrtten t down. A large database treated n data mnng can be classfed nto an unstructured db and structured db accordng to the structure. Data mnng has been developed mostly focused based on the structured database technques up to now. Therefore, data mnng nvestgators were aware of the need to research on the unstructured databases, there have recently been ncreasng studes wth them. Text mnng can be dvded nto data processng and data analyss. Data processng s a step of machnng to facltate the unstructured data to the data analyss, and data analyss s extractng meanngful nformaton from the text usng data mnng, machne learnng, statstcs. <Table > shows the classfcaton of data mnng and text data mnng applcatons. The researches n text mnng of textual materal are n progress due to the ncreasng of amount of data. Extractng a pattern or relatonshp from the text data, such as web pages, electronc documents, electronc mal, a process of managng the new nformaton, the classfcaton technques to mnng technques to the text data based on a gven keyword assgnng artcle and wthout advance nformaton, there are smlar bndng document clusterng technques to group. Document *Author for correspondence

2 Comparson of Performance n Text Mnng usng Categorzaton of Unstructured Data Table. A classfcaton of data mnng and text data mnng applcatons Non-textual data Textual data Fndng Patterns standard data mnng computatonal lngustcs Fndng Nuggets Novel classfcaton scheme s classfed accordng to a gven keyword, and accordng to a gven set of keywords, the document s determned whether classfed nto the category or not. Document classfcaton allows convenent access to the user a number of documents by structurng categores automatcally. So, t has emerged as a very mportant factor for effcent nformaton management and search. Utlzng a document classfcaton scheme of text mnng n prevous studes usng a neural network and k-nn classfcaton nto four categores, t was compared to the performance 3. In ths paper, we classfy the bg data consstng of Hangul and compared the performance between the varous classfcaton technques. In chapter, we descrbe how to formalze unstructured text data. And chapter 3 says theoretcal background of varous classfcaton technques, and chapter 4 extracts the result by applyng real data. Fnally chapter 5 presents the conclusons.? real Textual Data Mnng Non-Novel database queres nformaton retreval Fgure s a fgure showng a process to conduct a morpheme analyss to extract a keyword correspondng to the noun. In the morphologcal analyss, the scheme excluded the postpostonal partcle, verb, etc., and extracted only the classfcaton word as a noun. And then, the extracted nouns are structured to apply a statstcal analyss technque. Because stop words do not gve any nformaton wth appearng n several or every documents, the extracted nouns except wth the stop words wll be used.. Extractng General Format Document should convert nto structured data to be sutable for analyss because the unstructured data. A document can be expressed as a matrx of columns ndcatng the lne that represents a keyword extractng a set of documents by usng the concept of a set of words and documents 4. Ths matrx s called word-document matrx, t can be expressed n the form of a matrx of m n words and documents, such as Table. In the Table, f j means the frequency of the -th word s ncluded n the j-th document..3 Weghted Functon To ndcate a better correlaton of words, word-document matrx t can be converted by usng the weght functon.. Text Mnng The purpose of text mnng s to fnd useful patterns from the large documents and t means to apply an algorthm to the method of the text data and the statstcal machne learnng. In the aspect of fndng a pattern and extractng the nformaton from a bg data, text mnng s smlar to data mnng, except usng unformulated data.. Extractng Keywords Words that reflect the content and features of a document called the content word. Frst, we determnes each part of speech by analyzng the morpheme of the sentence n order to extract the content word. Statements nclude varous word of part of speech, and the noun s used to ntroduce and descrbe the concept of a statement. The noun s the most appeared to cent word, so we can revew t the most weght. Fgure. Table..Example of a keyword extracton. The World-Document Matrx Doc Doc j Doc n Word f f j f n Word f f j f n Word m f m f mj f mn Vol 9 (4) June 06 Indan Journal of Scence and Technology

3 Lee Junyeon, Shn Seungsoo and Km Jungju Weght functon s composed of local weght functon and global weght functon 5. Elements of the transformaton matrx s equaton (.) usng a word wth the document j, as can be calculated as the product of a functon of both weght L(,j) represents a local weght functon and G() represents global weght functon. aj = L(, j) G ( ) (.) The local weght functon uses a smple frequency, or a smple log converson or bnomal converson form generally. Smple frequency functon : L (, j) =, Log converson functon : f j L (, j) = log ( + ) f j, Bnomal converson functon : fj, L (, j) = 0 fj, = 0 Regonal weght functon gves weghted value to a word ncluded n document j s represented as L(,j). The weghts of the words for the entre document referred to globally weght functon, and represented as G(). Ths s also known as the term weghts, and are used to supplement the dsadvantages of regonal weght functon does not reflect the characterstcs of the words for the entre collecton of documents 6. Global weghtng functon has a functon usng the functon, the functon representng the words the nformaton that appears n quantty, functon and standardzed concept consderng the frequency and the word s the rato of the ndcated number of documents n the word n the entre document usng the entropy concept as follows, then and t s expressed as follows. The functon usng entropy: G () = + f g j fj log g log ( N) The functon of the word by quanttatve: N G () = log + d The functon consderng the rato (frequency of words vs. number of documents): G ()= g d In the global weght functon g s the frequency wth whch the word appears throughout the document collecton and, d s the number of document, and N means total number of document. Globally weghted usng the concept of entropy has a large value when the emergence of a rare word n the document. If a certan word once all appearance n all documents, f the word s not a characterstc that dfferentates the respectve document locally weghted usng the entropy concept has a value of 0, only appear a certan word one document of the entre document, It has a value of. Ths global weghtng functon usng the standardzaton concept of regonal weghts have the same characterstcs as used n the collecton of documents represents the percentage of words that appear n the document collecton Classfcaton Method 3. Baysan Method The Bayesan method s one of the most wdely used algorthms n the feld of document classfcaton, calculate the posteror probablty that t wll be assgned to each category receves a sngle document usng the Bayesan theory, such as the equaton (3.) 8. Pd ( Cj) P( C j) PC ( j d) = (3.) Pd ( ) In ths Formula, d means random document, Cj means j-th category. P(d) has the same value to all categores, doesn t need to calculate probablty. Bayesan method assumes all the words are ndependent from each other, assgnment of the category that occurs n the document s that mutually exclusve. So, the P(C j d) can be calculated as follow equaton (3.) PC ( d) = P( C ) P( w C ) j j j = n (3.) In (3.), P(w C j ) means (the number of w s occurrence n C j )/(the number of all words occurrence n C j ), and P(C j ) means (the number of documents allocated n C j )/(the number of all documents). Vol 9 (4) June 06 Indan Journal of Scence and Technology 3

4 Comparson of Performance n Text Mnng usng Categorzaton of Unstructured Data The Baysan method calculates the probablty of beng classfed n each category by assgnng a document to a category havng the maxmum value, because a number of categores are categores to calculate the condtonal probablty for each of the categores havng the hghest probablty belong to the document. 3. k-nn The k-nn classfcaton method s to select one pattern from among the stored pattern wth a dstance of at least the learnng data wth respect to any partcular pattern and classfes the category number whch belongs to the category of the gven pattern 9. The k-nn algorthm s prmarly used to the degree of smlarty between the new document (d x ) and the learnng document (d j ) to fnd the degree of smlarty s hgh top k neghborng documents n document study groups. Representatve degree of smlarty s the cosne smlarty degree s calculated as n equaton (3.3). sm( d, d ) = x j k k t xk t jk (t ) ( t ) xk k jk (3.3) In the equaton (3.3), t xk, t jk means the weght value of word k appeared n d x, d j. The degree of smlarty between the new document and document neghbors s used as the document category, the weght of the neghborhood, f neghborng pages when the category weghts, share category s hgh. When extractng k learnng documents among the new document and the order of smlarty usng the cosne smlarty, the category assgned to each k learnng documents s a canddate lst of categores to be assgned to the new document. The order to fnd the best category to be assgned to the new document category from the canddate lst, calculate the complance wth total frequency of each category or the smlarty score per category such as expresson (3.4) n the learnng document classfcaton. rel( Ck dx) sm( dx, dj) {( Ck d j) (3.4) j In the k-nn algorthm selecton of k t wll have a large mpact on the classfcaton result. Lkely k s not too small, there s a possblty of overchargng as the sum of the nose data for tranng, as opposed to the data groups classfed as near to that k s too large classfcaton exst. 3.3 Decson Tree Decson tree s made a decson rule to classfy the functon. In supervsed learnng problem t s sometmes mportant to the predcton of the fnal model and analyss even more emphass on predctve analyss or accordng to crcumstances. By dvdng the regons of each varable repeatedly wth supervsed learnng technques, decson tree creates a rule wth ncorrect predcton and correct analyss relatvely to the other supervsed learnng technques for the whole area. Rules created by the decson tree has the advantage of beng easy to understand and easy to mplement, because they have f-then structure and smlar to SQL database language format. The analyss of the decson tree s composed of growth of the decson tree, prunng, feasblty study, analyss and forecastng as shown n Table 3. The most wdely used n decson tree algorthm s CHAID. Ths can be appled to all types of the target and classfcaton varables, such as nomnal, the order-type, contnuous type. 3.4 SVM SVM s recevng attenton not only n the classfcaton problem, and also applcable to a regresson problem, accurate and applcable to varous types of data of the predcted predcton easer problem. Unlke the probablty estmaton of the object to mnmze the emprcal rsk, such as the logstc regresson and dscrmnant analyss conventonal methods classfcaton SVM s got wth the purpose to mnmze structural rsk look place only the classfcaton effcency tself over exstng probablty estmaton method overall, hgher predctablty. Table 3. Dvson Growng decson tree prunng feasblty study analyss and forecastng The Analyss Process of Decson Tree Descrpton Grow the trees to fnd an approprate optmum separaton crtera, and stop f they meet the approprate stoppng rules. Remove a branch whch wll ncrease the rsk of an error sgnfcantly or has an mproper nference rules. Evaluate the decson tree usng gan chart, rsk chart or verfcaton data Interpret and apply buld a wooden model after settng a predcton model predctons. 4 Vol 9 (4) June 06 Indan Journal of Scence and Technology

5 Lee Junyeon, Shn Seungsoo and Km Jungju The reason of effectveness n SVM s that t performs lnear classfcaton quckly and easly when the lnearty see the non-lnear data set at a low level at a hgher level and expand ts dmensons. Most of the pattern and can not be lnearly separated, to separate the non-lnear patterns and converts the nput space of the non-lnear pattern n the feature space of the lnear pattern. At ths tme, the kernel functons are used to classfy non-lnear patterns. K( xx, ) = (( x x ) + ) (3.5) Polynomal kernel functon s dependent on the drecton between the two vectors, a vector havng the same drecton, they eventually result there s gven a hgh value s for a polynomal kernel functon, a polynomal kernel functon s the same as equaton (3.5). 4. Experments In ths study, we used a Korea Herald artcle for one month (n January 06). Total 340 artcles related to North Korea and nuclear are used n the analyss. It was repeated 0 tmes performed to ensure the valdty of the generalzed model buldng, and analyzed through the same procedure as n <Fgure >. 4. Structurng Data Snce the newspaper artcle s unstructured data, we perform the structurng process at frst to apply formal analyss technques. The frst step of structurng s to extract a keyword from the data, the newspaper artcle, comprses a varety of combnatons, such as letters, specal characters, numbers. Table 4 shows a part of newspaper artcle and the keywords extracted. Even though some nouns appeared n all documents wthout meanng, we exclude them and put to the stop words. The vector of the rough wth the keywords extracted noun s created for every artcle. Usng ths vector, word-document matrx such as Table 5 was created. Table 5 s a part of word-document matrx generated by confgurng each of the newspaper artcles. In the vector, column s words extracted from artcle, and row s each newspaper artcle number. In Table 5, the number means Doc 8, Doc 9 and Doc nclude the word coal n Fgure. Analyss Process. d Table 4. Keywords Extracton Artcle Chna could stop buyng coal from North Korea to punsh the ally for ts recent nuclear test, a South Korean expert suggested Thursday amd calls for Bejng to take a frmer stance aganst Pyongyang. Chna has come under growng pressure from South Korea and the Unted States to help draw a strong sanctons resoluton from the U.N. Securty Councl to punsh the North for ts fourth nuclear test last week. Cho Kyung-soo, presdent of the North Korea Resources Insttute n Seoul, noted the North s hgh relance on trade wth Chna. Coal exports account for nearly half of all North Korean exports to Chna, he sad n a phone ntervew wth Yonhap News Agency. North Korea earned $.84 bllon from exports to Chna n 04, nearly 90 percent of the $3.6 bllon earned n total, accordng to data from the nsttute and the Korea Trade-Investment Promoton Agency. Table 5. terms Word-Document Matrx documents Extracted nouns coal, North Korea, ally, nuclear, test, South Korea, Thursday, calls, Bejng, stance, Pyongyang, Chna, pressure, Unted States, sancton, resoluton, U.N. Securty Councl, week, Cho Kyungsoo, presdent, Resources Insttute, Seoul, relance, trade, half, he, phone, ntervew, Yonhap News Agency, bllon, percent, data, Korea Trade Investment Pormoton Agency Coach Coagulaton Coal Coalton Coast Coat Coater Coatng Coatrack Coatroom the artcle. Word-document matrx s the basc form of a newspaper artcle whch converts unstructured data nto structured data. Ths study s due to the newspaper s vew to applyng classfcaton technques to formalze the newspaper nto one object, word-document matrx was appled to Vol 9 (4) June 06 Indan Journal of Scence and Technology 5

6 Comparson of Performance n Text Mnng usng Categorzaton of Unstructured Data the pre-analyss. The matrx transposton s located on the lne newspapers, and a consderable number of words havng the meanng of the word matrx rather than a varable number of newspaper artcles that correspond to the nature of the data object to the locaton column. 4. Comparson of Models To evaluate the performance of the classfcaton model, t was used for Correct Classfed Rate (CCR), Recall Rate (RR), and F-measure to be used n document classfcaton crtera. The Correct Classfed Rate s the rate of correct predcton, and Recall Rate s the rato actually ht accurate predctons. And F-measure means the combnatonal mean of CCR and RR, and ths s convenent expresson method to compare models. The calculaton of CCR and RR and F-measure can be wrtten as equaton (4.) usng the actual stuaton and predcton for the classfcaton n Table 6. A CCR : p = A+ B A RR : r = A+ C Fpr (, ) = pr p + r (4.) In order to compare the performance of the artcle were classfed usng the F-M value usng a CCR and RR, exhbted the model F-Measure specfc value correspondng to 0 tmes the Table 7. Table 6. The Classfcaton Table of Correctness n Predcton Model Predcton Table 7. Category Model Congruty Real Incongruty Congruty A B Incongruty C D The Mean and Stand Devaton n each Model Mean F-measure Standard Devaton Baysan k-nn Decson Tree SVM SVM model has the best F-measure value (59%) model showed the value compared to the other models n ths category. 5. Conclusons In ths paper, we converted the unstructured data to structured usng text mnng technques. And we compared those data by applyng the materal to the conventonal technques of Bayesan statstcal classfcaton, k-nn, decson tree, SVM to form a predcton model. After morphologcal analyss, we extracted only for the classfcaton word as a noun. By applyng a stopwords dctonary wth respect to the extracted to remove words that can cause nose n the analyss and the word-document matrx was produced usng the remanng words. By weghtng the matrx to ndcate a good correlaton between the word and document, t was converted nto a smple frequency matrx. Model Evaluaton crtera nclude the performance was compared usng a common nformaton classfcaton and rate of recall and F-measure that evaluaton crtera n the classfcaton document. It was also run for 0 teratons performed to ensure the valdty and generalzaton of the model buldng. The F-measure value of SVM model shows better performance than other models. After usng the standardzed Unstructured newspaper artcle n ths paper, applyng to each analyss were compared to the results. Overall, the performance was shown n the model s relatvely low, the reason s estmated to be lmtatons of the automatc document classfcaton algorthm. 5. Acknowledgement Ths work was supported by the Natonal Research Foundaton of Korea Grant funded by the Korean Government (NRF-04SA5B ). 6. References. Hearst Mart A. Untanglng Text Data Mnng. Proceedngs of the 37th annual meetng of the Assocaton for Computatonal Lngustcs on Computatonal Lngustcs. 999; 3(0). DOI: 0.35/ Kho Kwangsoo, Chung Wonkyo, Shn Youngkeun, Park Sangsung, Jang Dongsk. A Study on Development of Patent Informaton Retreval Usng Textmnng. Journal of the Korea Academa-Industral cooperaton Socety. 0; (8): Vol 9 (4) June 06 Indan Journal of Scence and Technology

7 Lee Junyeon, Shn Seungsoo and Km Jungju 3. Cho Taeho. Comparson of Neural Network and k-nn Algorthm for News Artcle Classfcaton. 998 Conference on Korea Informaton Scence Socety. 998; 5(): Bartere MM and Deshmukh PR. Cluster Orented Image Retreval Systems. IJCA Proceedngs on Emergng Trends n Computer Scence and Informaton Technology. 0; ETCSIT(3): Mttermayer M and Knolmayer G. Text Mnng Systems for Market Response to News: A survey, Workng paper n Insttut fur Wrtschaftsnformatk der Unverstat Bern. 006; 84: Chn KK. The Graduate School of the Unversty of Darwn College: Support Vector Machnes appled to speech pattern classfcaton. Dssertaton of PhD Cart, NC, USA: SAS Insttute Inc.: SAS Publshng. SAS Text Mner 4. Reference Van Drel MA, Bruggeman J, Vrend G, Brunner HG and Leunssen JA. A Text-Mnng Analyss of the Human Phenome. European Journal of Human Genetcs. 006; 4(50): Sebastan F. Machne learnng n automated text categorzaton. ACM Computng Surveys. 00; 34():-47. Vol 9 (4) June 06 Indan Journal of Scence and Technology 7