A Novel System for Document Classification Using Genetic Programming

Transcription

1 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber 2015 A Novel Syste for Docuent Classfcaton Usng Genetc Prograng Saad M. Darwsh, Adel A. EL-Zoghab, and Doaa B. Ebad Insttute of Graduate Studes and Researches, Alexandra Unversty, Egypt Eal: Saad.darwsh@alex-gsr.edu.eg, zoghab@gal.co, doaa_ebad@yahoo.co Abstract Wth the ncreasng avalablty of electronc docuents and the rapd growth of the World Wde Web, the task of autoatc categorzaton of docuents becae the key ethod for organzng the nforaton and knowledge dscovery. Docuent retreval, categorzaton, routng and flterng can all be forulated as classfcaton probles. The coplexty of natural languages and the extreely hgh densonalty of the feature space of docuents have ade ths classfcaton proble very dffcult. The proposed work tgates ths dffcult by provdng an algorth to classfy docuents nto ore than two categores (ult-class classfcaton) at the sae te by cobnng ult-objectve technque wth the genetc prograng of classfers based on ult-tree representaton of docuents. Ths cobnaton has the potental to attan lower errors because classfcaton accuracy on each class s represented as a dstnct objectve. Eprcal evaluatons show encouragng results and confr that the proposed algorth s feasble and effectve. Index Ters docuent classfcaton, prograng, ult-objectve technques, representaton I. genetc ult-tree INTRODUCTION Accordng to the growth n the aount of text docuents over the nternet and news sources whch ake docuent classfcaton s an portant task n docuent processng. Docuent classfcaton was wdely used n any contexts lke docuent ndexng, docuent analyss, docuent flterng, autoatc dstrbuton or archvng of docuents [1] [2]. Ths process dffcult to be anual wth huge nuber of docuents so autoatc classfcaton s better than anual classfcaton because t has ore accuracy and te effcency [3] [4]. Natural language processng, data nng, and achne learnng technques work together to autoatcally classfy docuents. Autoatc text classfcaton s the actvty of assgnng pre-defned category labels to natural language texts based on nforaton found n a tranng set of labeled docuents. A defnton of a docuent s that t s ade of a jont ebershp of ters whch have varous patterns of occurrence. Inforal, docuent classfcaton s to assgn a docuent dj to a category c by approxatng a functon :D C {T, F} by axzng Manuscrpt receved March 1, 2015; revsed August 1, do: /jat the concdence of a functon wth the actual categorzaton Ф, where D={d1,d2,,dn}s the set of docuents, n refers to the total nuber of docuents, C= {c1,c2,,c}s the set of classes, refers to the total nuber of predefned classes, T and F are boolean values for true and false respectvely [5] [6] [7]. Text classfcaton presents any challenges and dffcultes. Soe of the are [6]: achevng hgh accuracy n all applcaton contexts, dealng wth very large nubers of categores, approprate docuent representaton, densonalty reducton to handle algorthc ssues, and an approprate classfer functon to obtan a good generalzaton and avod over-fttng. Also, t s dffcult to capture hgh-level seantcs and abstract concepts of natural languages just fro a few key words. Thus to tackle these probles a nuber of ethods have been reported n the lterature for effectve text docuent classfcaton. Autoatc docuent classfcaton dvdes nto two types of approaches [1] [6] [8]: Rule based and Machne learnng. Rule based approach (knowledge engneerng approach) where a set of rules was defned anually that convert expert knowledge on how to classfy docuents under the gven set of categores. The advantages of ths approach are huan understandable, provdng accurate rules and easy to control when the nuber of rules s sall. The drawback was the hgh cost of huan power requred for defnng the rules set and antanng t. Machne learnng algorths autoatcally bulds a classfer by learnng the characterstcs of the categores fro a set of classfed docuents, and then uses the classfer to classfy docuents nto predefned categores. In ths case, a consderable savngs n ters of expert huan power and accuracy s coparable to that acheved by rule based approach. For these reasons, achne learnng based approach s replacng rule based approach for docuent classfcaton. However, these achne learnng ethods have soe drawbacks. In order to tran classfer, huan ust collect large nuber of tranng text ters, the process s very laborous. If the predefned categores changed, these ethods ust collect a new set of tranng text ters, so t s dffcult to prove the accuracy of these classfcaton ethods [1] [6]. The pre-defned docuents are the key resource n achne learnng approach. If they are not good enough to extract rules or equaton correctly, the result wll not

2 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber 2015 be satsfed. Accordng to the varety of the datasets whch consst of hundreds of thousands of docuents characterzed by a huge nuber of ters, both effcency and accuracy depend on classfcaton systes [4]. There are any achne learnng technques whch are used for docuent classfcaton[4] [9] [10] [11] such as Bayesan classfer, decson tree, K-nearest neghbor, support vector achnes, neural networks, genetc algorth, and genetc prograng (GP), etc. Although, ths nuber of achne learnng technques s used n the docuent classfcaton process, the process needs to prove the perforance because the ost of these technques suffer fro the coputaton te and accuracy. Besdes, cobnatons of ultple classfers dd not always prove the classfcaton accuracy copared to the best ndvdual classfer [3]. GP s a supervsed achne learnng technque and a powerful evolutonary algorth wdely used to evolve coputer progras autoatcally [11]. The prncple coponents of the GP are a set of functons and ternals that are able to represent the soluton of the proble. GP works by teratvely applyng genetc operators, such as crossover and utaton, to the ndvdual progras of the populaton usng a ftness easure. At the end of the GP run, the best ndvdual s presented as the soluton to the proble [8] [11] [12]. GP has ablty to dscover the underlyng data relatonshps and express the atheatcally [13] but GP classfers stll suffer fro several dffcultes, neffectve crossover and utaton, tranng te s long and error rates greater than those acheved by other classfcaton ethods on soe tasks. So to evolve GP classfers, ult-objectve technques are used [14]. Evolutonary ult-objectve (EMO) technques have been appled to genetc prograng for parsony enforceent n order to overcoe 'bloat', the tendency of genetc progras to develop unnecessary code or 'ntrons', and evolve progras for less coputatonal cost. Eltst Non-donated Sortng Genetc Algorth (NSGA-II) s one of these technques whch evolve classfers [15]. It has a better sortng algorth, ncorporates elts and no sharng paraeter needs to be chosen a pror. Mult-category classfcaton eans a classfcaton task wth ore than two classes. One of the ost popular technques used to deal wth ult-class classfcaton s dvdng the ult-class classfcaton nto ultple twoclass (bnary) classfcaton. The ost coon ways to decopose ult-class classfcaton to bnary classfcaton are one-vs.-all (OVA) and one-vs.-one (OVO). In OVA approach, by assung the nuber of classes s, bnary classfers are requred to dscrnate between one of the classes and all other classes. The learnng phase s done usng all tranng data, so ths approach needs hgh eory requreent. In OVO approach, (-1)/2 bnary classfers are requred to dscrnate between dfferent pars of classes; ths eans the nuber of classfers ncrease by ncreasng nuber of classes. The learnng phase s done by usng a subset fro the orgnal tranng data; the subset tranng data contan saples that belong to the two correspondng class labels [16][17]. The an contrbuton of ths research study s to propose a bo-nspred docuent classfcaton syste for that eploys ult-objectve GP wth ult-tree representaton [18]. The otvaton behnd ult-tree representaton concept s to well balance the exploraton and scalablty (nuber of classes) capablty of GP classfer for attanng better accuracy where each tree represents a classfer for a partcular class. By usng ult-tree representaton, GP can classfy all docuents fro several classes at the sae te and n one sngle run. Also, the odel cobnes GP wth NSGA-II to acheve ore accuracy, snce usng GP only do not gve the desred results. The paper s organzed as follows: Secton II brefly dscuses soe of the research related to docuent classfcaton. A bref overvew on the proposed docuent classfcaton syste s gven n secton III. The experental result and evaluaton of the proposed syste are reported n secton IV. Fnally n secton V, the concluson and future work are presented. II. LITERATURE REVIEW In ths secton, we wll cover the use of GP as a classfer n docuent classfcaton n the prevous works. For a coprehensve overvew of docuent classfcaton requreents, we refer to [2][9]. Several works have developed text classfcaton ethods usng evolutonary algorths, however, no studes can be found that uses evolutonary ult-objectve for ultclass classfcaton approaches. The authors n [19] used an approach used to segent age docuent and classfy the docuent regons as text, age, drawngs and table. Docuent age s dvded nto blocks usng run length searng rule and features are extracted fro every blocks. Dscpulus tool has been used to construct the genetc prograng based classfer odel. Usng GP to classfy unstructured docuents s started n [20]. The a of ths odel s to show that GP can be used to classfy a docuent based on the set of words whch presented n t. The agent s equpped wth a dctonary of words. Ths dctonary contans all words used n any of the docuents processed by the agent, and assgns to each word a nuber; ths nuber used to refer to that word. The ftness functon for the GP solutons s the nuber of docuents whch classfed ncorrectly. Ths odel was created as a two class proble (nterestng or unnterestng). The result showed that GP s a useful ethod for classfcaton n spte of the odel was needed to a large proveent. In [8], the authors eployed genetc prograng to create copact classfcaton rules usng cobnatons of N-Gras (sequences of N letters). Ths syste ncorporated the advantageous of achne learnng and rule based approaches to produce a cobnaton of both of the. Set of features of the syste ncludes word cobnatons and negatve nforaton (the features whch are not found n a specfc docuent) for dscrnaton purposes. The ftness of the GP ndvduals starts by evaluatng the rule produced by the 195

3 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber 2015 GP aganst all docuents n the tranng set, and then calculates F1-easure. The syste deals wth a large nuber of features wthout take n consderaton the frequency of the N-Gra pattern. Furtherore, t s very dffcult to decde the nuber of gras to be consdered for effectve docuent representaton. Here, the GP process ddn't depend on elts whch eans the best ndvduals don't keep to the next generaton, and the researchers suggested cobnng ths odel wth any other classfer (e.g. n [5]) to get a practcal value. In general, cobnatons of ultple classfers dd not always prove the classfcaton accuracy copared to the best ndvdual classfer. Prevous researches reveal that all of the benefts of GP are not attaned as expected n docuent classfcaton proble as there are any challenges that work as barrers n success of the of GP and ake t ore dffcult to acheve expected benefts. The prevous works deal wth ult-class docuent classfcaton as any bnary classfcaton probles. Ang to fll ths gap n lterature, ths paper proposes a ult-objectve GP algorth for ult-class unstructured docuent classfcaton. III. PROPOSED DOCUMENT CLASSIFICATION SYSTEM Fgure 1. The proposed docuent classfcaton odel. The dea developed n ths paper s partally nspred by exstng work n [18] that explots GP wth ult-tree representaton concept to deal wth all classes at the sae te nstead of convertng ult-class classfcaton nto ult-two class classfcatons. To avod large tree sze of the classfer, the proposed syste adds NSGA-II ult objectve technque n the learnng phase to prove the accuracy and reduce learnng te. Cobnng ultobjectve technques wth the genetc prograng of classfers presents opportuntes for evolvng saller classfers that should generalze better whle searchng the soluton space ore effectvely. Ths would lead to classfers that attan lower errors and are evolved for less coputatonal cost than would be the case usng the standard GP algorth. Fg. 1 shows the an syste coponents and how they are lnked to each other. The syste deals wth unstructured docuents wth no layout characterstcs and uses a predefned set of docuents n tranng phase (supervsed learnng technque), and the results are then tested on the testng data. The learnng phase s done usng all tranng data. Our ult-tree classfer that contans trees s evolved n the sae te and n a sngle run of GP, where s the nuber of classes. Each odel's stage s explaned n the followng subsectons. Tokenzaton: a docuent s treated as a strng, and then parttoned nto a lst of tokens (words). Reovng of stop-words: words such as a, an, the, etc that occur coonly n all docuents are reoved. Steng: the root of the words s transfored nto an orgnal for, e.g. connecton to connect, coputng to copute. There are any rules that control the steng step as llustrated n [22]. At the end of the preprocessng step, the ters are prepared and saved n ters lst. Ths lst s used n the next step to create ter frequency atrx for docuents. A. Pre-Processng The frst step s to transfor docuents nto clear word forat. The docuents are represented by a great aount of features (ters, words). Each docuent s seen as a set of words wth no utual relatons between the words. Ths set of words s used to classfy docuent. Before startng n pre-processng step, all words are converted to lower case. The preprocessng steps are [21]: 196 B. Indexng A text docuent s typcally represented as a vector of ter weghts (word features) fro a set of ters (dctonary), where each ter occurs at least once n a certan nu nuber of docuent. In our case, we gnore the structure of a docuent and the order of words n the docuent. The feature vectors represent the words observed n the docuents. The choce of docuent representaton has a profound pact on the qualty of the classfer. So, we use ter frequency nverse docuent frequency (TF-IDF) weghtng ethod for docuent representaton [1]. TF-IDF approach s coonly used to weght each word n the text docuent accordng to how unque t s. In other words, the TF-IDF approach captures the relevancy aong words, text docuents and a partcular category to select nforatve features. TFIDF s defned as [23]: n wj tfj log 1 df j (1)

4 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber 2015 where wj s the weght of word n the docuent dj, tfj codes ter frequency that easures how frequently a ter t occurs n a docuent dj, dfj s nverse docuent frequency that easures how portant a ter s, n characterzes the total nuber of docuents and dfj s the docuent frequency that easures the nuber of docuents n whch t occurs. As a result of ths step, the followng docuent-ter atrx D s appeared usng TF-IDF weghtng ethod: w21 w T 1 w22 w T 2 w2n w T n w11 w12 D w1n feature ndces pass to test phase to reove unwanted features fro the test docuent-ter atrx. D. Denson Reducton Densonalty reducton s a very portant step n text classfcaton, because rrelevant and redundant features often degrade the perforance of classfcaton algorths both n speed and classfcaton accuracy and also ts tendency to reduce over fttng. The proposed syste utlzes Lnear Dscrnant Analyss (LDA) as a well-known ethod for denson reducton of atrx X [25]. Before applyng LDA, tranng data vectors are noralzed wth zero-ean. LDA starts by fndng egenvectors of Sw-1Sb, where Sb s the between-class varance and Sw s the wthn-class varance. LDA tres to copute a transforaton that axzes the rato of Sb to Sw. Egenvectors of Sw-1Sb for the transforaton atrx W n whch Y=W X [26]. W s the transforaton atrx, and Y s the new copact atrx that represents the old atrx X. It eans that each new feature s a cobnaton of the orgnal features. To apply the sae step for test data, we need to save egenvectors and ean of all classes; we refer to that by appng. After applyng ths stage, appng passes to test phase to reduce the denson of the test saples atrx. (2) where T denotes the length of ters vector. Each colun represents a specfc word and each row represents specfc docuent [21]. After applyng ths stage on the tranng docuents, the ters lst passes to the test phase to buld the docuent-ter atrx for the test docuents. C. Features Selecton Docuent classfcaton s often characterzed by the hgh densonalty feature space and a relatvely sall nuber of tranng saples. The nuber of potental features often exceeds the nuber of tranng docuents. Feature selecton s used to select a subset of features fro the orgnal docuents. FS s perfored by keepng the words wth hghest score accordng to the predeterned easure of the portance of the word [21]. Inforaton gan (IG) ethod s one of feature selecton ethods that easures the aount of nforaton obtaned for category predcton by knowng the presence or absence of a ter n a docuent. The nforaton gan of ter tj, j=1,2, T s defned as [24]: IG( t j ) P( c t P( c ) log P( c ) P( t j ) 1 1 j E. Mult-Objectve Genetc Prograng The a of ths stage s to buld a classfer that can classfy the test data autoatcally. A classfer s a appng, D : Rp N where Rp s the p-densonal real space and N s the set of label vectors for -class proble and s defned as [18]: N y R : y 0,1, P( c t j )log P( c t j ), (3) )log P( c t j ), 1 where c represents the th category, P(c) s the probablty of the th category, P(tj) and P( t j ) are the probabltes that the ter tj appears or not n the docuents, respectvely, P( c t j ) s the condtonal probablty of the th category gven that ter tj appeared, and P( c t j ) s the condtonal probablty of the th category gven that ter tj does not appeared [23]. After fnshng ths loop, each ter n the ters set has ts nforaton gan value.ters whose nforaton gan s less than soe predeterned threshold are reoved fro the feature space [24]. In other words, the docuent-ter atrx wll be sparse after ths step by reovng the ters whch have nforaton gan value less than the predeterned threshold. We assue that D converts to X. After applyng ths stage, the selected y 1 1 (4) For any vector (tranng data) x Rp, D(x) s a vector n -denson wth only one coponent as 1 and all others as 0 where s the nuber of classes. In ths paper, our objectve s to fnd a D usng the cobnaton between GP and NSGA-II. We shall use a ult-tree concept for desgnng classfer. The beauty of usng ths concept s that we can get a classfer for the ult-class proble n a sngle run of GP. In general, the applcaton of GP to bnary classfcaton probles has been deonstrated wth reasonable success. In ult-objectve optzaton, the goodness of a soluton s deterned by the donance [27]. Let x and y are two solutons, x donates y, f and only f, x s less than or equal y n all objectves and x1 s strctly better than x2 for least one objectve. A set of all the solutons that are not donated by any other soluton s called the non-donated soluton set. In NSGA-II, rankng the ndvduals s based on non-donated-fronts. The selecton n NSGA-II s done usng tournaent selecton ethod; a soluton fro the lowest front s selected, f there are ore than one ndvdual, the ndvdual wth a hgher crowdng dstance (easure of how close an ndvdual s to ts neghbors) s selected [27]. Regardng the ntalzaton paraeters of GP; ternal set Ƭ={feature varables, R}, where R contans rando P( t j ) 197

5 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber 2015 generated real nubers n rang [0.0 10] and functon sets Ƒ= {+,-,/,*}. The trees are ntalzed by usng the raped half and half ethod. Ths ethod s the ost coon way of creatng the ntal GP populatons that produces dverse trees [12]. Indvduals or chroosoes are represented by usng ult-tree concept where each ndvdual conssts fro trees. Each tree represents a classfer for a specfc class [18]. The ftness functon deternes how an ndvdual s able to classfy the docuents. In our approach, we apply the ult-objectve GP to prove the classfers that suffer fro large tree sze. Here, ftness objectves are: (1) classfcaton accuracy that equals the nuber of correctly classfed docuents to the total nuber of docuents durng tranng phase. (2) Tree coplexty that s calculated by countng the nuber of the nodes n the tree. Each ndvdual s evaluated aganst these two ftness objectves. In our case we need axu accuracy wth lowest tree sze. All objectves are of the nzaton type, t eans a nzaton type objectve can be converted to a axzaton type by ultplyng negatve one [27]. Tournaent selecton uses to select ndvdual for crossover or utaton. It generates offsprng wth crossover and utaton and selects the next generaton accordng to non-donated sortng and crowdng dstance coparson [15]. After tournaent selecton, we wll use roulette wheel selecton to select tree fro the ndvdual, whch s selected by tournaent selecton to apply crossover and utaton depend on the probablty of unft tree that used to gve ore change to the trees wth low perforance. Here, we odfed the crossover and utaton to be approprate wth ultrepresentaton as clarfed n the followng steps where K s the nuber of NSGA-II fronts and g nuber of generaton: Step 1: Create a rando ntal populaton P0 of sze N. Step 2: P0 s sorted based on the non-donaton soluton. Ftness s assgned to each soluton accordng to ts non-donaton level. Set g = 0. Step 3: Apply tournaent selecton to select parents fro P0 for crossover, and utaton to create offsprng populaton Q0 of sze N. Step 4: If one of the stoppng condton s occurred, stop and return Pg. Step 5: Cobne parent Pg and offsprng Qg populatons, Set Rg = Pg Qg, where Rg s of sze 2N. Step 6: Rg s sorted accordng to non-donaton level, dentfy the non-donated fronts F1,..,Fk n Rg. Step 7: For = 1,, k do followng: - Calculate crowdng dstance of the solutons n F. - Create Pg+1 Case 1: If Pg+1 + F N, then Pg+1= Pg+1 F; Case 2: If Pg+1 + F > N, then Pg+1= Pg+1 (N - Pg+1 ) F; Step 8: Use crowded tournaent selecton based to select parents fro Pg+1. Apply crossover and utaton to Pg+1 to create offsprng populaton Qg+1 of sze N. Step 9: Set g = g + 1, and go to Step The ternaton condton deternes when GP process s stopped. Here we set two stoppng condton, f all tranng saples Ntr are classfed correctly or after predefned nuber of generaton M. F. Post Preprocessng The a of ths stage s to enhance resultant classfer f t could not classfy all tranng saples correctly. In text classfcaton, a text docuent ay partally atch any categores. A classfer ay assgn a docuent to ultple classes or sngle class dependng on the constrants. We need to fnd the best atchng category for the text docuent. 1) Iprovng usng OR-ng technque After gettng the best classfer (BCF), t s possble that the ternal GP populaton contans two chroosoes (ndvduals) one of the s good for a partcular segent of the feature space, whle another one s good for another segent of the feature space. To get beneft fro that snce the overall perforance, n ters of sclassfcaton of the two ndvduals could be coparable or dfferent. Therefore, cobnng the two ndvduals by OR-ng ay result n a uch better classfcaton tree as coprehensvely explaned n [18]. The cobned classfer whch correctly classfed a hgher nuber fro the tranng saples s taken as a resultant classfer CF. Usng ths ethod ay ncrease the nuber of correctly classfed saples and ay not whch eans the best ndvdual through the GP process s the best at all. 2) Conflct resoluton It can be happened that ore than one tree of CF show postve responses. In ths case, we face a conflctng stuaton. To solve that, we use a set of heurstc rules followed by a weghtng schee as eployed n [18]. The a of the heurstc rules s to dentfy the typcal stuatons when the classfer cannot ake unabguous decsons and utlze that nforaton to ake decsons; the decson here eans to assgn class label to a docuent n the conflct cases. May be heurstc rules can't solve conflct stuaton f there s no heurstc rule at all. So, a weghtng schee s used. The objectve of ths atrx s to calculate the false postve cases and the false negatve cases whch wll use to enhance the classfer CF behavor accordng to the; see [18] for ore detals. It s possble that nether the weght-based schee nor the heurstc rule s able to assgn a class label to the conflcted saples. Note that the OR-ng operaton, the heurstc rule generaton, and coputaton of the weghts are done only once after the GP ternates. Consequently, the addtonal coputaton nvolved for the post-processng, as we shall see later, s uch less than the te requred for the GP to evolve. IV. EXPERIMENTAL RESULTS To evaluate the perforance of the proposed odel, we conduct a set of experents to easure ts effcency and copare t wth standard GP classfer and prevous work n [8]. Our experents were pleented usng MatLab

6 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber b. All the experents were based on a PC wth Wndows 7 Ultate 64-bt, wth Intel (R) Core(TM) 3 CPU, 2.53GHz and 4GB RAM. Experents were perfored usng the Reuters dataset ( 78.htl). We used the top 6 categores and dscarded docuents wth no label or wth ultple labels. The dstrbuton of docuents nto the categores s unbalanced n ths dataset. Furtherore, 20 Newsgroups dataset ( 20newsgroups.htl) s used also to show the ablty of the syste to deal wth balanced dataset (roughly equal nuber of docuents nsde each category). To evaluate the perforance of the proposed classfer, cro-average and acro-average F-easure are coonly used [28], where TP s the nuber of correctly classfed docuents for a class, FP s the nuber of docuents wrongly classfed for a class, and FN s the nuber of docuents whch belong to class but not assgned to t. Here, preparng the tranng docuents affect draatcally n the classfcaton result. 2 * * F( acro avg ) 1, 1TP TP FN 1 TABLE I. F( cro avg ), Class Earn Acq Crude Trade Money-fx Interest Mcro_ average Macro_ average Average-GP run te (n) Average-best ndvdual node count 2* * (5) 1TP TP FP 1 Hrsch et al.[8] (6) (7) Proposed syste F-Measure_ average around n/class RESULTS ON 20 NEWSGROUPS- TOP (AVERAGE) Class cop.graphcs cop.os.s-wndows.sc cop.sys.b.pc.hardware cop.sys.ac.hardware cop.wndows.x sc.forsale Mcro_ average Many experents were done to deterne the best paraeters values n each proposed syste's step to acheve the best classfcaton perforance. In the experents, the best LDA denson to reduce the data s -1. For Reuter's dataset, the best nforaton gan threshold s 0.2 and for 20news-group dataset s 0.1. We tran the classfer wth 90% fro the dataset and the rest for testng. Ths large percentage for tranng the classfer s used because the proposed odel classfes docuents for all classes at the sae te. Furtherore, ncreasng nuber of generatons ore than 30 do not affect on the result of the classfer and also ncreasng populaton sze for ore than 250 do not nfluence on the process; t takes ore te only wthout ncreasng the classfcaton accuracy. Snce each word n a docuent s a key to classfy t, tokenzng the docuent to a set of sngle (gra) words s ore effcent than usng b-gra tokenzaton, ths gves ore accuracy. In Table I and Table II, the results show that cobnng genetc prograng wth NSGA-II proves the classfcaton perforance rather than usng GP only. Ths proveent n the perforance affects on GP run te. The reason behnd that s the coplexty of the NSGA-II whch s O(BN2) where B s the nuber of objectve and N s the populaton sze. We run the syste 10 tes and calculate the average for each coparson crtera. Our proposed odel deals wth all classes at the sae te; to exane the classfcaton result per RESULTS ON REUTERS-TOP (AVERAGE) Standard GP TABLE II. TP, TP TP FN TP FP category aganst bnary classfcaton result, Table II shows that there s provng n the classfcaton results rather than bnary classfcaton results for the odel n [8]. Standard Proposed syste GP F-Measure_Average Macro_ average Average -GP run te (n) Average best ndvdual node count V. CONCLUSION AND FUTURE WORK Ths paper proposes a docuent classfcaton syste based on ult-objectve genetc prograng as a supervsed learnng technque. The syste deals wth ult-class docuent classfcaton wthout dvdng t to bnary classfcatons; unlke the ajorty of others classfcaton technques. Ths capablty s resultng fro eployng ult-tree representaton to represent each ndvdual. The syste acheves a good perforance copared to standard genetc prograng. To acheve a reasonable perforance, the tranng data ust be balanced or closer to balance. In the future, we suggest usng ncreental learnng to prove classfcaton results and test the syste on dfferent datasets. Also, we plan to test the syste's behavor when the nuber of classes s ore than 6. REFERENCES [1] [2] 199 F. Sebastan, Machne learnng n autoated text categorzaton, ACM Coputng Surveys (CSUR), vol. 34, no. 1, pp. 1-47, N. Chen and D. Blosten, A survey of docuent age classfcaton: Proble stateent, classfer archtecture and

7 Journal of Advances n Inforaton Technology Vol. 6, No. 4, Noveber 2015 [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] perforance evaluaton, Internatonal Journal of Docuent Analyss and Recognton (IJDAR), vol. 10, no. 1, pp. 1-16, N. VasfSs and M. R. F. Derakhsh, Text classfcaton wth achne learnng algorths, Journal of Basc and Appled Scentfc Research, vol. 3, no. 1, pp , Ndh and V. Gupta, Recent trends n text classfcaton technques, Internatonal Journal of Coputer Applcatons, vol. 35, no. 6, pp , S. Raasundara and S. Vctor, Algorths for text categorzaton: A coparatve study, World Appled Scences Journal, vol. 22, no. 9, pp , P.Y. Pawar and S. Gawande, A coparatve study on dfferent types of approaches to text categorzaton, Internatonal Journal of Machne Learnng and Coputng, vol. 2, no. 4, pp , M. Ikonoaks, S. Kotsants, and V. Tapakas, Text classfcaton usng achne learnng technques, WSEAS Transactons on Coputers, vol. 4, no. 8, pp , L. Hrsch, M. Saeed, and R. Hrsch, Evolvng text classfcaton rules wth genetc prograng, Internatonal Journal of Appled Artfcal Intellgence, vol. 19, no. 7, pp , Bhuka, S. S. Sehra, and A. Nayyar, A revew paper on algorths used for text classfcaton, Internatonal Journal of Applcaton or Innovaton n Engneerng & Manageent, vol. 2, no. 3, pp , Mar B. Baharudn, L. H. Lee, and K. Khan, A revew of achne learnng algorths for text-docuents classfcaton, Journal of Advances n Inforaton Technology, vol. 1, no. 1, pp. 4-20, D. Kuar and S. Benwal, Genetc algorth and prograng based classfcaton: A survey, Journal of Theoretcal and Appled Inforaton Technology, vol. 54, no. 1, Aug. 2013, pp H. Jabeen and A. R. Bag, Revew of classfcaton usng genetc prograng, Internatonal Journal of Engneerng Scence and Technology, vol. 2, no. 2, pp , J. Kshore, L. M. Patnak, V. Man, and V. Agrawal, Applcaton of genetc prograng for ultcategory pattern classfcaton, IEEE Transactons on Evolutonary Coputaton, vol. 4, no. 3, pp , D. Parrott, X. L, and V. Ceselsk, Mult-objectve technques n genetc prograng for evolvng classfers, n Proc. the IEEE Congress on Evolutonary Coputaton, USA, Sep. 2-5, 2005, pp K. Deb, S. Agrawal, A. Pratap, and T. Meyarvan, A fast eltst non-donated sortng genetc algorth for ult-objectve optzaton: NSGA-II, IEEE Transactons on Evolutonary Coputaton, vol. 6, no. 2, pp , Apr N. Mehra and S. Gupta, Survey on ultclass classfcaton ethods, Internatonal Journal of Coputer Scence and Inforaton Technologes, vol. 4, no. 4, pp , M. Galar, A. Fernandez, E. Barrenechea, H. Bustnce, and F. Herrera, An overvew of enseble ethods for bnary classfers n ult-class probles: Experental study on one-vs-one and one-vs-all Schees, Pattern Recognton Journal, vol. 44, no. 8, Aug. 2011, pp D. P. Mun, N. R. Pal, and J. Das, A novel approach to desgn classfers usng genetc prograng, IEEE Transactons on Evolutonary Coputaton, vol. 8, no. 2, pp , Apr N. Pryadharshn and V. MS, Genetc prograng for docuent segentaton and regon classfcaton usng dscpulus, Internatonal Journal of Advanced Research n Artfcal Intellgence, vol. 2, no. 2, pp , B. Svngen, Usng genetc prograng for docuent classfcaton, n Proc. 11th Internatonal Florda Artfical Intellgence Research (FLAIRS), USA, May 1998, pp V. Korde and C. N. Mahender, Text classfcaton and classfers: A survey, Internatonal Journal of Artfcal Intellgence & Applcatons, vol. 3, no. 2, pp , Mar [22] A. G. Jvan, A coparatve study of steng algorths, Internatonal Journal of Coputer Technology and Applcatons, vol. 2, no. 6, pp , Nov-Dec [23] H. Uguz, A two-stage feature selecton ethod for text categorzaton by usng nforaton gan, prncpal coponent analyss and genetc algorth, Knowledge-Based Systes, vol. 24, no. 7, pp , [24] Y. Xu, G. J. Jones, J. L, B. Wang, and C. Sun, A study on utual nforaton-based feature selecton for text categorzaton, Journal of Coputatonal Inforaton Systes, vol. 3, no. 3, pp , [25] K. Torkkola, Dscrnatve features for text docuent classfcaton, Journal of Foral Pattern Analyss & Applcatons, vol. 6, no. 4, pp , [26] G. Brck, B. Dr, and A. C. Sonez, Abstract feature extracton for text classfcaton, Turksh Journal of Electrcal Engneerng & Coputer Scences, vol. 20, no. 1, pp , [27] A. Konak, D. W. Cot, and A. E. Sth, Mult-objectve optzaton usng genetc algorths: A tutoral, Internatonal Journal of Relablty Engneerng & Syste Safety, vol. 91, no. 9, pp , [28] A. Ozgur, L. Ozgur, and T. Gungor, Text categorzaton wth class-based and Corpus-based keyword selecton, Lecture Notes n Coputer Scence, vol. 3733, pp , Adel A. El-Zoghab receved hs B.Sc. n coputer engneerng fro Alexandra Unversty n 1987, hs M.Sc. and Ph.D. n nforaton technology fro Alexandra Unversty and Old Donon Unversty n 1991 & 1994 respectvely. Hs research and professonal nterests nclude ntellgent systes and achne learnng, nter networkng & routng protocols, and dstrbuted systes. He has publshed any papers n nternatonal journals and conferences worldwde durng the past three decades. Currently, he s a professor of coputer scence and IT and the head of Dept. of Inforaton Technology snce August Saad M. Darwsh receved hs Ph.D. degree fro the Alexandra Unversty, Egypt. Hs research work concentrates on the feld of age processng, optzaton technques, securty technologes, coputer vson, pattern recognton and achne learnng. He has publshed n journals and conferences and severed as TPC of any nternatonal conferences. Snce Feb. 2012, he has been an Assocate Professor n the departent of nforaton technology, Insttute of Graduate Studes and Research, Egypt. Doaa B. Ebad receved the B.Sc. degree n coputer scence and statstc fro the Faculty of Scence, Alexandra Unversty, Egypt n 2011, she s dong the M.Sc. degree n the Insttute of Graduate Studes and Research (IGSR), Departent of Inforaton Technology, Unversty of Alexandra, she has been a Deonstrator n the departent of nforaton technology, IGSR. Her research and professonal nterests nclude Machne learnng, Mult-objectve technques.