Feature Selection for Natural Language Call Routing Based on Self-Adaptive Genetic Algorithm

Transcription

1 IOP Conference Seres: Materals Scence and Engneerng PAPER OPEN ACCESS Feature Selecton for Natural Language Call Routng Based on Self-Adaptve Genetc Algorthm To cte ths artcle: A Koromyslova et al 017 IOP Conf. Ser.: Mater. Sc. Eng Vew the artcle onlne for updates and enhancements. Related content - Dscrmnatve feature selecton for vsual trackng Junka Ma, Habo Luo, We Zhou et al. - Sentment analyss of feature rankng methods for classfcaton accuracy Shashank Joseph, Calvn Mugaur and S Sumathy - A new Self-Adaptve dspatchng System for local clusters Bowen Kan, Jngyan Sh and Xaofeng Le Recent ctatons - Context Analyss of Customer Requests usng a Hybrd Adaptve Neuro Fuzzy Inference System and Hdden Markov Models n the Natural Language Call Routng Problem Samr Rustamov et al Ths content was downloaded from IP address on 30/06/018 at 1:1

2 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 Internatonal Conference on Recent Trends n Physcs 016 (ICRTP016) Journal of Physcs: Conference Seres 755 (016) IOP Publshng do: / /755/1/ Feature Selecton for Natural Language Call Routng Based on Self-Adaptve Genetc Algorthm A Koromyslova 1, M Semenkna 1, R Sergenko 1Reshetnev Sberan State Aerospace Unversty, Krasnoyarsky rabochy avenue 31, Krasnoyarsk, Russa, Ulm Unversty, Lse-Metner-Str. 9, Ulm, Germany, E-mal: akoromyslova@mal.ru, semenkna88@mal.ru, roman.sergenko@unulm.de Abstract: The text classfcaton problem for natural language call routng was consdered n the paper. Seven dfferent term weghtng methods were appled. As dmensonalty reducton methods, the feature selecton based on self-adaptve GA s consdered. k-nn, lnear SVM and ANN were used as classfcaton algorthms. The tasks of the research are the followng: perform research of text classfcaton for natural language call routng wth dfferent term weghtng methods and classfcaton algorthms and nvestgate the feature selecton method based on self-adaptve GA. The numercal results showed that the most effectve term weghtng s TRR. The most effectve classfcaton algorthm s ANN. Feature selecton wth self-adaptve GA provdes mprovement of classfcaton effectveness and sgnfcant dmensonalty reducton wth all term weghtng methods and wth all classfcaton algorthms. 1. Introducton Natural language call routng s an mportant problem n the desgn of modern automatc call servces and the solvng of ths problem could lead to mprovement of the call servce [1]. Generally natural language call routng can be consdered as two dfferent problems. The frst one s speech recognton of calls and the second one s topc categorzaton of users utterances for further routng. Topc categorzaton of users utterances can be also useful for multdoman spoken dalogue system desgn [1]. In ths work we treat call routng as an example of a text classfcaton applcaton. In the vector space model [16] text classfcaton s consdered as a machne learnng problem. The complexty of text categorzaton wth a vector space model s compounded by the need to extract the numercal data from text nformaton before applyng machne learnng algorthms. Therefore, text classfcaton conssts of two parts: text preprocessng and classfcaton algorthm applcaton usng the obtaned numercal data. Text preprocessng comprses three stages: - Textual feature extracton. - Term weghtng - Dmensonalty reducton. The frst one s the textual feature extracton based on raw preprocessng of the documents. Ths process ncludes deletng punctuaton, transformng captal letters to lowercase, and addtonal procedures such as stop-words flterng [4] and stemmng [14]. Stop-words lst contans pronouns, prepostons, artcles and other words that usually have no mportance for the classfcaton. Usng stemmng t s possble to on dfferent forms of the same word nto one textual feature. The second stage s the numercal feature extracton based on term weghtng. For term weghtng we use bag-of-words model, n whch the word order s gnored. There exst dfferent unsupervsed and supervsed term weghtng methods. The most well-known unsupervsed term weghtng method s TFIDF [15]. The followng supervsed term weghtng methods are also consdered n the paper: Content from ths work may be used under the terms of the Creatve Commons Attrbuton 3.0 lcence. Any further dstrbuton of ths work must mantan attrbuton to the author(s) and the ttle of the work, ournal ctaton and DOI. Publshed under lcence by IOP Publshng Ltd 1

3 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 Gan Rato (GR) [3], Confdent Weghts (CW) [10], Term Second Moment (TM) [], Relevance Frequency (RF) [11], Term Relevance Rato (TRR) [9], and Novel Term Weghtng (NTW) [18]; these methods nvolve nformaton about the classes of the documents. As a rule, the dmensonalty for text classfcaton problems s hgh even after stop-words flterng and stemmng. Due to the hgh dmensonalty, the classfcaton may be napproprate tmeconsumng, especally for real-tme systems such as natural language call routng. Therefore, the next stage of preprocessng s the dmensonalty reducton based on numercal features; t s possble wth feature selecton or feature transformaton. In our research we use a feature selecton method based on genetc algorthm or GA-based wrapper. One of the most complcated problems wth GA applcatons s settng algorthm parameters. A conventonal genetc algorthm has at least three methods of selecton (proportonal, tournament, and rank), three methods of recombnaton (one-pont, two-pont, and unform). Mutaton probablty requres tunng as well. The amount of varous combnatons can be estmated at tens. Exhaustve search of combnatons requres a lot of tme and computatonal power, especally for such tmeconsumng problems as GA applcatons for machne learnng. Parameters combnaton selecton at random can be also nsuffcent as algorthm effcency on same problem can dffer very much for varous parameters settng. Ths problem can be solved wth self-confgurng GA [17] or coevolutonary GA [19]. Therefore, we propose a use of self- adaptve GA for the feature selecton n the feld of natural language call routng. As classfcaton algorthms we use the k-nn algorthm, the lnear SVM, and artfcal neural networks (ANN). Some comparatve studes of machne learnng algorthms n the feld of text classfcaton showed hgh classfcaton effectveness of k-nn, SVM-based algorthms, and ANN [, 7, 8, 10, 13]. The tasks of our research are the followng: - Perform research of text classfcaton for natural language call routng wth dfferent term weghtng methods and classfcaton algorthms. - Investgate the feature selecton method based on self-adaptve GA. The paper s organzed as follows: In Secton, we descrbe the consdered corpus for natural language call routng. Secton 3 descrbes the consdered term weghtng methods. The GA-based feature selecton s descrbed n Secton 4. The self- adaptve GA are descrbed n Secton 5. Secton 6 contans short descrpton of classfcaton algorthms. The results of numercal experments are presented n Secton 7. Fnally, we provde concludng remarks n Secton 8.. Corpus descrpton The data for testng and evaluaton conssts of 9,156 user utterances recorded n Englsh language from caller nteractons wth commercal automated agents. Utterances are short and contan only one phrase for further routng. The database contans calls n textual format after speech recognton. The database s provded by the company Speech Cycle (New York, USA). Utterances from ths database are manually labelled by experts and dvded nto 0 classes (such as appontments, operator, bll, nternet, phone and techncal support). One of them s a specal class TE-NOMATCH whch ncludes utterances that cannot be put nto another class or can be put nto more than one class. The database contans 45 unclassfed calls and they were removed. The database contans also 3,561 empty calls wthout any words. These calls were placed n the class TE-NOMATCH automatcally and they were also removed from the database. As a rule, the calls are short n the database; many of them contan only one or two words. The average length of an utterance s 4.66 words, the maxmal length s 19 words. There are a lot of dentcal utterances n the database; the corpus contans only 4,458 unque non-empty classfed calls. The corpus s unbalanced. The largest class contans 7.05% and the smallest one contans 0.04% of the unque calls. Due to the very hgh frequency of a small number of utterances n the corpus, we formulate two dfferent problem defntons.

4 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 Problem defnton 1. The whole database wth 68,550 classfed non-empty calls s used for tranng and test sets formng. Numbers of repettons of the utterances n tranng and test sets are used as weghts for classfcaton. Ths problem defnton s the closest to the real stuaton but frequently repeated utterances decrease dfference between preprocessng and classfcaton methods. Addtonally, there are some dentcal utterances n tranng and test sets smultaneously. In ths case the over-fttng problem of classfcaton may be hdden. Therefore, ths problem defnton s not very approprate for the comparatve study. Problem defnton. Before tranng and test samples formng, all utterance duplcates were removed from the database. It means that there s no ntersecton between tranng and test sets and frequency of utterances s gnored. Therefore, the problem defnton 1 s sutable for the qualty estmaton of the real natural language call routng system; the problem defnton s the most approprate for the comparatve study of dfferent preprocessng and classfcaton methods. For statstcal analyss we performed 0 dfferent dvsons of the database nto tranng and test samples randomly. Ths procedure was performed for two problem defntons separately. The tran samples contan 90% of the calls and the test samples contan 10% of the calls. For each tranng sample we have desgned a dctonary of unque words whch appear n the tranng sample after deletng punctuaton and transformng captal letters to lowercase. The sze of the dctonary vares from 3,75 to 3,39 words for problem defnton 1 and from 3,77 to 3,311 for problem defnton. 3. Term weghtng methods As a rule, term weghtng s a multplcaton of two parts: the part based on the term frequency n a document (TF) and the part based on the term frequency n the whole tranng database. The TF-part s fxed for all consdered term weghtng methods and s calculated as followng: TF log n tf 1 ; tf, where n s the number of tmes the th word occurs n the th document, N s the document sze (number of words n the document). The second part of the term weghtng s calculated once for each word from the dctonary and does not depend on an utterance for classfcaton. We consder seven dfferent methods for the calculaton of the second part of term weghtng Inverse Document Frequency (IDF) IDF s a well-known unsupervsed term weghtng method whch was proposed n [15]. There are some modfcatons of IDF and we use the most popular one: N d f D log, n where D s the number of documents n the tranng set and n s the number of documents that have the th word. 3.. Gan Rato (GR) Gan Rato (GR) s manly used n term selecton [4], but n [3] t was shown that t could also be used for weghtng terms. The defnton of GR s as follows: 3

5 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 GR t, M t c c c c t cc,, c Pt, c c t c,, t P M t, c logp P t, c log P c t Pc, where P(t, c) s the relatve frequency that a document contans the term t and belongs to the category c; P(t)s the relatve frequency that a document contans the term t and P(c) s the relatve frequency that a document belongs to category c. Then, the weght of the term t s the max value between all categores as follows: where C s a set of all classes. GR t maxgrt, c, c C 3.3. Confdent Weghts (CW) Ths supervsed term weghtng approach has been proposed n [0]. Frstly, the proporton of documents contanng term t s defned as the Wlson proporton estmate p(x, n) by the followng equaton: p x, n, x 0.5Z n Z where x s the number of documents contanng the term t n the gven corpus, n s the number of documents n the corpus and Z, where the t-dstrbuton (Students law) when n < 30 and the normal dstrbuton when n 30. In ths work α=0.95 and 0.5Z (as recommended by the authors of the method). For each term t and each class c two functons p pos (x, n) and p neg (x, n) are calculated. For p pos (x, n) x s the number of documents whch belong to the class c and have term t; n s the number of documents whch belong to the class c. For p neg (x, n) x s the number of documents whch have the term t but do not belong to the class c; n s the number of documents whch do not belong to the class c. The confdence nterval (p -, p + ) at 0.95 s calculated usng the followng equaton: р M 0.5Z, р 1- р n Z р М; р р М. The strength of the term t n the category c s defned as the follows:, str t, _ ppos _ log, f p с _ ppos pneg 0, otherwse. pos > p neg, 4

6 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 The maxmum strength (Maxstr) of the term t s calculated as follows: Maxstr (t ) max str (t, c c C 3.4. Term Second Moment (TM) Ths supervsed term weghtng method was proposed n []. Let P(c t) be the emprcal estmaton of the probablty that a document belongs to the category c wth the condton that the document contans the term t; P(c ) s the emprcal estmaton of the probablty that a document belongs to the category c wthout any condtons. The dea s the followng: the more P(c t) s dfferent from P(c ), the more mportant the term t s. Therefore, we can calculate the term weght as the followng: where C s a set of all classes. TM(t ) = С 1 ) (P(c t) - P(c )), 3.5. Relevance Frequency (RF) The RF term weghtng method was proposed n [11] and s calculated as the followng: rf(t ) = max rf (t, c ), с С rf (t, c ) log а max1,. а, where a s the number of documents of the category c whch contan the term t and of documents of all the other categores whch also contan ths term Term Relevance Rato (TRR) The TRR method [9] uses tf weghts and t s calculated as the followng: TRR t log P t P t, c P TRR t k k 1 t c, V Tc tf Tc tf lk l1k 1 c c, maxtrrt, c, c C а s the number where c s a class of the document, c s all of the other classes of c, V s the vocabulary of the tranng data and T c s the document set of the class c. 5

7 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/ Novel Term Weghtng (NTW) Ths method was proposed n [1, 18]. The detals of the procedure are the followng. Let L be the number of classes; n s the number of documents whch belong to the th class; N s the number of occurrences of the th word n all documents from the th class. T = N /n s the relatve frequency of occurrences of the th word n the th class; R = max T ; S = arg max T s the class whch we assgn to the th word. The term relevance C s calculated by the followng: С L 1 T 1 R 1 L 1 L T. 1, S 4. GA-based feature selecton The popular knd of wrappers s feature selecton based on genetc algorthms (GA). GA s used as an optmzaton algorthm for fndng the optmal or the sub-optmal subset of the orgnal feature set wth the predefned classfcaton algorthm [3]. The detals of the GA-based wrapper are the followng: 1. Randomly ntalze the populaton of the bnary strngs wth the length equals to dmensonalty of the consdered classfcaton problem. Set generaton counter g = 0.. Apply the classfcaton algorthm on the valdaton set for all ndvduals form the populaton (wth ncludng only features wth 1 value n the chromosomes). Set the classfcaton effectveness measure (.e. ) on the valdaton set as a ftness functon value for all ndvduals. 3. Save the ndvdual wth the best value of ftness functon. 4. Check f the generaton counter s greater than some predefned value G (maxmal number of generatons): f g > G than go to the step 7; otherwse go to step Usng GA operators: selecton, crossover and mutaton, form the next populaton. 6. Increment the generaton counter g = g + 1. Go to step. 7. Put the best ndvdual as a soluton of the feature selecton procedure; apply the chosen feature subset for the test set. 5. Self-adaptve GA For solvng the problem of GA settng parameters we use the self-confgurng algorthm that was proposed n [17]. The scheme of the self-confgurng GA s presented n Fgure 1. In the self-confgurng GA (SCGA), dfferent types of selecton, recombnaton, and dfferent levels of mutaton are performed smultaneously. In the begnnng of the algorthm, all types of GA operators have the same probablty to be use for a new off-sprng generaton. After that, the dynamc adaptaton of probabltes s performed accordng usefulness of GA operator types n terms of ftness functon. 6

8 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 Fgure 1. Self-confgurng GA (SCGA). The detals of the self-adustng GA are the followng: 1. Put the probabltes of all used types of GA operators: p =1/N, where = 1,,N, N s the number of types of the th operator, = 1,,N, N s the number of GA operators. In our case N=3: selecton, recombnaton, and mutaton. p Set threshold probabltes for all used types of GA operators: N 3. Generate new populaton. For each off-sprng we randomly choose types of selecton, recombnaton, and mutaton accordng probabltes p. 4. Recalculate the probabltes wth the followng: 4.1. For each th operator do: Set S For each th type of the th operator do: If p p 1 T N AND p p, where T s the number of generatons, then: S S p p ; p p If p p 1 T N then: S S 1 T N ; p p T N Calculate average ftness functon F of all off-sprngs of the current generaton that were generated wth the th type of the th operator Fnd the best type d of the th operator wth the maxmal ftness functon on the current generaton and recalculate ts probablty: p d = p d +S 5. Check stop crteron. If TRUE then: END; else: go to the step 3. Moreover two co-evolutonary algorthms (CEA) [19] have been realzed. Both use sland model wth four slands and cooperatve-compettve strategy for resources allocatng. Frst of them (CEA) used four best genetc algorthms (for dfferent types of problems), second one use four ndependent (on the start and on the probabltes recalculaton stage) SCGA (CESCGA). 6. Classfcaton Algorthms As classfcaton algorthms we use the k-nn algorthm, the lnear SVM algorthm, and ANN. The classfcaton crteron s the macro [6] whch s approprate for classfcaton problems wth 7

9 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 unbalanced classes. For k-nn we performed valdaton of k from 1 to 15 on the valdaton sample. We used 80% of the tran sample for the frst level of learnng and 0% for the valdaton. The same scheme of valdaton s used for feature selecton based on self-adustng GA Genetc Algorthms for Automated Artfcal Neural Network Desgn Artfcal neural network (ANN) s a set of nterconnected neurons. Typcally, the transfer functons of all the neurons n the neural network are fxed, and the weghts are the parameters of the neural network. In case of GA usng for ANN weghts tunng: weghts are recorded sequentally n the chromosome as a bnary code. The number of bts that are used to encode a sngle weghtng coeffcent depends on the accuracy and the spread of possble values of weghts. In case of GA usng for ANN structure desgn (GA-ANN): 1. Hdden layers are coded sequentally;. Each neuron s encoded n four bts; 3. Each neuron wll exst wth probablty equal to If n the network a neuron s not presented, ts place n the chromosome s marked wth zeros. Otherwse, t s randomly selected as one of the ffteen actvaton functons, whose number s wrtten n bnary code Snce the effcency of the genetc algorthm for ANN desgn depends on the dmenson of the problem n hand, t s reasonable to avod the use of unnformatve features. The modfcaton of the genetc algorthm for the choce of the most nformatve features durng the automated desgn of ANN assumes the use of addtonal bts n the GA chromosomes. These bts determne whether an nput s ncluded n the nput layer or not. 7. Results of numercal experments Tables 1-10 show the results of the numercal experments for problem defntons 1 and wthout dmensonalty reducton and after feature selecton wth the self-adaptve GA-based wrapper. The best results n terms of are bold (for each case ndependently). The column contans number of features after GA-based wrapper performng. The numercal results showed that the most effectve term weghtng s TRR. The most effectve classfcaton algorthm s ANN. The most effectve optmzaton technque s SCGA. Feature selecton wth self-confgurng GA provdes mprovement of classfcaton effectveness and sgnfcant dmensonalty reducton wth all term weghtng methods and wth all classfcaton algorthms. Table 1. Results for problem defnton 1 wth k-nn Term weghtng method All terms Feature selecton IDF GR CW RF TM TRR NTW Table. Results for problem defnton wth k-nn Term weghtng method All terms Feature selecton IDF GR

10 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 CW RF TM TRR NTW Table 3. Results for problem defnton 1 wth SVM Term weghtng method All terms Feature selecton IDF GR CW RF TM TRR NTW Table 4. Results for problem defnton wth SVM Term weghtng method All terms Feature selecton IDF GR CW RF TM TRR NTW Table 5. Results for problem defnton 1 wth SCGA-ANN Term weghtng method All terms Feature selecton IDF GR CW RF TM TRR NTW Table 6. Results for problem defnton wth SCGA-ANN Term weghtng method All terms Feature selecton IDF GR CW RF TM TRR NTW

11 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 Table 7. Results for problem defnton 1 wth CEA-ANN Term weghtng method All terms Feature selecton IDF 0,673 0, GR 0,711 0, CW 0,739 0, RF 0,8 0,88 1 TM 0,69 0, TRR 0,85 0, NTW 0,813 0, Table 8. Results for problem defnton wth CEA-ANN Term weghtng method All terms Feature selecton IDF 0,655 0, GR 0,647 0, CW 0,634 0,67 91 RF 0,655 0, TM 0,599 0, TRR 0,68 0, NTW 0,675 0, Table 9. Results for problem defnton 1 wth CESCGA-ANN Term weghtng method All terms Feature selecton IDF 0,644 0, GR 0,639 0, CW 0,61 0,66 19 RF 0,64 0, TM 0,586 0, TRR 0,674 0, NTW 0,667 0, Table 10. Results for problem defnton wth CESCGA-ANN Term weghtng method All terms Feature selecton IDF 0,67 0, GR 0,697 0,76 65 CW 0,738 0, RF 0,817 0, TM 0,685 0, TRR 0,8 0,87 45 NTW 0,804 0, Conclusons The text classfcaton problem for natural language call routng was consdered n the paper. Seven dfferent term weghtng methods were appled. As dmensonalty reducton methods, the feature 10

12 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 selecton based on self-confgurng GA s consdered. k-nn, lnear SVM and ANN were used as classfcaton algorthms. The numercal results showed that the most effectve term weghtng s TRR. The most effectve classfcaton algorthm s ANN. Feature selecton wth self-confgurng GA provdes mprovement of classfcaton effectveness and sgnfcant dmensonalty reducton wth all term weghtng methods and wth all classfcaton algorthms. Acknowledgements The reported study was funded by Russan Foundaton for Basc Research, Government of Krasnoyarsk Terrtory, Krasnoyarsk Regon Scence and Technology Support Fund to the research proect References [1] Akhmedova S, Semenkn E and Sergenko R 014 Automatcally generated classfers for opnon mnng wth dfferent term weghtng schemes Informatcs n Control, Automaton and Robotcs (ICINCO 014) pp [] Baharudn B, Lee L H and Khan K 010 A revew of machne learnng algorthms for textdocuments classfcaton Journal of advances n nformaton technology 1(1) pp 4 0 [3] Debole F and Sebastan F 004 Supervsed term weghtng for automated text categorzaton In Text mnng and ts applcatons pp [4] Fox C 1989 A stop lst for general text In ACM SIGIR Forum 4 pp 19 1 [5] Gasanova T, Sergenko R, Akhmedova S, Semenkn E and Mnker W 014 Opnon mnng and topc categorzaton wth novel term weghtng Proceedngs of the 5th Workshop on Computatonal Approaches to Subectvty, Sentment and Socal Meda Analyss, ACL 014 pp [6] Goutte C and Gausser E 005 A probablstc nterpretaton of precson, recall and f-score, wth mplcaton for evaluaton Advances n nformaton retreval pp [7] Han E-H S, Karyps G, and Kumar V 001 Text Categorzaton Usng Weght Adusted k-nearest Neghbor Classfcaton [8] Joachms T 00 Learnng to Classfy Text Usng Support Vector Machnes: Methods, Theory and Algorthms [9] Ko Y 01 A study of term weghtng schemes usng class nformaton for text classfcaton Proceedngs of the 35th nternatonal ACM SIGIR conference on Research and development n nformaton retreval pp [10] Kwon O-W and Lee J-H 003 Text categorzaton based on k-nearest neghbor approach for web ste classfcaton Informaton Processng & Management 39(1) pp 5 44 [11] Lan M, Tan C L, Su J and Lu Y 009 Supervsed and tradtonal term weghtng methods for automatc text categorzaton IEEE Transactons on Pattern Analyss and Machne Intellgence 31(4) pp [1] Lee C, Jung S, Km S and Lee G 009 Example based dalog modelng for practcal mult-doman dalog system Speech Communcaton 51(5) pp [13] Moraru D I, Vntan L N and Tresp V 005 Metaclassfcaton usng SVM classfers for text documents Intl. Jrnl. of Appled Mathematcs and Computer Scences 1(1) [14] Porter M F 001 Snowball: A language for stemmng algorthms [15] Salton G and Buckley C 1988 Term-weghtng approaches n automatc text retreval Informaton processng & management 4(5) pp [16] Sebastan F 00 Machne learnng n automated text categorzaton ACM computng surveys (CSUR) 34(1) pp

13 V Internatonal Workshop on Mathematcal Models and ther Applcatons 016 IOP Publshng IOP Conf. Seres: Materals Scence and Engneerng 173 (017) do: / x/173/1/01008 [17] Semenkn E and Semenkna M 01 Self-confgurng genetc programmng algorthm wth modfed unform crossover 01 IEEE Congress on Evolutonary Computaton [18] Sergenko R, Gasanova T, Semenkn E and Mnker W 014 Text categorzaton methods applcaton for natural language call routng 11 th Internatonal Conference on Informatcs n Control, Automaton and Robotcs (ICINCO 014) pp [19] Sergenko R and Semenkn E 010 Compettve cooperaton for strategy adaptaton n coevolutonary genetc algorthm for constraned optmzaton IEEE Congress on Evolutonary Computaton [0] Soucy P and Mneau G W 005 Beyond tfdf weghtng for text categorzaton n the vector space model IJCAI 5 pp [1] Suhm B, Bers J, McCarthy D, Freeman B, Getty D, Godfrey K and Peterson P 00 A comparatve study of speech n the call center: Natural language call routng vs. touch-tone menus Proceedngs of the SIGCHI conference on Human Factors n Computng Systems pp [] Xu H and L C 007 A novel term weghtng scheme for automated text categorzaton Seventh Internatonal Conference on Intellgent Systems Desgn and Applcatons (ISDA 007) pp [3] Yang J and Honavar V 1998 Feature subset selecton usng a genetc algorthm Feature extracton, constructon and selecton pp [4] Yang Y and Pedersen J O 1997 A comparatve study on feature selecton n text categorzaton ICML 97 pp