Supervised and Unsupervised Text Classification via Generic Summarization

Transcription

1 Intenatonal Jounal of Compute Infomaton Systems and Industal Management Applcatons. ISSN Volume 5 (03) pp MIR Labs, Supevsed and Unsupevsed Text Classfcaton va Genec Summazaton Dmty Tsaev, Mhal Petovsy and Igo Mashechn 3 Compute Scence Depatment, Lomonosov Moscow State Unvesty, Moscow, Russa tsaev@mlab.cs.msu.su Compute Scence Depatment, Lomonosov Moscow State Unvesty, Moscow, Russa mchael@cs.msu.su 3 Compute Scence Depatment, Lomonosov Moscow State Unvesty, Moscow, Russa mash@cs.msu.su Abstact: Ths pape pesents a new genec text summazaton method usng Non-negatve Matx Factozaton (NMF) to estmate sentence elevance. Poposed sentence elevance estmaton s based on nomalzaton of NMF topc space and futhe weghtng of each topc usng sentences epesentaton n topc space. The poposed method shows bette summazaton qualty and pefomance than state of the at methods on DUC 00 standad dataset. In addton, we study how ths method can mpove the pefomance of supevsed and unsupevsed text classfcaton tass. In ou expements wth Reutes-578 and 0 Newsgoups benchma datasets we apply developed text summazaton method as a pepocessng step fo futhe mult-label classfcaton and clusteng. As a esult, the qualty of classfcaton and clusteng has been sgnfcantly mpoved. Keywods: genec text summazaton, latent semantc analyss, non-negatve matx factozaton, mult-label classfcaton, clusteng. I. Intoducton Automatc classfcaton of has become an mpotant eseach ssue snce the oveload of electonc text nfomaton. Thee ae manly two machne leanng appoaches to esolve ths tas: supevsed appoach, whee pedefned categoy labels ae povded fo tanng set of, and unsupevsed document classfcaton (also nown as document clusteng), whee the classfcaton must be done entely wthout efeence to extenal nfomaton. In ths pape, we consde both of these appoaches. As a supevsed classfcaton we have chosen a mult-label classfcaton, whch s a futhe genealzaton of tadtonal mult-class leanng tas. In mult-label case the classes ae not mutually exclusve and any sample may belong to seveal classes n the same tme. As an unsupevsed classfcaton we use two clusteng methods, both flat and heachcal. Flat clusteng ceates a flat set of clustes wthout any explct stuctue that would elate clustes to each othe. Heachcal clusteng ceates a heachy of clustes. Anothe text mnng tas we consde s automatc text summazaton. It becomes vey mpotant ecently because of upasng nfomaton oveload. Text summaes can be ethe quey-based summaes o genec summaes. A quey-based summay pesents the contents of the document that ae closely elated to the ntal use quey. As opposed to that, a genec summay s amed at a boad communty of eades and should contan all man topcs of the text [], []. Ths pape pesents a new text summazaton method, whch constucts genec summaes n extacts fom. These ae the summaes completely consstng of fagments taen fom the ognal text. Phases, sentences o paagaphs can be used as the text fagments. A sentence s usually used to expess content n summazaton. We wll consde text sentences as basc fagments below. Howeve, fo longe content can be epesented by a set of paagaphs as basc fagments. The developed method has been expementally vefed on DUC 00 benchma dataset [], [3] wth state of the at methods. Genec summay of the document contans the fagments (sentences), whch descbe all man topcs of the text. Theefoe n ths pape we also study the applcablty of summaes nstead of ognal texts n mult-label classfcaton and clusteng tass. Snce the document may have moe than one topc the mult-label classfcaton tas has been chosen as a moe geneal appoach n compason to tadtonal mult-class classfcaton. In mult-label case each document can belong to seveal classes,.e. may have seveal topcs. In addton to supevsed classfcaton, we expemented wth flat and heachcal clusteng, to study how ou method can mpove classfcaton of unlabeled. The emande of ths pape s oganzed as follows: Secton II pesents a poposed genec text summazaton MIR Labs, USA

2 50 method and ts expemental compason wth state of the at methods. Secton III s devoted to expemental nvestgaton of ou appoach, whee each full text document s eplaced wth ts summay, n mult-label classfcaton and clusteng tass. Fnally, n Secton IV, we conclude the pape. II. Genec Text Summazaton Methods Nowadays the most state of the at methods of automatc text summazaton whch buld genec summaes n the extacts fom ae based on Latent Semantc Analyss (LSA) [], [4]-[6]. In these methods the ognal text s epesented n the fom of a numecal matx. Matx columns coespond to text sentences (o othe fagments), and each sentence s epesented n the fom of a vecto n the text tem space. Futhe, LSA s appled to the eceved matx to constuct sentences epesentaton n the text topc space. The dmensonalty of the topc space s much less than dmensonalty of the ntal tem space. The choce of the most mpotant sentences s caed out on the bass of sentences epesentaton n the topc space. The numbe of mpotant sentences s defned by the length of the demanded summay (the length s usually measued n the numbe of wods). LSA pefoms one of the matx decomposton algothms on the ognal text matx to constuct sentences epesentaton n text topc space, theeby bngng out the semantc connectedness pesent among the sentences [7]. Sngula Value Decomposton (SVD) s the tadtonal matx decomposton algothm used fo LSA, wheen lowe dmensonal components fom the decomposton ae tuncated. On tuncaton, the lngustc nose pesent n the vecto epesentaton s emoved, and the semantc connectedness s made vsble. One of the dsadvantages of usng SVD s that the tuncated matx wll have negatve components, whch s not natual fo ntepetng the textual epesentaton. Nonnegatve Matx Factozaton (NMF) addesses ths ssue by constuctng non-negatve pats-based epesentaton as the matx decomposton algothm fo pefomng LSA [6]-[8]. Futhe we descbe the poposed genec text summazaton method usng NMF to estmate sentence elevance. And also we adduce ts expemental compason wth state of the at methods usng SVD and NMF. A. Poposed Genec Text Summazaton Method The fst step s the ceaton of a tem by sentences matx A = [A, A,, A n ], whee each column A epesents the weghted tem fequency vecto of sentence n the document unde consdeaton. The sentence vecto A j = [a j, a j,, a mj ] T s defned as: a j = L(t j )G, whee t j denotes the fequency wth whch tem occus n sentence j, L(t j ) s the local weght fo tem n sentence j, and G s the global weght fo tem n the whole document. We have expemented wth vaous weghtng schemes [9] and eceved the best esults by usng bnay local weght and entopy global weght: Bnay local weght: L(t j ) =, f tem appeas at least once n sentence j; L(t j ) = 0, othewse. Entopy global weght (): G Tsaev, Petovsy and Mashechn p log p (, () N j j ) j log N whee p j = t j / F, F s the total numbe of tmes that tem occus n the whole document, N s the numbe of sentences n the document. If thee ae m tems and n sentences n the document we obtan an m n matx A fo the document. The next step s to apply NMF to matx A: A WH. Matx W deves a mappng between the m dmensonal tem space and the dmensonal topc space. Each column of H epesents coespondng text sentence as an addtve combnaton of the bass topcs. Then we nomalze the topc space: A = WH = NomWNomH, whee NomW = WNom -, NomH = NomH, Nom = dag( W,, W ). We use Eucldean nom fo columns of matx W (): W m W,. () Columns of matx NomH coespond to n sentences n the nomalzed topc space. The -th ow NomH = [nomh,, nomh n ] ndcates weghts of -th topc n all n sentences. The geate nom of NomH ows, the geate weghts of coespondng topcs n all text. Poceedng fom t, we calculate topc weghts as noms of ows of matx NomH. Weght of -th topc s (3), (4): NomH n NomH NomH j, (3) W n j H j j W H,. (4) The weghted sentences epesentaton n the nomalzed topc space s matx WeghtedH (5), (6): WeghtedH dag W H,, W H NomH, (5) WeghtedH dag W H,, W H H. (6) Sentences fo the summay ae selected accodng to the sum of topc weghts. Relevance of j-th sentence s (7), (8): R ( WeghtedH ) ( WeghtedH j j ), (7) R ( WeghtedH ) ( W H H ). (8) j Fnally equed numbe of sentences wth the hghest elevance values s selected fo the summay [], [3]. B. Expements We use the ROUGE (Recall-Oented Undestudy fo Gstng Evaluaton) pacage to evaluate the poposed method [0], [3]. It ncludes measues to automatcally detemne the j

3 Supevsed and Unsupevsed Text Classfcaton va Genec Summazaton 5 qualty of a summay by compang t to othe model (deal) summaes ceated by humans. The measues count the numbe of ovelappng unts such as n-gam, wod sequences, and wod pas between the compute-geneated summay to be evaluated and the deal summaes ceated by humans. ROUGE measues ROUGE-, ROUGE-L, ROUGE-S, ROUGE-W s ecommended to use fo evaluaton sngle-document summazaton methods on datasets DUC 00 and DUC 00 [3]. In the last edtons of Document Undestandng Confeences (DUC), ROUGE was used as an automatc evaluaton method. As expemental data, we use the DUC 00 standad dataset. Ths dataset conssts of 533 and 95 model summaes. We evaluated state of the at summazaton methods such as the SVD-Classc (Gong and Lu appoach [4]), the SVD-Squae (Stenbege and Ježe appoach [5], []), the NMF-Genec (Lee, Pa, Ahn, Km appoach [6]), and ou poposed method based on NMF. Also we consde method whch extacts sentences wth the hghest wod count,.e. sentences fo the summay ae selected accodng to the numbe of wods, except stop wods. And fnally we consde andom sentence extacton method. We denote these methods as Wod Count and Random, espectvely. The numbe of topcs fo LSA s selected much smalle than dmensonaltes of the text matx,.e. << mn(m, n). Fo DUC 00 dataset aveage numbe of document matx ows s 39, and aveage numbe of columns s 37. The dagam (Fg. ) shows change of the ROUGE- f-measue dependng on numbe of topcs, whee numbe of topcs vaes fom to 0. ROUGE- f-measue has been selected because the esults of usng vaous ROUGE measues ae smla. sentences that a eade could ntepet, we could smply extact the top pseudo sentences, whee = (p/00)n. Howeve, because the lnea combnatons of tems ae not eally eadable sentences, we use fou methods mentoned above to extact the actual sentences that ovelap the most n tems of vecto length wth top pseudo sentences []. We bult summaes consstng of 00 wods snce model summaes of DUC 00 dataset consst of 00 wods. Fo each method we calculate numbe of topcs as = (00/nw)n, whee nw s the total numbe of tems n the text. Table I shows the ROUGE f-measue values of fou methods usng ROUGE evaluaton. In addton we show the ROUGE scoes fo Random and Wod Count methods n Table II. Measue SVD- Classc SVD- Squae NMF- Genec Ou Method ROUGE ROUGE-L ROUGE-S ROUGE-W TABLE I. ROUGE F-MEASURE VALUES Measue Random Wod Count ROUGE ROUGE-L ROUGE-S ROUGE-W TABLE II. ROUGE F-MEASURE VALUES We use MATLAB functon svds() fo mplementaton SVD. NMF also s mplemented on MATLAB. The numbe of teatons n SVD and NMF algothms s estcted 300. We have eceved, that NMF wos faste SVD fo 0 pecent. The expements demonstate bette summazaton qualty and pefomance of ou poposed method n compason wth othe methods. III. Usng developed text summazaton to mpove classfcaton tass Fgue. ROUGE- f-measue / numbe of topcs When we pefom one of the matx decomposton on an m n text matx, we can vew the new dmensons as some sot of pseudo sentences: lnea combnatons of the ognal tems (left sngula vectos n SVD case, and matx W n NMF case). Fom a summazaton pont of vew, the numbe of extacted sentences s dependent on the summay ato (a ato of the summay length wth espect to the length of the ognal text). We now what pecentage of the full text the summay should be: pat of the nput to the summaze s that a p% summay s needed. If the pseudo sentences wee eal We use ou NMF-based method and SVD-Squae method, as shown the best esults n pevous secton, to eplace the full document text by ts summay as a pepocesso step fo futhe classfcaton. The length of the summay s defned fom a pecentage of ntal text nfomaton amounts. We use topc weghts to estmate an nfomaton amount. In SVD-Squae method weghts ae defned as a squae of coespondng sngula values [0], [5], []. In ode to the summay contaned p% nfomaton of the ntal text, we pefom the full decomposton of m n text matx fo = mn(m, n) dmensons. All topcs poduce 00% nfomaton and the contbuton coesponds to the weghts. The numbe of summay topcs s selected poceedng fom ato of the sum of maxmum weghts wth espect to the sum of all weghts and ths ato should equal to p/00 (9):

4 5 sot( weght,' descend ') p, (9) 00 weght j j whee weght s a sequence of topc weghts, sot(weght, descend ) s a sequence of topc weghts whch elements ae soted n the descendng ode. Futhe we use ou NMF-based o SVD-Squae methods, coespondng to NMF o SVD decompostons, to extact most elevant sentences. In ths secton we adduce expements wth eplacement the full document text by ts summay fo mult-label classfcaton and clusteng tass. A tadtonal vecto space epesentaton s used n all classfcaton methods consdeed n ths pape [9]. Theefoe the nomalzed weghtng scheme tfdf was used fo vecto epesentaton, both, and the summaes [4]. A. Mult-label Classfcaton The nave appoach to mult-label leanng s based on one-aganst-all bnay decomposton ( vs. all). Fo each class sepaate bnay classfcaton subpoblem s fomulated. In ths subpoblem all samples fom the mult-label tanng set ae dvded nto two dsjont subsets: postve samples, whose belong to ths class, and negatve samples that do not belong. Then tadtonal bnay leanng algothm s appled to each bnay subpoblem. As a esult, a set of ndependent bnay classfes ae taned. Each classfe s assocated wth class and pedcts whethe a gven sample belongs to ths patcula class. In ths pape we also use mult-label classfcaton method based on paed compasons appoach (.e. one-aganst-one bnay decomposton, vs. ), whch s descbed n []. In ths method each pa of possbly ovelappng classes s sepaated by two pobablstc bnay classfes, whch solate the ovelappng and non-ovelappng aeas. Then ndvdual pobabltes geneated by bnay classfes ae combned togethe to estmate fnal class pobabltes fttng extended Badley-Tey model wth tes. As the bnay classfe n these mult-label classfcaton methods we use a Suppot Vecto Machne (SVM) [3]. In addton to these methods we apply lnea theshold functon defned on the class elevances vecto space [4]. Results of mult-label classfcaton expements we evaluate by Loss cteon. measues aveage symmetc set dffeence () between pedcted and elevant sets of classes fo test (0): Tsaev, Petovsy and Mashechn the most popula benchma datasets fo mult-label classfcaton. Reutes-578 ae pesented n SGML fomat. Theefoe the texts we have selected by followng ctea: the TOPICS node contans one o moe elements; attbute TYPE of TEXT node possess value NORM. We have dvded the obtaned dataset on tanng and test usng values of attbute LEWISSPLIT. The value TEST of attbute LEWISSPLIT ndcates the document was used fo testng, the othe values of ths attbute ndcate the document was used fo tanng. In addton to ths dataset we also use ts shot veson, whee ae not less than 5 bytes. Table III shows chaactestcs of the obtaned datasets. Dataset Tanng Test Numbe of classes(topcs) complete educed TABLE III. MULTI-LABEL BENCHMARK DATASETS Dagams n Fg. ae showng changes of Loss cteon fo vs. all mult-label classfcaton dependng on pecentage of the nfomaton amount selected fo summaes n ou NMF-based summazaton method. The dagam 00 wods coesponds to a case of usng addtonal summay length lmtaton the mnmum summay length should be 00 wods; the dagam 0 wods coesponds to a case wthout ths lmtaton; the lne full text s a case of full document text classfcaton. Fom ths data follows the best esults have been eceved wth condtons: 30% theshold of the nfomaton amount selected fo summaes and 00 wods lmtaton. Smla esults have been eceved fo vs mult-label classfcaton. We wll use these two condtons futhe n classfcaton tass. Fgue. Loss / nfomaton amount pecentage Los s F( x ) y, (0) Y whee s numbe of test ; Y s the set of document classes; x s a test document; F(x ) s the set of pedcted classes fo document x ; y s the set of elevant classes fo document x. We evaluated mult-label classfcaton of full texts and the summaes on Reutes-578 dataset [5]. Ths s one of Expemental esults wth mult-label classfcaton of Reutes-578 datasets and ts summaes ae pesented n the Table IV and the Table V, coespondng to ou NMF-based and SVD-Squae summazaton methods. Ou summazaton method shows bette esults, than SVD-Squae method. In addton, we have eceved the text pepocessng by NMF decomposton wth ou weghts calculaton wos faste SVD fo appoxmately 0 pecent. A compason of the dataset szes (n wod count and n

5 Supevsed and Unsupevsed Text Classfcaton va Genec Summazaton 53 megabytes) and dmensonalty of featue space (numbe of dffeent tems) wth ts summazed vesons by poposed NMF-based method s esulted n Table VI and Table VII. Method Dataset Nomal vs. educed 0,00997 ±,6e-05 vs. complete 0, ±,8e-05 vs. all educed 0,00839 ±3,6e-05 vs. all complete 0, ±,5e-05 Summazed 0, ±3,e-05 0, ±,7e-05 0, ±5,e-05 0, ±,5e-06 Impovng 5,5% 48,8% TABLE IV. MULTI-LABEL CLASSIFICATION RESULTS (NMF CASE) Method Dataset Nomal vs. educed 0,00997 ±,6e-05 vs. complete 0, ±,8e-05 vs. all educed 0,00839 ±3,6e-05 vs. all complete 0, ±,5e-05 Summazed 0,00957 ±,6e-05 0, ±9,e-06 0,00835 ±,0e-05 0, ±,5e-06 7% 3% Impovng 0,4% 45,5% 0,% -0,9% TABLE V. MULTI-LABEL CLASSIFICATION RESULTS (SVD CASE) Dataset Intal (wods/mb) complete ,8Mb educed 584 3Mb Dataset Summazed (wods/mb) ,9Mb 98536,Mb TABLE VI. REUTERS-578 SIZE REDUCTION Tanng Test Impovng 55% 9% 6,7% 6,7% Numbe of classes(topcs) complete ,9% educed ,7% TABLE VII. REUTERS-578 FEATURE SPACE REDUCTION Fom the obtaned expemental esults follows the usng summaes nstead of full texts mpoves qualty of mult-label classfcaton. Theefoe, t s possble to daw a concluson, that the text summazaton methods well defnes man topcs of and on the bass selects sentences, whch n the best way descbe them. But ou pesented NMF-based method shows bette classfcaton qualty and pefomance than SVD analogue. Fom the obtaned expemental esults follows the usng summaes nstead of full texts mpoves qualty of mult-label classfcaton. Theefoe, t s possble to daw a concluson, that the text summazaton methods well defnes man topcs of and on the bass selects sentences, whch n the best way descbe them. But ou pesented NMF-based method shows bette classfcaton qualty and pefomance than SVD analogue. B. Clusteng We consde two most popula clusteng algothms based on matx decomposton, such as SVD, NMF. The fst s the Pncpal Decton Dvsve Pattonng () algothm sepaates the ente set of nto two pattons by usng pncple dectons, whch ae obtaned afte SVD decompostons tem-document matx. Each of two pattons wll be sepaated nto two sub-pattons usng the same pocess ecusvely. The esult s heachcal of pattons aanged nto a bnay tee. Theeby fo specfed we can eceve n[, ] clustes [6]. The second s a document clusteng method based on the NMF of the tem-document matx of the gven document copus. In the latent semantc space deved by the NMF, each axs captues the base topc of a patcula document cluste, and each document s epesented as an addtve combnaton of the base topcs. The cluste membeshp of each document can be easly detemned by fndng the base topc (the axs) wth whch the document has the lagest pojecton value [7]. We use two extenal ctea of clusteng qualty: Rand ndex and F-measue [8]. The Rand ndex (RI) measues the pecentage of clusteng algothm decsons that ae coect. Thee ae N(N )/ decsons, one fo each of the pas of n the collecton, whee N s the numbe of. We want to assgn two to the same cluste f and only f they ae smla. A tue postve (TP) decson assgns two smla to the same cluste; a tue negatve (TN) decson assgns two dssmla to dffeent clustes. Thee ae two types of eos we can commt. A false postve (FP) decson assgns two dssmla to the same cluste. A false negatve (FN) decson assgns two smla to dffeent clustes. The Rand ndex s smply accuacy (): TPTN RI. () TP FP FN TN The Rand ndex gves equal weght to false postves and false negatves. Sepaatng smla s sometmes wose than puttng pas of dssmla n the same cluste. We use the F-measue to penalze false negatves moe stongly than false postves by selectng a value β >, thus gvng moe weght to ecall (): TP P, TP FP TP R, TP FN ( ) PR F. () P R We evaluated and NMF clusteng of full texts and the summaes by poposed NMF-based method on 0 Newsgoups dataset [9]. Ths s one of the most popula benchma datasets fo clusteng. We have chosen the whch sze sn't less than 5 bytes and emove duplcates. As a esult we have obtaned 5800 dstbuted on 0 pedefned clustes. Expemental esults wth and NMF clusteng of 0 Newsgoups dataset and ts summaes ae pesented n Table VIII and Table IX. It s woth notng we chose dmensons equal to 5 and 6 fo algothm whch has

6 54 constucted pattons nto 6 and 3 clustes, as a full bnay tee of depth 4 and 6, espectvely. It s woth notng that algothm has constucted patton nto 6 clustes, as a full bnay tee of depth 4. Dffeently fom n NMF the equed numbe of clustes s specfed as an nput paamete. Reductons of the dataset sze and the featue space ae esulted n the table X. Fom the obtaned expemental esults follows that usng summaes nstead of full texts slghtly mpoves qualty of NMF and clusteng, but sgnfcant educes the sze of the pocessed data. Method (6 clustes) (3 clustes) NMF (0 clustes) Method (6 clustes) (3 clustes) NMF (0 clustes) Sze (wods/mb) Featue space Nomal RI Summazed RI 0, , ,8% 0,9439 0, ,% 0, , ,6% TABLE VIII. RI CLUSTERING RESULTS Nomal F Summazed F Impovng RI Impovng F 0, , ,5% 0, , ,6% 0, , ,% TABLE IX. F CLUSTERING RESULTS Intal Summazed Impovng ,9Mb ,3Mb 36% 35,% ,8% TABLE X. 0 NEWSGROUPS SIZE AND FEATURE SPACE REDUCTIONS IV. Concluson Ths pape pesents a new genec text summazaton method usng NMF to estmate sentence elevance. Poposed sentence elevance estmaton s based on nomalzaton of NMF topc space (o featue space) and futhe weghtng of each topc usng sentences epesentaton n topc space. NMF has the advantage ove SVD that t poduces a natual addtve pats-based epesentaton of data, owng to ts non-negatvty whch can be helpful n ntepetaton of semantc featues (topcs). The poposed method shows bette summazaton qualty and pefomance than state of the at methods on DUC 00 standad dataset. In addton, we use ths text summazaton method to eplace full text by ts summay n supevsed and unsupevsed text classfcaton tass. Ou expements show applcablty of ths appoach and even mpovement of the classfcaton qualty of mult-label classfcaton and clusteng on benchma datasets. Theefoe, t s possble to daw the followng conclusons. The pesented method of text summazaton defnes man topcs of well. It Tsaev, Petovsy and Mashechn emoves nose and mpoves classfcaton pefomance. It s woth to use ths method as a pepocesso step n eal text mnng systems, because the summaes whch t poduces, ae ease to stoe and pocess and vey nfomatve at once. As an example of eal text mnng systems t s possble to adduce system fo elevance assessment of eseach publcatons n educatonal eseach, whch was ealzed wthn the Euopean Educatonal Reseach Qualty Indcatos (EERQI) poject as pat of the Euopean Seventh Famewo Pogamme [5]. In EERQI poject methods of automatc semantc analyss fo the detecton of ey sentences n a text ae used. One of esults of the eseach s that hghlghtng of ey sentences maes t possble to apdly flte out bad qualty: pocessng the hghlghted texts too 4 tmes shote tme [6]. Also n the EERQI poject tested the ole of ey sentence n elevance anng. In the EERQI seach and quey engne, the basc anng algothm of the publcly avalable Lucene seach engne was used. They compaed the esults of ths elevance anng wth the lst of n whch the quey wod(s) occu(s) n ey sentences. Lucene uses tem fequences and nvese document fequences fo anng the eteved. The esults show that the top aned elevant atcles etuned by Lucene and those selected by the tool ae dsjont, whch ndcates that the two appoaches ae complementay. Snce the tool etuns a consdeable numbe of elevant atcles that would appea late n Lucene s aned lst, they consde that ths appoach s pomsng and that the ntegaton of the two tools s benefcal fo the use [6]. Thus text summazaton methods and appoaches to the applcaton n classfcaton, clusteng and nfomaton eteval tass ae actual eseach ssues. Acnowledgment The poducton of ths publcaton has been made possble though the fnancal suppot of the Mnsty of Educaton and Scence of the Russan Fedeaton (the state contact # ) and of the Russan Foundaton fo Basc Reseach (the pojects # , # ). Refeences [] Kael Ježe; Josef Stenbege. Automatc Text Summazaton (The state of the at 007 and new challenges). In Poceedngs of Znalost 008, Batslava, Slovaa, pp., Febuay 008, ISBN [] Document Undestandng Confeences, [3] Chn-Yew Ln. Loong fo a few good metcs: Automatc summazaton evaluaton - how many samples ae enough?. In: Poc. of NTCIR 004, Toyo, Japan, pp , 004. [4] Yhong Gong, Xn Lu. Genec Text Summazaton Usng Relevance Measue and Latent Semantc Analyss. In SIGIR-00, 00. [5] Josef Stenbege, Kael Ježe. Text Summazaton and Sngula Value Decomposton. In Lectue Notes fo Compute Scence vol. 457, Spnge-Velag, pp , 004.

7 Supevsed and Unsupevsed Text Classfcaton va Genec Summazaton 55 [6] Ju-Hong Lee, Sun Pa, Chan-Mn Ahn, Daeho Km. Automatc genec document summazaton based on non-negatve matx factozaton. Infomaton Pocessng and Management: an Intenatonal Jounal, Pages: 0-34, 009. [7] Raesh Pete, Shvapatap G, Dvya G, Soman KP. Evaluaton of SVD and NMF Methods fo Latent Semantc Analyss. Intenatonal Jounal of Recent Tends n Engneeng, Vol., No. 3, May 009. [8] Danel Lee, Sebastan Seung. Leanng the pats of objects by non-negatve matx factozaton. Natue, 40, pp , 999. [9] Susan Dumas. Impovng the eteval of nfomaton fom extenal souces. In Behavo Reseach Methods, Instuments & Computes, 3(), pp. 9 36, 99. [0] Recall-Oented Undestudy fo Gstng Evaluaton, [] Josef Stenbege. Text Summazaton wthn the LSA Famewo. Doctoal Thess, Plsen, 007. [] Mhal Petovsy. Paed Compasons Method fo Solvng Mult-label Leanng Poblem. Poceedngs of Intenatonal Confeence on Hybd Intellgent Systems, Neuo-Computng and Evolvng Intellgence, New Zealand, IEEE Pess, 6 pages, 006. [3] J. Platt. Pobablstc Outputs fo Suppot Vecto Machnes and Compason to Regulazed Lelhood Methods. Adv. n Lage Magn Classfes, MIT Pess, pp. 6 74, 999. [4] Mhal Petovsy, Valentna Glazova. Lnea Methods fo Reducton fom Ranng to Multlabel Classfcaton. In Lectue Notes fo Compute Scence vol. 4304, Spnge-Velag, pp. 5-56, 006. [5] Reutes-578 Text Categozaton Collecton, eutes578/. [6] D.L. Boley. Pncpal decton dvsve pattonng. Data Mnng and Knowledge Dscovey, (4), pp , 998. [7] We Xu, Xn Lu, Yhong Gong. Document clusteng based on non-negatve matx factozaton. Poceedngs of the 6th annual ntenatonal ACM SIGIR confeence on Reseach and development n nfomaon eteval, Toonto, Canada, July 8-August 0, 003. [8] Chstophe D. Mannng, Pabhaa Raghavan and Hnch Schütze. Intoducton to Infomaton Reteval. Cambdge Unvesty Pess, 008. [9] The 0 Newsgoups data set, [0] Chs H. Q. Dng. A pobablstc model fo latent semantc ndexng. In Jounal of the Amecan Socety fo Infomaton Scence and Technology, 56(6), pp , 005. [] I. Man. and M.T. Maybuy. Advances n Automatc Text Summazaton. Cambdge. MA: The MIT Pess, 44 pp., 999. [] I.V. Mashechn, M.I. Petovsy, D. S. Popov and D.V. Tsaev. Automatc text summazaton usng latent semantc analyss. Pogammng and Compute Softwae, pp , 0. [3] D.V. Tsaev, I.V. Mashechev, M.I. Petovsy. Text Summazaton Method Based on Nomalzed Non-Negatve Matx Factozaton. Intenatonal Confeence on Mechancal and Electcal Technology, 3d, (ICMET-Chna 0), Volumes 3, pp , 0. [4] D.V. Tsaev, M.I. Petovsy, I.V. Mashechn. Usng NMF-based text summazaton to mpove supevsed and unsupevsed classfcaton. th Intenatonal Confeence on Hybd Intellgent Systems (HIS), Malacca, MALAYSIA, pp , 0. [5] Euopean Educatonal Reseach Qualty Indcatos (EERQI) Poject, [6] Euopean Educatonal Reseach Qualty Indcatos (EERQI) Poject Fnal Repot, Autho Bogaphes D. V. Tsaev s a post-gaduate student of Faculty of Computatonal Mathematcs and Cybenetcs of Lomonosov Moscow State Unvesty (MSU). He eceved hs Dploma (007) n Appled Mathematcs and Infomatcs fom Faculty of Computatonal Mathematcs and Cybenetcs, MSU, Russan Fedeaton. Hs pmay eseach aeas ae: text mnng, ncludng text summazaton, mult-label classfcaton, cluste analyss. M. I. Petovsy s an assocate pofesso of Faculty of Computatonal Mathematcs and Cybenetcs of Lomonosov Moscow State Unvesty (MSU). He eceved hs Dploma (997) and PhD (003) n Appled Mathematcs fom Faculty of Computatonal Mathematcs and Cybenetcs, MSU, Russan Fedeaton. Snce 999 he has been wong at MSU as teache assstant, assstant pofesso and cuently as assocate pofesso (snce 006). Hs pmay eseach aeas ae: statstcal leanng theoy, ncludng enel methods, fuzzy methods, ensemble leanng, obustness; text and data mnng, ncludng mult-label classfcaton, anng, nfomaton extacton; and data mnng applcatons, ncludng ntellgent ntuson detecton, use behavo modelng, eystoe dynamcs, mnng stuctued data. I. V. Mashechn s a pofesso of Faculty of Computatonal Mathematcs and Cybenetcs of Lomonosov Moscow State Unvesty (MSU). He eceved hs Dploma (978), PhD (98) and Docto of Compute Scence (998) n Appled fom Faculty of Computatonal Mathematcs and Cybenetcs, MSU, Russan Fedeaton. Snce 978 he has been wong at MSU as engnee, assstant pofesso, assocate pofesso and cuently as full pofesso (snce 999). 30 yeas expeence n patcpaton and poject management n eseach and development of IT technologes. Development of CRAB tme-shang system fo the BESM-6 Sovet hgh-pefomance compute. Development of esouce quota and plannng system fo tme-shang system of the BESM-6 compute. Development of mult-functonal hgh-level languages coss-pogammng system, based on machne-ndependent ntemedate epesentaton. Cuently hs pmay eseach aeas ae: system pogammng and development of ntellgent and data mnng systems.