Sentiment Classification and Polarity Shifting

Transcription

1 Sentment Classfcaton and Polarty Shftng Shoushan L Sopha Yat Me Lee Yng Chen Chu-Ren Huang Guodong Zhou Department of CBS The Hong Kong Polytechnc Unversty {shoushan.l, sophaym, chenyng3176, Natural Language Processng Lab School of Computer Scence and Technology Soochow Unversty gdzhou@suda.edu.cn Abstract Polarty ng marked by varous lngustc structures has been a challenge to automatc sentment classfcaton. In ths paper, we propose a machne learnng approach to ncorporate polarty ng nformaton nto a document-level sentment classfcaton system. Frst, a feature selecton method s adopted to automatcally generate the tranng data for a bnary classfer on polarty ng detecton of sentences. Then, by usng the obtaned bnary classfer, each document n the orgnal polarty classfcaton tranng data s splt nto two parttons, polarty-ed and polarty-uned, whch are used to tran two base classfers respectvely for further classfer combnaton. The expermental results across four dfferent domans demonstrate the effectveness of our approach. 1 Introducton Sentment classfcaton s a specal task of text classfcaton whose objectve s to classfy a text accordng to the sentmental polartes of opnons t contans (Pang et al., 2002), e.g., favorable or unfavorable, postve or negatve. Ths task has receved consderable nterests n the computatonal lngustc communty due to ts potental applcatons. In the lterature, machne learnng approaches have domnated the research n sentment classfcaton and acheved the state-of-the-art performance (e.g., Kennedy and Inkpen, 2006; Pang et al., 2002). In a typcal machne learnng approach, a document (text) s modeled as a bag-of-words,.e. a set of content words wthout any word order or syntactc relaton nformaton. In other words, the underlyng assumpton s that the sentmental orentaton of the whole text depends on the sum of the sentmental polartes of content words. Although ths assumpton s reasonable and has led to ntal success, t s lngustcally unsound snce many functon words and constructons can the sentmental polartes of a text. For example, n the sentence The char s not comfortable, the polarty of the word comfortable s postve whle the polarty of the whole sentence s reversed because of the negaton word not. Therefore, the overall sentment of a document s not necessarly the sum of the content parts (Turney, 2002). Ths phenomenon s one man reason why machne learnng approaches fal under some crcumstances. As a typcal case of polarty ng, negaton has been pad close attenton and wdely studed n the lterature (Na et al., 2004; Wlson et al., 2009; Kennedy and Inkpen, 2006). Generally, there are two steps to ncorporate negaton nformaton nto a system: negaton detecton and negaton classfcaton. For negaton detecton, some negaton trgger words, such as no, not, and never, are usually appled to recognze negaton phrases or sentences. As for negaton classfcaton, one way to mport negaton nformaton s to drectly reverse the polarty of the words whch contan negaton trgger words as far as term-countng approaches are consdered (Kennedy and Inkpen, 2006). An alternatve way s to add some negaton features (e.g., negaton bgrams or negaton phrases) nto 635 Proceedngs of the 23rd Internatonal Conference on Computatonal Lngustcs (Colng 2010), pages , Bejng, August 2010

2 machne learnng approaches (Na et al., 2004). Such approaches have acheved certan success. There are, however, some shortcomngs wth current approaches n ncorporatng negaton nformaton. In terms of negaton detecton, frstly, the negaton trgger word dctonary s ether manually constructed or reles on exstng resources. Ths leads to certan lmtatons concernng the qualty and coverage of the dctonary. Secondly, t s dffcult to adapt negaton detecton to other languages due to ts language dependence nature of negaton constructons and words. Thrdly, apart from negaton, many other phenomena, e.g., contrast transton wth trgger words lke but, however, and nevertheless, can the sentmental polarty of a phrase or sentence. Therefore, consderng negaton alone s nadequate to deal wth the polarty ng problem, especally for document-level sentment classfcaton. In terms of negaton classfcaton, although t s easy for term-countng approaches to ntegrate negaton nformaton, they rarely outperform a machne learnng baselne (Kennedy and Inkpen, 2006). Even for machne learnng approaches, although negaton nformaton s sometmes effectve for local cases (e.g., not good), t fals on long-dstance cases (e.g., I don t thnk t s good). In ths paper, we frst propose a feature selecton method to automatcally generate a large scale polarty ng tranng data for polarty ng detecton of sentences. Then, a classfer combnaton method s presented for ncorporatng polarty ng nformaton. Compared wth prevous ones, our approach hghlghts the followng advantages:frst of all, we apply a bnary classfer to detect polarty ng rather than merely relyng on trgger words or phrases. Ths enables our approach to handle dfferent knds of polarty ng phenomena. More mportantly, a feature selecton method s presented to automatcally generate the labeled tranng data for polarty ng detecton of sentences. The remander of ths paper s organzed as follows. Secton 2 ntroduces the related work of sentment classfcaton. Secton 3 presents our approach n detals. Expermental results are presented and analyzed n Secton 4. Fnally, Secton 5 draws the concluson and outlnes the future work. 2 Related Work Generally, sentment classfcaton can be performed at four dfferent levels: word level (Webe, 2000), phrase level (Wlson et al., 2009), sentence level (Km and Hovy, 2004; Lu et al., 2005), and document level (Turney, 2002; Pang et al., 2002; Pang and Lee, 2004; Rloff et al., 2006). Ths paper focuses on document-level sentment classfcaton. In the lterature, there are manly two knds of approaches on document-level sentment classfcaton: term-countng approaches (lexcon-based) and machne learnng approaches (corpus-based). Term-countng approaches usually nvolve dervng a sentment measure by calculatng the total number of negatve and postve terms (Turney, 2002; Km and Hovy, 2004; Kennedy and Inkpen, 2006). Machne learnng approaches recast the sentment classfcaton problem as a statstcal classfcaton task (Pang and Lee, 2004). Compared to term-countng approaches, machne learnng approaches usually acheve much better performance (Pang et al., 2002; Kennedy and Inkpen, 2006), and have been adopted to more complcated scenaros, such as doman adaptaton (Bltzer et al., 2007), mult-doman learnng (L and Zong, 2008) and sem-supervsed learnng (Wan, 2009; Dasgupta and Ng, 2009) for sentment classfcaton. Polarty ng plays a crucal role n phrase-level, sentence-level, and document-level sentment classfcaton. However, most of prevous studes merely focus on negaton ng (polarty ng caused by the negaton structure). As one poneer research on sentment classfcaton, Pang et al. (2002) propose a machne learnng approach to tackle negaton ng by addng the tag not to every word between a negaton trgger word/phrase (e.g., not, sn't, ddn't, etc.) and the frst punctuaton mark followng the negaton trgger word/phrase. To ther dsappontment, consderng negaton ng has a neglgble effect and even slghtly harms the overall performance. Kennedy and Inkpen (2006) explore negaton ng by ncorporatng negaton bgrams as addtonal features nto machne learnng approaches. The 636

3 expermental results show that consderng sentment ng greatly mproves the performance of term-countng approaches but only slghtly mproves the performance of machne learnng approaches. Other studes such as Na et al. (2004), Dng et al. (2008), and Wlson et al. (2009) also explore negaton ng and acheve some mprovements 1. Nonetheless, as far as machne learnng approaches are concerned, the mprovement s rather nsgnfcant (normally less than 1%). More recently, Ikeda et al. (2008) frst propose a machne learnng approach to detect polarty ng for sentence-level sentment classfcaton, based on a manually-constructed dctonary contanng thousands of postve and negatve sentmental words, and then adopt a term-countng approach to ncorporate polarty ng nformaton. 3 Sentment Classfcaton wth Polarty Shftng Detecton Documents Polarty-ed Sentences Postve/Negatve Polarty Shftng Detector Polarty-uned Sentences Polarty Classfer Fgure 1: General framework of our approach The motvaton of our approach s to mprove the performance of sentment classfcaton by robust treatment of sentment polarty ng between sentences. Wth the help of a bnary classfer, the sentences n a document are dvded nto two parts: sentences whch contan polarty ng structures and sentences wthout any polarty ng structure. Fgure 1 llustrates the general framework of our approach. Note that ths framework s a general one, that s, dfferent polarty ng detecton methods can be appled to dfferentate polarty-ed sentences from those polarty-uned sentences and dfferent 1 Note that Dng et al. (2006) also consder but-clause, another mportant structure for sentment ng. Wlson et al. (2009) use conjunctve and dependency relatons among polarty words. polarty classfcaton methods can be adopted to ncorporate sentment ng nformaton. For clarfcaton, the tranng data used for polarty ng detecton and polarty classfcaton are referred to as the polarty ng tranng data and the polarty classfcaton tranng data, respectvely. 3.1 Polarty Shftng Detecton In ths paper, polarty ng means that the polarty of a sentence s dfferent from the polarty expressed by the sum of the content words n the sentence. For example, n the sentence I am not dsapponted, the negaton structure makes the polarty of the word 'dsapponted' dfferent from that of the whole sentence (negatve vs. postve). Apart from the negaton structure, many other lngustc structures allow polarty ng, such as contrast transton, modals, and pre-suppostonal tems (Polany and Zaenen, 2006). We refer these structures as polarty ng structures. One of the great challenges n buldng a polarty ng detector les on the lack of relevant tranng data snce manually creatng a large scale corpus of polarty ng sentences s tme-consumng and labor-ntensve. Ikeda et al. (2008) propose an automatc way for collectng the polarty ng tranng data based on a manually-constructed large-scale dctonary. Instead, we adopt a feature selecton method to buld a large scale tranng corpus of polarty ng sentences, gven only the already avalable document-level polarty classfcaton tranng data. Wth the help of the feature selecton method, the top-ranked word features wth strong sentmental polarty orentaton, e.g., great, love, worst are frst chosen as the polarty trgger words. Then, those sentences wth the top-ranked polarty trgger words n both categores of postve and negatve documents are selected. Fnally, those canddate sentences takng opposte-polarty compared to the contanng trgger word are deemed as polarty-ed. The basc dea of automatcally generatng the polarty ng tranng data s based on the assumpton that the real polarty of a word or phrase s decded by the major polarty category where the word or phrase appears more often. As a result, the sentences n the 637

4 frequently-occurrng category would be seen as polarty-uned whle the sentences n the nfrequently-occurrng category would be seen as polarty-ed. In the lterature, varous feature selecton methods, such as Mutual Informaton (MI), Informaton Gan (IG) and B-Normal Separaton (BNS) (Yang and Pedersen, 1997; Forman 2003), have been employed to cope wth the problem of the hgh-dmensonal feature space whch s normal n sentment classfcaton. In ths paper, we employ the theoretcal framework, proposed by L et al. (2009), ncludng two basc measurements,.e. frequency measurement and rato measurement, where the frst measures, the document frequency of a term n one category, and the second measures, the rato between the document frequency n one category and other categores. In partcular, a novel method called Weghed Frequency and Odds (WFO) s proposed to ncorporate both basc measurements: λ P( t c ) 1 λ WFO( t, c ) = P( t c ) {max(0,log )} P( t c ) where P( t c ) denotes the probablty that a document x contans the term t wth the condton that x belongs to category c ; P( t c ) denotes the probablty that a document x contans the term t wth the condton that x does not belong to category c. The left part of the formula P( t c ) mples the frst basc measurement and the rght part log( P( t c ) / P( t c )) mples the second one. The parameter λ ( 0 λ 1)s thus to tune the weght between the two basc measurements. Especally, when λ equals 0, the WFO method fades to the MI method whch fully prefers the second basc measurement. Fgure 2 llustrates our algorthm for automatcally generatng the polarty ng tranng data where c1 and c2 denote the two sentmental orentaton categores,.e. negatve and postve. Step A segments a document nto sentences wth punctuatons. Besdes, two specal words, but and and, are used to further segment some contrast transton structures and compound sentences. Step B employs the WFO method to rank all features ncludng the words. Step D extracts those polarty-ed and polarty-uned sentences contanng ttop where Nmax denotes the upper-lmt number of sentences n each category of the polarty ng tranng data and #(x) denotes the total number of the elements n x. Apart from that, the frst word n the followng sentence s also ncluded to capture a common knd of long-dstance polarty ng structure: contrast transton. Thus, mportant trgger words lke however and but may be consdered. Fnally, Step E guarantees the balance between the two categores of the polarty ng tranng data. Gven the polarty ng tranng data, we apply SVM classfcaton algorthm to tran a polarty-ng detector wth word ungram features. Input: The polarty classfcaton tranng data: the negatve sentmental document set D and the postve sentmental c 1 document set D c. 2 Output: The polarty ng tranng data: the polarty-uned sentence set S and the polarty- ed sentence set S. Procedure: A. Segment documents un D c 1 and D c 2 to sngle sentences S and S c 1 c 2. B. Apply feature selecton on the polarty classfcaton tranng data and get the ranked features, ( t,..., t,..., t ) top 1 top top N C. S = {}, S un = {} D. For t n top ( ttop 1,..., t..., ttop N ) : D1) f #( S )> N max : break D2) Collect all sentences Stop, c and 1 whch contan ttop from S c 1 and respectvely D3) f #( S )>#( S ): c1 put put else: put put c2 c2 S nto S S nto S un c1 S nto S c1 S nto S un c2 Stop, c2 E. Randomly select Nmax sentences from Sun as the output of S un Fgure 2: The algorthm for automatcally generatng the polarty ng tranng data S c 2 638

5 3.2 Polarty Classfcaton wth Classfer Combnaton After polarty ng detecton, each document n the polarty classfcaton tranng data s dvded nto two parts, one contanng polarty-ed sentences and the other contanng polarty-uned sentences, whch are used to form the polarty-ed tranng data and the polarty-uned tranng data. In ths way, two dfferent polarty classfers, f and f 2, can be traned on the polarty-ed tranng data and the polarty-uned tranng data respectvely. Along wth classfer f 3, traned on all orgnal polarty classfcaton tranng data, we now have three base classfers n hand for possble classfer combnaton va a multple classfer system. The key ssue n constructng a multple classfer system (MCS) s to fnd a sutable way to combne the outputs of the base classfers. In MCS lterature, varous methods are avalable for combnng the outputs, such as fxed rules ncludng the votng rule, the product rule and the sum rule (Kttler et al., 1998) and traned rules ncludng the weghted sum rule (Fumera and Rol, 2005) and the meta-learnng approaches (Vlalta and Drss, 2002). In ths study, we employ the product rule, a popular fxed rule, and stackng (Džerosk and Ženko, 2004), a well-known traned rule, to combne the outputs. Formally, each base classfer provdes some knd of confdence measurements, e.g., posteror probabltes of the test sample belongng to each class. Formally, each base classfer fl ( l = 1, 2,3) assgns a test sample (denoted as x l ) a posteror probablty vector P( x l ) : P( x ) = ( p( c x ), p( c x )) t l 1 l 2 l where p( c1 x l ) denotes the probablty that the l-th base classfer consders the sample belongng c 1. The product rule combnes the base classfers by multplyng the posteror possbltes and usng the multpled possblty for decson,.e. assgn y c when j = arg max p( c x ) j l l= 1 Stackng belongs to well-known meta-learnng (Vlalta and Drss, 2002). The 3 I key dea behnd meta-learnng s to tran a meta-classfer wth nput attrbutes that are the outputs of the base classfers. Hence, meta-learnng usually needs some development data for generatng the meta-tranng data. Let x ' denote a feature vector of a sample from the development data. The output of the l-th base classfer f on ths sample s the probablty l dstrbuton over the category set { c1, c 2},.e. P( x' l ) = ( p( c1 x' l ), pl ( c2 x' l )) A meta-classfer can be traned usng the development data wth the meta-level feature meta 2 3 vector x R meta x = ( P( x' l= 1), P( x' l= 2), P( x' l= 3)) Stackng s a specfc meta-learnng rule, n whch a leave-one-out or a cross-valdaton procedure on the tranng data s appled to generate the meta-tranng data nstead of usng extra development data. In our experments, we perform stackng wth 10-fold cross-valdaton to generate the meta-tranng data. 4 Expermentaton 4.1 Expermental Settng The experments are carred out on product revews from four domans: books, DVDs, electroncs, and ktchen applances (Bltzer et al., 2007) 2. Each doman contans 1000 postve and 1000 negatve revews. For sentment classfcaton, all classfers ncludng the polarty ng detector, three base classfers and the meta-classfer n stackng are traned by SVM usng the SVM-lght tool 3 wth Logstc Regresson method for probablty measurng (Platt, 1999). In all the experments, each dataset s randomly and evenly splt nto two subsets: 50% documents as the tranng data and the remanng 50% as the test data. The features nclude word ungrams and bgrams wth Boolean weghts. 4.2 Expermental Results on Polarty Shftng Data To better understand the polarty ng phenomena n document-level sentment classfcaton, we randomly nvestgate Ths data set s collected by Bltzer et al. (2007): 3 It s avalable at: 639

6 polarty-ed sentences, together wth ther contexts (.e. the sentences before and after t), automatcally generated by the WFO ( λ = 0 ) feature selecton method. We fnd that nearly half of the automatcally generated polartyed sentences are actually polarty-uned sentences or dffcult to decde. That s to say, the polarty ng tranng data s nosy to some extent. One man reason s that some automatcally selected trgger words do not really contan sentment nformaton, e.g., hear, nformaton etc. Another reason s that some reversed opnon s gven n a revew wthout any explct polarty ng structures. To gan more nsghts, we manually checked 100 sentences whch are explctly polarty-ed and can also be judged by human accordng to ther contexts. Table 1 presents some typcal structures causng polarty ng. It shows that the most common polarty ng type s Explct Negaton (37%), usually expressed by trgger words such as not, no, or wthout, e.g., n the sentence I am not happy wth ths flashcard at all. Another common type of polarty ng s Contrast Transton (20%), expressed by trgger words such as however, e.g., n the sentence It s large and stylsh, however, I cannot recommend t because of the ld. Other less common yet productve polarty ng types nclude Excepton and Untl. Excepton structure s usually expressed by the trgger phrase the only to ndcate the one and only advantage of the product, e.g., n the sentence The only thng that I lke about t s that bamboo s a renewable resource. Untl structure s often expressed by the trgger word untl to show the reversed polarty, e.g. n the sentence Ths unt was a great addton untl the probe went bad after only a few months. Polarty Shftng Structures Trgger Words/Phrases Dstrbuton (%) Explct Negaton not, no, wthout 37 Contrast Transton but, however, 20 unfortunately Implct Negaton avod, hardly, 7 False Impresson look, seem 6 Lkelhood probably, perhaps 5 Counter-factual should, would 5 Excepton the only 5 Untl untl 3 Table 1: Statstcs on varous polarty ng structures 4.3 Expermental Results on Polarty Classfcaton For comparson, several classfers wth dfferent classfcaton methods are developed. 1) Baselne classfer, whch apples SVM wth all ungrams and bgrams. Note that t also serves as a base classfer n the followng combned classfers. 2) Base classfer 1, a base classfer for the classfer combnaton method. It works on the polarty-uned data. 3) Base classfer 2, another base classfer for the classfer combnaton method. It works on the polarty-ed data. 4) Negaton classfer, whch apples SVM wth all ungrams and bgrams plus negaton bgrams. It s a natural extenson of the baselne classfer wth the consderaton of negaton bgrams. In ths study, the negaton bgrams are collected usng some negaton trgger words, such as not and never. If a negaton trgger word s found n a sentence, each word n the sentence s attached wth the word _not to form a negaton bgram. 5) Product classfer, whch combnes the baselne classfer, the base classfer 1 and the base classfer 2 usng the product rule. 6) Stackng classfer, a combned classfer smlar to the Product classfer. It uses the stackng classfer combnaton method nstead of the product rule. Please note that we do not compare our approach wth the one as proposed n Ikeda et al. (2008) due to the absence of a manually-collected sentment dctonary. Besdes, t s well known that a combnaton strategy tself s capable of mprovng the classfcaton performance. To justfy whether the mprovement s due to the combnaton strategy or our polarty ng detecton or both, we frst randomly splt the tranng data nto two portons and tran two base classfers on each porton, then apply the stackng method to combne them along wth the baselne classfer. The correspondng results are shown as Random+Stackng n Table 2. Fnally, n our experments, t-test s performed to evaluate the sgnfcance of the performance mprovement between two systems employng dfferent methods (Yang and Lu, 1999). 640

7 Doman Baselne Base Classfer 1 Base Classfer 2 Negaton Classfer Random + Stackng Shftng + Product Shftng + Stackng Book DVD Electronc Ktchen Table 2: Performance comparson of dfferent classfers wth equally-splttng between tranng and test data Performance comparson of dfferent classfers Table 2 shows the accuracy results of dfferent methods usng 2000 polarty ed sentences and 2000 polarty-uned sentences to tran the polarty ng detector (N max =2000). Compared to the baselne classfer, t shows that: 1) The base classfer 1, whch only uses the polarty-uned sentences as the tranng data, acheves smlar performance. 2) The base classfer 2 acheves much lower performance due to much fewer sentences nvolved. 3) Includng negaton bgrams usually allows nsgnfcant mprovements (p-value>0.1), whch s consstent wth most of prevous works (Pang et al., 2002; Kennedy and Inkpen, 2006). 4) Both the product and stackng classfers wth polarty ng detecton sgnfcantly mprove the performance (p-value<0.05). Compared to the product rule, the stackng classfer s preferable, probably due to the performance unbalance among the ndvdual classfers, e.g., the performance of the base classfer 2 s much lower than the other two. Although stackng wth two randomly generated base classfers,.e. Random + Stackng, also consstently outperforms the baselne classfer, the mprovements are much lower than what has been acheved by our approach. Ths suggests that both the classfer combnaton strategy and polarty ng detecton contrbute to the overall performance mprovement. Effect of WFO feature selecton method Fgure 3 presents the accuracy curve of the stackng classfer when usng dfferent Lambda ( λ ) values n the WFO feature selecton method. It shows that those feature selecton methods whch prefer frequency nformaton, e.g., MI and BNS, are better n automatcally generatng the polarty ng tranng data. Ths s reasonable snce hgh frequency terms, e.g., s, t, a, etc., tend to obey our assumpton that the real polarty of one top term should belong to the polarty category where the term appears frequently Performance of the Stackng Classfer Lambda= Book DVD Electronc Ktchen Fgure 3: Performance of the stackng classfer usng WFO wth dfferent Lambda ( λ ) values Performance of the Stackng Classfer Book DVD Electronc Ktchen Fgure 4: Performance of the stackng classfer over dfferent szes of the polarty ng tranng data (wth N max sentences n each category) Effect of a classfer over dfferent szes of the polarty ng tranng data Another factor whch mght nfluence the overall performance s the sze of the polarty ng tranng data. Fgure 4 presents the overall performance on dfferent numbers of the polarty ng sentences when usng the stackng classfer. It shows that 1000 to 4000 sentences are enough for the performance mprovement. When the number s too large, the nosy tranng data may harm polarty ng detecton. When the number s too small, t s not enough for the automatcally generated polarty ng tranng data to capture varous polarty ng structures. 641

8 Doman: Book Doman: DVD % 40% 50% 60% 70% 80% 90% 100% The tranng data szes % 40% 50% 60% 70% 80% 90% 100% The tranng data szes 0.9 Doman: Electronc 0.9 Doman: Ktchen % 40% 50% 60% 70% 80% 90% 100% The tranng data szes % 40% 50% 60% 70% 80% 90% 100% The tranng data szes Baselne BaseClassfer 1 BaseClassfer 2 Stackng Fgure 5: Performance of dfferent classfers over dfferent szes of the polarty classfcaton tranng data Effect of dfferent classfers over dfferent szes of the polarty classfcaton tranng data Fgure 5 shows the classfcaton results of dfferent classfers wth varyng szes of the polarty classfcaton tranng data. It shows that our approach s able to mprove the overall performance robustly. We also notce the bg dfference between the performance of the baselne classfer and that of the base classfer 1 when usng 30% tranng data n Book doman and 90% tranng data n DVD doman. Detaled exploraton of the polarty ng sentences n the tranng data shows that ths dfference s manly attrbuted to the poor performance of the polarty ng detector. Even so, the stackng classfer guarantees no worse performance than the baselne classfer. 5 Concluson and Future Work In ths paper, we propose a novel approach to ncorporate polarty ng nformaton nto document-level sentment classfcaton. In our approach, we frst propose a machne-learnng-based classfer to detect polarty ng and then apply two classfer combnaton methods to perform polarty classfcaton. Partcularly, the polarty ng tranng data s automatcally generated through a feature selecton method. As shown n our expermental results, our approach s able to consstently mprove the overall performance across dfferent domans and tranng data szes, although the automatcally generated polarty ng tranng data s prone to nose. Furthermore, we conclude that those feature selecton methods, whch prefer frequency nformaton, e.g., MI and BNS, are good choces for generatng the polarty ng tranng data. In our future work, we wll explore better ways n generatng less-nosy polarty ng tranng data. In addton, snce our approach s language-ndependent, t s readly applcable to sentment classfcaton tasks n other languages. For avalablty of the automatcally generated polarty ng tranng data, please contact the frst author (for research purpose only). Acknowledgments Ths research work has been partally supported by Start-up Grant for Newly Apponted Professors, No. 1-BBZM n the Hong Kong Polytechnc Unversty and two NSFC grants, No and No We also thank the three anonymous revewers for ther helpful comments. 642

9 References Bltzer J., M. Dredze, and F. Perera Bographes, Bollywood, Boom-boxes and Blenders: Doman Adaptaton for Sentment Classfcaton. In Proceedngs of ACL-07. Dasgupta S. and V. Ng Mne the Easy and Classfy the Hard: Experments wth Automatc Sentment Classfcaton. In Proceedngs of ACL-IJCNLP-09. Dng X., B. Lu, and P. Yu A Holstc Lexcon-based Approach to Opnon Mnng. In Proceedngs of the Internatonal Conference on Web Search and Web Data Mnng, WSDM-08. Džerosk S. and B. Ženko Is Combnng Classfers wth Stackng Better than Selectng the Best One? Machne Learnng, vol.54(3), pp , Forman G An Extensve Emprcal Study of Feature Selecton Metrcs for Text Classfcaton. The Journal of Machne Learnng Research, 3(1), pp Fumera G. and F. Rol A Theoretcal and Expermental Analyss of Lnear Combners for Multple Classfer Systems. IEEE Trans. PAMI, vol.27, pp , 2005 Ikeda D., H. Takamura, L. Ratnov, and M. Okumura Learnng to Shft the Polarty of Words for Sentment Classfcaton. In Proceedngs of IJCNLP-08. Kennedy, A. and D. Inkpen Sentment Classfcaton of Move Revews usng Contextual Valence Shfters. Computatonal Intellgence, vol.22(2), pp , Km S. and E. Hovy Determnng the Sentment of Opnons. In Proceedngs of COLING-04. Kttler J., M. Hatef, R. Dun, and J. Matas On Combnng Classfers. IEEE Trans. PAMI, vol.20, pp , 1998 L S., R. Xa, C. Zong, and C. Huang A Framework of Feature Selecton Methods for Text Categorzaton. In Proceedngs of ACL-IJCNLP-09. L S. and C. Zong Mult-doman Sentment Classfcaton. In Proceedngs of ACL-08: HLT, short paper. Lu B., M. Hu, and J. Cheng Opnon Observer: Analyzng and Comparng Opnons on the Web. In Proceedngs of WWW-05. Na J., H. Su, C. Khoo, S. Chan, and Y. Zhou Effectveness of Smple Lngustc Processng n Automatc Sentment Classfcaton of Product Revews. In Conference of the Internatonal Socety for Knowledge Organzaton (ISKO-04). Pang B. and L. Lee A Sentmental Educaton: Sentment Analyss usng Subjectvty Summarzaton based on Mnmum Cuts. In Proceedngs of ACL-04. Pang B., L. Lee, and S. Vathyanathan Thumbs up? Sentment Classfcaton usng Machne Learnng Technques. In Proceedngs of EMNLP-02. Platt J Probablstc Outputs for Support Vector Machnes and Comparsons to Regularzed Lkelhood Methods. In: A. Smola, P. Bartlett, B. Schoelkopf and D. Schuurmans (Eds.): Advances n Large Margn Classers. MIT Press, Cambrdge, Polany L. and A. Zaenen Contextual Valence Shfters. Computng atttude and affect n text: Theory and applcaton. Sprnger Verlag. Rloff E., S. Patwardhan, and J. Webe Feature Subsumpton for Opnon Analyss. In Proceedngs of EMNLP-06. Turney P Thumbs Up or Thumbs Down? Semantc Orentaton Appled to Unsupervsed Classfcaton of Revews. In Proceedngs of ACL-02. Vlalta R. and Y. Drss A Perspectve Vew and Survey of Meta-learnng. Artfcal Intellgence Revew, 18(2), pp Wan X Co-Tranng for Cross-Lngual Sentment Classfcaton. In Proceedngs of ACL-IJCNLP-09. Webe J Learnng Subjectve Adjectves from Corpora. In Proceedngs of AAAI Wlson T., J. Webe, and P. Hoffmann Recognzng Contextual Polarty: An Exploraton of Features for Phrase-Level Sentment Analyss. Computatonal Lngustcs, vol.35(3), pp , Yang Y. and X. Lu, X A Re-Examnaton of Text Categorzaton methods. In Proceedngs of SIGIR-99. Yang Y. and J. Pedersen A Comparatve Study on Feature Selecton n Text Categorzaton. In Proceedngs of ICML