Opinion Feature Extraction Using Class Sequential Rules

Transcription

1 Opiio Feature Extractio Usig Class Sequetial Rules Miqig Hu ad Big Liu Departmet of Computer Sciece Uiversity of Illiois at Chicago 851 South Morga Street Chicago, IL {mhu1, Abstract The paper studies the problem of aalyzig user commets ad reviews of products sold olie. Aalyzig such reviews ad producig a summary of them is very useful to both potetial customers ad product maufacturers. By aalyzig reviews, we mea to extract features of products (also called opiio features) that have bee commeted by reviewers ad determie whether the opiios are positive or egative. This paper focuses o extractig opiio features from Pros ad Cos, which typically cosist of short phrases or icomplete seteces. We propose a laguage patter based approach for this purpose. The laguage patters are geerated from Class Sequetial Rules (CSR). A CSR is differet from a classic sequetial patter because a CSR has a fixed class (or target). We propose a algorithm to mie CSR from a set of labeled traiig sequeces. To perform extractio, the mied CSRs are trasformed ito laguage patters, which are used to match Pros ad Cos to extract opiio features. Experimetal results show that the proposed approach is very effective. Itroductio 1 The Web has dramatically chaged the way that cosumers expressig their opiios. They ca ow post reviews of products at merchat sites (e.g., amazo.com ad c et.com), dedicated review sites (e.g., epiios.com), Iteret forums ad blogs. These reviews provide excellet sources of cosumer opiios o products, which are very useful to both potetial customers ad product maufacturers. Techiques are ow beig developed to exploit these sources to help compaies ad idividuals to gai such iformatio effectively ad easily (e.g., Hu ad Liu 2004). I this paper, we focus o cosumer reviews of products, which are of very similar ature to blogs with ugrammatical seteces, icomplete seteces (setece fragmets), short phrases, ad missig puctuatios. There are three mai review formats o the Web. Format (1) - Pros ad Cos: The reviewer is asked to describe Pros ad Cos separately. C et.com uses this format. Format (2) - Pros, Cos ad detailed review: The reviewer is asked to describe Pros ad Cos separately ad also write a detailed review. epiios.com ad MSN uses this format. Format (3) - free format: The reviewer writes freely, i.e., o separatio of Pros ad Cos. Amazo.com uses this format. Compilatio copyright 2006, America Associatio for Artificial Itelligece ( All rights reserved. I this paper, we propose to aalyze ad summarize reviews of format (2). We aim to idetify product features that have bee commeted o by customers i a set of reviews ad use them to summarize the reviews. To summarize reviews, we display the umber of positive ad egative reviews for each product feature, which show whether the customers like or dislike the feature. Note that for reviews of format (2), opiio orietatios (positive or egative) of features are kow as Pros ad Cos are separated. I (Hu ad Liu 2004), several techiques were proposed to idetify both product features ad their opiio orietatios from reviews of format (3). For format (3) (ad also (1)), reviewers typically use full seteces. However, for format (2), Pros ad Cos ted to be very brief. For example, uder Cos, oe may oly write: heavy, bad picture quality, battery life too short, which are elaborated i the detailed review. (Liu, Hu ad Cheg, 2005) proposed a iitial method to extract product features from Pros ad Cos based o associatio rules. However, associatio rule miig is ot suitable for this task because associatio rule miig is uable to cosider the sequece of words, which is very importat i atural laguage texts. Thus, may complex ad hoc post-processig methods are used i order to fid patters to extract features. I this work, we propose a more pricipled miig method based o sequetial patter miig. I particular, we mie a special kid of sequetial patters called Class Sequetial Rules (CSR). As its ame suggests, the sequece of words is cosidered automatically i the miig process. Ulike stadard sequetial patter miig, which is usupervised, we mie sequetial rules with some fixed targets or classes. Thus, the ew method is supervised. To our kowledge, this is the first work that mies ad uses such kid of rules. The mied CSRs are used to extract product features from Pros ad Cos i format (2). Note that we do ot aalyze detailed reviews i format (2) as they are elaboratios of Pros ad Cos. Aalyzig short segmets i Pros ad Cos produce more accurate results. Our experimetal results show that the proposed method is highly effective. Related Work I (Hu ad Liu 2004), some methods are proposed to aalyze customer reviews of format (3). However, sice reviews of format (3) are usually complete seteces, the techiques i (Hu ad Liu 2004) are thus ot suitable for Pros ad Cos of format (2). The work of [Popescu ad Etzioi 2005] also

2 works o complete seteces ad thus ot suitable for setece fragmets ad short phrase i Pros ad Cos. I (Liu, Hu ad Cheg, 2005), a method is also proposed to extract product feature from Pros ad Cos of format (2). However, as we discussed i the Itroductio sectio, the method is very complex ad rather ad hoc because associatio rules caot aturally capture word relatios. (Moriaga et al, 2002) compares differet products i a category through search to fid the products reputatio. It does ot aalyze reviews, ad does ot idetify product features. Below, we preset some other related research. Termiology Fidig: There are basically two techiques for termiology fidig: symbolic approaches that rely o ou phrases, ad statistical approaches that exploit the fact that words composig a term ted to be foud close to each other ad reoccurrig (Bourigault, 1995, Daille, 1996, Jacquemi, ad Bourigault, 2001, Justeso ad Katz, 1995). However, usig ou phrases teds to produce too may o-terms, while usig reoccurrig phrases misses may low frequecy terms, terms with variatios, ad terms with oly oe word. As show i (Hu ad Liu 2004) usig the existig termiology fidig system FASTR (FASTR) produces very poor results. Furthermore, usig ou phrases is ot sufficiet for fidig product features. We also eed to cosider other laguage compoets (e.g., verbs ad adjectives). Setimet Classificatio: Setimet classificatio classifies opiio texts or seteces as positive or egative. Work of (Hearst, 1992) o classificatio of etire documets uses models ispired by cogitive liguistics. (Das ad Che, 2001) uses a maually crafted lexico i cojuctio with several scorig methods to classify stock postigs. (Tog, 2001) geerates setimet timelies as it tracks olie discussios about movies. (Turey, 2002) applies a usupervised learig techique based o mutual iformatio betwee documet phrases ad the words excellet ad poor to fid idicative words of opiios for classificatio. (Pag, Lee ad Vaithyaatha, 2002) examies several supervised machie learig methods for setimet classificatio of movie reviews. (Dave, Lawrece ad Peock, 2003) also experimets a umber of learig methods for review classificatio. (Agrawal et al, 2003) fids that supervised setimet classificatio is iaccurate. They proposed a method based o social etwork for the purpose. However, social etworks are ot applicable to customer reviews. (Hatzivassiloglou ad Wiebe, 2000) ivestigates setece subjectivity classificatio. Other related works iclude (Nasukawa ad Yi, 2003, Nigam ad Hurst 2004, Riloff ad Wiebe, 2003, Wilso, Wiebe ad Hwa, 2004, Yu, ad Hatzivassiloglou, 2003). Our work differs from setimet ad subjectivity classificatio as they do ot idetify features commeted o by customers or what customers praise or complai about. Thus, we solve a related but differet problem. Problem Statemet We first describe the problem statemet, ad the discuss the ew automatic techique for idetifyig product features from Pros ad Cos i reviews of format (2). Let P be a product ad R = {r 1, r 2,, r k } be a set of reviews of P. Each review r j cosists of a list of Pros ad Cos. Defiitio (product feature): A product feature f i r j is a attribute/compoet of the product that has bee commeted o i r j. If f appears i r j, it is called a explicit feature i r j. If f does ot appear i r j but is implied, it is called a implicit feature i r j. For example, battery life i the followig opiio segmet is a explicit feature: Battery life too short Size is a implicit feature i the followig opiio segmet as it does ot appear i each setece but it is implied: Too small Figure 1 shows a review of format (2). Pros ad Cos are separated ad very brief. We do ot study full reviews as they basically elaborate o Pros ad Cos. Figure 1: A example review of format (2) The task: Our objective i this paper is to fid all the explicit ad implicit product features o which reviewers have expressed their (positive or egative) opiios. Class Sequetial Rules Miig We propose a supervised sequetial patter miig method to fid laguage patters to idetify opiio (product) features from Pros ad Cos. Let I = {i 1, i 2,, i } be a set of all items ad C = {c 1, c 2,, c m } be a set of class items ad C I. A sequece is a ordered list of items, deoted by <i 1 i 2 i l >. A sequece with legth l is called a l-sequece. A sequece s 1 = <a 1 a 2 a > is called a subsequece of aother sequece s 2 = <b 1 b 2 b m > or s 2 a supersequece of s 1, if there exist itegers 1 j 1 < j 2 < < j -1 j such that a 1 = b j1, a 2 = b j2,, a 1 = b j. A sequece database S is a set of tuples <sid, s>, where sid is a sequece_id ad s a sequece. A tuple is said to cotai a sequece s i, if s i is a subsequece of s. A class sequetial rule (CSR) is a implicatio of the form X Y, where X is a sequece <s 1 x 1 s 2 x 2 x l s r+1 > (s i = <> or <i 1 i 2 i k > ad i m C, ad x i deotes a possible class at this positio, x i I, for 1 i l) ad Y is a sequece <s 1 c k1 s 2 c k2 c kr s l+1 > (c ki C, for 1 i r). The support of a CSR csr i i a sequece database S is the umber of tuples i the database cotaiig Y. The cofidece of a CSR csr i is the support of Y divided by the support of X ad Y. A tuple <sid, s> is said to cover a CSR csr i, if X is a subsequece of s. A tuple is said to cotai a CSR csr i, if Y is a subsequece of s. Table 1 gives a example sequece database which has 5 tuples, with c 1 ad c 2 deotig the classes. We have a CSR

3 <<ab>x<gh>> <<ab>c 1 <gh>> with support of 2 ad cofidece of 2/3, as sequece 10 ad 50 cotais the rule while sequece 10, 20 ad 50 covers the rule. Give a sequece database, miimum support ad miimum cofidece as thresholds, class sequetial rule miig fids the complete set of class sequetial rules i the database. I this paper, we mie the class sequetial rules that have product features as class items. sequece_id sequece 10 <abdc 1 gh> 20 <abeghk> 30 <c 2 kea> 40 <dc 2 kb> 50 <abc 1 fgh> Table 1. A example of sequece database ClassPrefix-Spa: Class Sequetial Rules Miig We ow preset the algorithm ClassPrefix-Spa to mie class sequetial rules. Although there are several efficiet sequetial patter miig algorithms, oe of them addresses the specific problem of miig class sequetial rules. Here we adapt the patter growth method i (Pei et al, 2004) for the task, the geeral idea is outlied as follows: as we are oly iterested i patters that cotai classes, we first fid the patters that have classes as suffix. The takig the geerated patters as prefix, we ca fid all the class sequetial rules by patter growth. The algorithm recursively projects a sequece database ito a set of smaller databases associated with the prefix patter mied so far, ad the mies locally frequet patters i each projected database. Let us examie the proposed approach for miig CSRs based o our ruig example. Example. For the same sequece database S i Table 1 with miimum support = 2, CSRs i S ca be mied i the followig steps: 1. Divide search space for each class ad fid legth-1 patters. The complete set of patters ca be partitioed ito the followig three subsets accordig to the two classes: 1) the oes with class c 1 {sequece 10 ad 50}, 2) the oes with class c 2 {sequece 30 ad 40}, ad 3) the oes without ay class {sequece 20}. The legth-1 patters are: <c 1 >:2, <c 2 >:2 (2 is the support). 2. Fid subsets of suffix patters for each class. The subsets of patters that have class as suffixes ca be mied by costructig the correspodig sets of projected databases ad miig each recursively. The projected databases as well as suffix patters foud i them are listed i Table 2. a. Fid patters with suffix <c 1 >. Oly subsequeces edig with <c 1 > should be cosidered. For example, i <abdc 1 gh>, oly the subsequece <abd> should be cosidered for miig patters with suffix <c 1 >. The sequeces i S are projected with regards to <c 1 > to form the projected database, which cosists of two prefix sequeces: <abd> ad <ab>. By scaig the projected database for <c 1 >, its locally frequet items are a:2 ad b:2. Thus, all the legth-2 patters suffixed with <c 1 > are foud, ad they are: <ac 1 > ad <bc 1 >. Recursively, all patters with suffix <c 1 > ca be partitioed ito two subsets: 1) those suffixed with <ac 1 >, ad 2) those with <bc 1 >. These subsets ca be mied by costructig respective projected databases ad miig each recursively as follows: i. The projected database for <ac 1 > cosists of o subsequece. Thus the processig of this projected database termiates. ii. The projected database for <bc 1 > cosists of subsequeces edig with <bc 1 >: <a> ad <a>. Recursively miig the projected database returs oe patter: <abc 1 >. It forms the complete set of patters suffixed with <bc 1 >. b. Fid patters with suffix <c 2 >. This ca be doe by costructig projected databases ad miig them respectively. The projected databases ad the patters foud are show i Table 2. Class Sequece projected db Suffix patters c 1 <abdc 1 gh>, <abd>, <ab> <c 1 >, <ac 1 >, <abc 1 fgh> <bc 1 >, <abc 1 > c 2 <c 2 kea>, <>, <c 2 > <dc 2 kb> <d> o_class <abeghk> - - Table 2. Projected databases ad patters 3. Fid patters with geerated suffix patters as prefixes. The sequece database ca be partitioed ito five subsets accordig to the five geerated prefixes: 1) the oes with prefix <c 1 >, 2) the oes with prefix <ac 1 >,, ad 5) the oes with prefix <c 2 >. The projected databases ad the patters are listed i Table 3. prefix projected db Prefix patters <c 1 > <gh>, <fgh> <c 1 g>, <c 1 h>,<c 1 gh> <ac 1 > <gh>, <fgh> <ac 1 g>, <ac 1 h>, <ac 1 gh> <bc 1 > <gh>, <fgh> <bc 1 g>, <bc 1 h>,<bc 1 gh> <abc 1 > <gh>, <fgh> <abc 1 g>, <abc 1 h>, <abc 1 gh> <c 2 > <kea>, <kb> <c 2 k> Table 3. Projected databases ad patters a. Fid patters with prefix <c 1 >. Oly the subsequece prefixed with the occurrece of <c 1 > will be cosidered. The projected database for <c 1 > thus icludes two suffix sequeces: <gh> ad <fgh>. Note that the curret projected database for <c 1 > is differet from the database whe projected for geeratig patters i step 2.a. We are ow projectig forwards but previously we projected backwards. By scaig the projected database for <c 1 >, its locally frequet items are g:2 ad h:2. Thus, all the legth-2 patters prefixed with <c 1 > are foud, ad they are: <c 1 g> ad <c 1 h>.

4 Recursively, all patters with prefix <c 1 > ca be partitioed ito two subsets: 1) those prefixed with <c 1 g>, ad 2) those with <c 1 h>. These subsets ca be mied by costructig respective projected databases ad miig each recursively, i.e., geeratig legth-3 patter <c 1 gh>. b. Fid patters with prefix <ac 1 >, <bc 1 >, <abc 1 >, ad <c 2 >. This ca be doe by followig the same procedure as fidig patters with prefix <c 1 >. 4. Compute the cofidece of CSRs. The sequece database S is scaed to compute the cofidece of the rules. The algorithm is preseted as follows: Algorithm: ClassPrefix-Spa({c 1, c 2,, c }, S) Iput: {c 1, c 2,, c } is the set of classes of iterests, S is the sequece database. Output: complete set of class sequetial rules. Method: 1. Let csr_set = {}; 2. Sca S oce, for each class c i {c 1, c 2,, c } (a) costruct projected database S c backwards; (b) call Pre-Suf-fixSpa(c, 1, S c, backward ). 3. For each csr i csr_set (a) costruct projected database S csr forwards; (b) let l = legth of csr; (c) call Pre-Suf-fixSpa(csr, l, S csr, forward ). 4. Sca S oce, for each csr i csr_set, compute cofidece; 5. Retur csr_set. Subroutie Pre-Suf-fixSpa(s, l, S s, d, csr_set) Iput: s is a class sequetial rule; l is the legth of s; S s is the projected database of s; d is the directio of costructio of projected database: csr_set is for storig the geerated csrs. Method: 1. Sca S s oce, fid each frequet item a, such that (a) a ca be appeded to the last elemet of s to form a class sequetial rule if d = forward ; or (b) a ca be appeded to the first elemet of s to form a class sequetial rule if d = backward ; 2. For each frequet item a, apped to s to form a class sequetial rule s, ad isert s to csr_set. 3. For each a (a) costruct projected database S s backwards if d = backward, forwards if d = forward ; (b) call Pre-Suf-fixSpa(s, l+1, S s, d, csr_set). ClassPrefix-Spa is the mai algorithm for geeratig class sequetial rules. I step 2, we first costruct the projected databases for each class backwards, ad the use the projected databases to mie patters of the format i 1 i 2 class, that is, the patters with class as suffix. The growth directio is backward as we first fix the last item (class item) i the patter, the we grow the patter by appedig item i frot of the patter each time. I step 4, the mied patters so far are take as prefix, ad we grow the patters forward by appedig a item at the ed of the patter each time. The steps will geerate patters of the form i 1 i 2 class i l i l+1...i. I step 4, the database is scaed oce agai, to cout the coverage of each csr ad to calculate the cofidece. Pre-Suf-fixSpa is the fuctio that grows patters. Each time it appeds oe frequet item foud from curret projected database to the give patter to form a ew patter (step 1 ad 2). It the recursively costructs projected database for each frequet item ad mies ew patters (step 3). Due to space limitatios, we do ot elaborate the process of costructio of projected database, which is similar to that i (Pei et al, 2004). The major differece is that we ca project the database i two directios. Miig CSRs for Feature Extractio I this work, we aim to fid CSRs with the followig target classes: <NN> [feature], <JJ> [feature], <VB> [feature] ad <RB> [feature], which allow us to extract various types of product features (<NN>: ous, <VB>: verbs, <JJ>: adjectives, ad <RB>: adverbs). Our approach is based o the followig observatio: Each setece segmet i Pros ad Cos cotais at most oe product feature. Setece segmets are separated by,,., ;, -, &, ad, ad but. For example, Pros i Figure 1 ca be separated ito 5 segmets. great photos <photo> easy to use <use> good maual <maual> may optios <optio> takes videos <video> Cos i Figure 3 ca be separated ito 3 segmets: battery usage <battery> icluded software could be improved <software> icluded 16MB is stigy <16MB> <memory> We ca see that each segmet describes a product feature o which the reviewer has expressed a opiio (the last two ca be see as full seteces). The product feature for each segmet is listed withi <>. Notice that <16MB> is a value of feature <memory>, which is a implicit feature as it does ot appear i the setece segmet. Aother importat poit to ote is that a feature may ot be a ou or ou phrase, which is used i (Hu ad Liu 2004). Verbs may be features as well, e.g., use i easy to use. Of course, we ca also use its correspodig ou as the feature, e.g., usage or simply use. Give a maually labeled traiig review set, we perform the followig preprocessig before miig CSRs: 1. Perform Part-Of-Speech (POS) taggig ad remove digits ad some puctuatios: We use the NLProcessor liguistic parser (NLProcessor, 2000) to geerate the POS tag of each word. POS taggig is crucial as it allows us to geerate geeral laguage patters. We remove digits i seteces, e.g., chagig 16MB to MB. Digits ofte represet cocepts that are too specific to be used i rule discovery, which aims to geeralize. We use two examples from above to illustrate the results of this step: <NN> Battery <NN> usage <VB> icluded <NN> MB <VB>is <JJ> stigy <NN> idicates a ou, <VB> a verb, ad <JJ> a ad-

5 jective. 2. Replace the actual feature words i a setece with [feature]: This replacemet is ecessary because differet products have differet features. The replacemet esures that we ca fid geeral laguage patters which ca be used for ay product feature. After replacemet, the above two examples become: <NN> [feature] <NN> usage <VB> icluded <NN> [feature] <VB> is <JJ> stigy Note that MB is also replaced with [feature] as it idicates a implicit feature. It is possible that a feature may cotai more tha oe word, e.g., auto mode stiks, which will be chaged to <NN> [feature] <NN> [feature] <VB> stiks 3. Use -gram to produce shorter segmets from log oes: For example, <VB> icluded <NN> [feature] <VB> is <JJ> stigy will geerate 2 3-gram segmets: <JJ> icluded <NN> [feature] <VB> is <NN> [feature] <VB> is <JJ> stigy We oly use 3-grams (3 words with their POS tags) here, which works well. The reaso for usig -gram rather tha full seteces is because most product features ca be foud based o local iformatio ad POS taggig. Usig log seteces ted to geerate a large umber of spurious rules. 4. Perform word stemmig: This is performed as i iformatio retrieval tasks to reduce a word to its stem. After the four-step pre-processig ad labelig (taggig), the resultig setece (3-gram) segmets are saved i a file (called a trasactio file) for the geeratio of class sequetial patters. I this file, each lie cotais oe processed (labeled) setece segmet. We the use class sequetial patter miig to fid all laguage patters. We use 1% as the miimum support, but do ot set miimum cofidece. As the patters geerated are small i umber, further patter pruig by settig a miimum cofidece may cause some review segmets ot covered by ay patter. Experimetal result also idicates that usig miimum cofidece decreases the recall ad precisio. Two example rules are give below (we omit supports ad cofideces). (a) <NN> x <NN> x <NN> [feature] <NN> [feature] (b) <JJ> easy to <VB> x <JJ> easy to <VB> [feature] We observe that both POS tags ad words may appear i rules. We also ote that whe usig patter (b) for feature extractio, it may cause ambiguity, e.g., is <JJ> the POS tag for easy, or ay word i frot of easy? To tackle this problem, we do post-processig to reassemble the CSRs ito the followig four ew rules (we use patter (b) as example), (1) <JJ> -1, -1 easy, -1 to, <VB> x <JJ> -1, -1 easy, -1 to, <VB> [feature] (2) <JJ> easy, -1 to, <VB> x <JJ> easy, -1 to, <VB> [feature] (3) <JJ> -1, -1 easy, -1 to, <VB> -1, -1 x <JJ> -1, -1 easy, -1 to, <VB> -1, -1 [feature] (4) <JJ> easy, -1 to, <VB> -1, -1 x <JJ> easy, -1 to, <VB> -1, -1 [feature] Note that i the ew rules, we require each word to have its correspodig POS tag i frot of it. I the case that there is o POS tag attached with the word, we use 1 to idicate the do ot care situatio. Similarly, 1 is used whe we do ot care about the word but oly the word type. As show i the above example, each POS tag together with the followed word/[feature] refers to oe word i a setece (separated by comma). We cout support ad cofidece for each ew rule by scaig the sequece database agai. I this way, we ca have a set of laguage rules for feature extractio without ambiguity. Extractio of Product Features The resultig laguage rules are used to idetify product features from ew reviews after POS taggig. A few situatios eed to be hadled. 1. A geerated rule does ot ecessarily require matchig a part of a setece segmet with the same legth as the rule. I other words, we allow gaps for patter matchig. For example, rule (right-had-side oly) <NN> [feature], <NN> -1 ca match the segmet size of pritout. This is achieved by allowig user to set a value for the maximum legth that a patter could expad. We also allow user to set the maximum legth of review segmet that a patter should be applied. These two values eable a user expert to refie the patters for better extractig product features. However, i our experimets reported below, we did set ay of these values, i.e., o maual ivolvemet. 2. If a setece segmet satisfies multiple rules, we search for a matchig oe i the followig orders: rules of class <NN> [feature], the <JJ> [feature], <VB> [feature] ad lastly <RB> [feature]. Ad for rules of each class, we select the rule that gives the highest cofidece as higher cofidece idicates higher predictive accuracy. The reaso for this orderig is because as we observed that the ou features appear more frequetly tha other types. 3. For those setece segmets that o rule applies, we use ous or ou phrases produced by NLProcessor as features if such ous or ou phrases exist. Note that our rule miig method does ot apply to cases that a segmet oly has a sigle word, e.g., heavy ad big. I this case, we treat these sigle words as features. Experimet Results We ow evaluate the proposed automatic techique to see how effective it is i idetifyig product features from Pros ad Cos i reviews of format (2). We use the same data as [Liu, Hu ad Cheg, 2005) i our experimet. The data cosists of a traiig set ad a testig set. The traiig set has Pros ad Cos of te products. The test set has Pros ad Cos of five (differet) products. Usig the rules discovered from the traiig set, we extract features from the test set. We use recall (r) ad precisio (p) to evaluate the results, EC 1 i, ad EC 1 i, r = p = C i Ei 1 where is the total umber of reviews of a particular product, EC i is the umber of extracted features from review i that are 1

6 correct, C i is the umber of actual features i review i, E i is the umber of extracted features from review i. This evaluatio is based o the result of every review as it is crucial to extract features correctly from every review. We geerate laguage patters ad product features separately for Cos ad Pros as this produces better results. Table 4 shows the results. With recall averages at for Pros ad for Cos, it shows that the proposed extractio usig CSRs is highly effective. Table 5 compares the proposed techique of usig CSRs with the techique of usig associatio rules i (Liu, Hu ad Cheg, 2005). From the two tables, we ca see that the proposed techique geerates comparable results as the associatio rules. However, feature extractio usig associatio rules eeds a lot of extra post-processig ad maual ivolvemet as associatio rule miig is uable to cosider the sequece of words, which is very importat for atural laguage texts. The proposed feature extractio usig sequetial patter miig is thus a more pricipled techique. Pros Cos recall prec Recall prec data data data data data Avg Table 4: Recall ad precisio results of CSRs Pros Cos recall Prec. recall Prec. data data data data data Avg Table 5: Recall ad precisio results of associatio rules Coclusios ad Future Work Aalyzig reviews o the Web has may applicatios. It is ot oly importat for idividual cosumers, but also importat for product maufacturers. I this paper, we focused o oe type of product reviews, i.e., Pros ad Cos expressed as short phrases or setece segmets. Our objective was to extract opiio (product) features that have bee commeted o by cosumers. As the method i (Liu, Hu ad Cheg, 2005) is very ad hoc, we proposed a more appropriate miig method called class sequetial rule miig to perform the task which captures the sequetial relatioships of words i seteces. I our future works, we will further improve the results ad also study how to use the proposed method to aalyze reviews of full seteces. Refereces Agrawal, R., Rajagopala, S., Srikat, R., Xu, Y. Miig ewsgroups usig etworks arisig from social behavior. WWW 03. Bourigault, D. Lexter: A termiology extractio software for kowledge acquisitio from texts. KAW 95, Buescu, R., ad Mooey, R. Collective Iformatio Extractio with Relatioal Markov Networks. ACL-2004, Daille, B. Study ad Implemetatio of Combied Techiques for Automatic Extractio of Termiology. The Balacig Act: Combiig Symbolic ad Statistical Approaches to Laguage. MIT Press, Das, S. ad Che, M., Yahoo! for Amazo: Extractig market setimet from stock message boards. APFA 01, Dave, K., Lawrece, S., ad Peock, D. Miig the Peaut Gallery: Opiio Extractio ad Sematic Classificatio of Product Reviews. WWW 03, FASTR. Fellbaum, C. WordNet: a Electroic Lexical Database, MIT Press, Freitag, D ad McCallum, A. Iformatio extractio with HMM structures leared by stochastic optimizatio. AAAI-00, Hatzivassiloglou, V. ad Wiebe, J. Effects of adjective orietatio ad gradability o setece subjectivity. COLING 00, Hearst, M, Directio-based Text Iterpretatio as a Iformatio Access Refiemet. I P. Jacobs, editor, Text-Based Itelliget Systems. Lawrece Erlbaum Associates, Hu, M ad Liu, B. "Miig ad summarizig customer reviews". KDD-04, Jacquemi, C., ad Bourigault, D. Term extractio ad automatic idexig. I R. Mitkov, editor, Hadbook of Computatioal Liguistics. Oxford Uiversity Press, Justeso, J. & Katz, S. Techical Termiology: some liguistic properties ad a algorithm for idetificatio i text. Natural Laguage Egieerig 1(1):9-27, Liu, B., Hu, M. ad Cheg, J Opiio Observer: Aalyzig ad comparig opiios o the Web. WWW Moriaga, S., Yamaishi, K., Tateishi, K, ad Fukushima, T Miig Product Reputatios o the Web. KDD 02, Nasukawa, T. & Yi, J Setimet aalysis: Capturig favorability usig atural laguage processig. Proceedigs of the 2d Itl Cof. o Kowledge Capture (K-CAP 2003). Nigam, K. ad Hurst, M Towards a robust metric of opiio. AAAI Sprig Symp.o Explorig Attitude ad Affect i Text. NLProcessor, Pag, B., Lee, L., ad Vaithyaatha, S. Thumbs up? Setimet classificatio usig machie learig techiques. EMNLP-02. Popescu, A-M ad Etzioi. O "Extractig Product Features ad Opiios from Reviews. EMNLP-05. Pei, J. Ha, J., Mortazavi-Asl, B., Wag, J., Pito, H., Che, Q., Dayal, U., ad Hsu, M.-C. Miig Sequetial Patters by Patter-Growth: The PrefixSpa Approach. IEEE Trasactios o Kowledge ad Data Egieerig, 16(10), Riloff, E ad Wiebe, J Learig extractio patters for subjective expressios. EMNLP-03.. Tog, R A Operatioal System for Detectig ad Trackig Opiios i o-lie discussio. SIGIR 2001 Workshop o Operatioal Text Classificatio, Turey, P. Thumbs Up or Thumbs Dow? sematic orietatio applied to usupervised classificatio of reviews. ACL Wiebe, J., Bruce, R., O Hara, T. Developmet ad use of a gold stadard data set for subjectivity classificatios. ACL 99, Wilso, T, Wiebe, J, & Hwa, R. Just how mad are you? Fidig strog ad weak opiio clauses. AAAI-04, Yu, H ad Hatzivassiloglou, V. Towards aswerig opiio questios: Separatig facts from opiios ad idetifyig the polarity of opiio seteces. EMNLP-03, 2003.