ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING

Transcription

1 ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING Pravesh Kumar Sngh 1, Mohd Shahd Husan 2 1 M.Tech, Department of Computer Scence and Engneerng, Integral Unversty, Lucknow, Inda erpraveshkumar@gmal.com 2 Assstant Professor, Department of Computer Scence and Engneerng, Integral Unversty, Lucknow, Inda sddqusahl@gmal.com ABSTRACT Although opnon mnng s n a nascent stage of development but stll the ground s set for dense growth of researches n the feld. One of the mportant actvtes of opnon mnng s to extract opnons of people based on characterstcs of the obect under study. Feature extracton n opnon mnng can be done by varous ways lke that of clusterng, support vector machnes etc. Ths paper s an attempt to apprase the varous technques of feature extracton. The frst part dscusses varous technques and second part makes a detaled apprasal of the maor technques used for feature extracton. KEYWORDS Opnon Mnng, Feature Extracton, Clusterng, Support Vector Machnes. 1. INTRODUCTION People are generally more nterested n other s opnon and when t comes to company product then t becomes even more mportant. So, the nformaton gatherng behavour makes us to collect and understand other s vews. Increasng networkng capabltes provde a way for surprsngly large multtude of resources that contans optons lke blogs, revews etc. As a result, now busnesses are tryng to analyse these floatng opnons for compettve advantages. Now wth the burst of actvtes n the area of sentment analyss they deal wth opnons n the frst class obect as there s ncreasng nterest n the systems of opnon mnng. Opnons Sentence Extracton Feature Extracton Sentence Analyss Sentence Opnon A fundamental step n opnon-mnng and sentmentanalyss applcatons s feature extracton. The combned opnon mnng procedure s llustrated n fgure 1. Several methods are used for feature extracton among whch, followng are the mportant ones: Overall Revew Score Calculaton Results and Revews 1) Naïve Bayes Classfer (NB) 2) Support Vector Machne (SVM) Fgure 1: Opnon Mnng Process Sundarapandan et al. (Eds) : ACITY, AIAA, CNSA, DPPR, NeCoM, WeST, DMS, P2PTM, VLSI pp , CS & IT-CSCP 2013 DOI : /cst

2 86 Computer Scence & Informaton Technology (CS & IT) 3) Mult Layer Perceptron (MLP) 4) Clusterng Classfer In ths paper, I have categorzed the work done for feature extracton and classfcaton n opnon mnng and sentment analyss. We have also tred to fnd out the performance, advantages and dsadvantages of dfferent technques. 2. DATA SETS Ths secton provdes bref detals of datasets used by us n our experments. 2.1 Product Revew Dataset The followng dataset s taken from the work of bltzer and s of mult-doman characterstcs. It conssts of product revews taken from amazon.com whch belongs to a total of 25 categores lke toys, vdeos, revews etc. From the randomly selected fve domans ve and 4000-ve revews are randomly samples. 2.2 Move Revew Dataset The second dataset s taken from the work of pang & lee (2004). It contans move revew wth of the feature of 1000+ve and 1000-ve processed move revews. Table 1: Result of Smple n-gram Move Revew Product Revews Ungram only Bgram Trgram (un+b) gram (un+b+tr) gram 3. CLASSIFICATION TECHNIQUES 3.1 Naïve Bayes Classfer Ths classfer s based on the probablty statement that was gven by Bayes. Ths theorem provdes condtonal probablty of occurrence of event E 1 when E 2 has already occurred, the vce versa can also be calculated by followng mathematcal statement. P(E2 E1)P(E1) P (E1 E 2) = E2 Ths bascally helps n decdng the polarty of data n whch opnons / revews / arguments can be classfed as postve or negatve whch s facltated by collecton of postve or negatve examples already fed. Naïve Bayes algorthm s mplemented to estmate the probablty of a data to be negatve or postve. The aforesad probablty s calculated by studyng postve and negatve examples & then calculatng the frequency of each pole whch s techncally termed as learnng. Ths learnng s actually supervsed as there s an exstence of examples 1. Thus, the condtonal probablty of a word wth postve or negatve meanng s calculated n vew of a plethora of postve and negatve examples & calculatng the frequency of each of class.

3 Computer Scence & Informaton Technology (CS & IT) 87 P(Sentment)P(Sentence Sentment) P (Sentment Sentence) = P(Sentence) Numberof word occurence nclass+ 1 So, P(Word Sentment ) = 1 Numberof wordsbelongngto a class+ Total nos of Word Algorthm s shown n fgure 2: +ve Sentence Classfer Tranng Set Classfer ve Sentence Sentence Revew Classfer Book Revew Algorthm has followng steps Fgure 2 S1: Intalze P(postve) num popozt (postve)/ num_total_propozt S2: Intalze P(negatve) num popozt (negatve) / num_total_propozt S3: Convert sentences nto words for each class of {postve, negatve}: num_total_cuvnte for each word n {phrase} P(word class) < num_apart (word class) 1 num_cuv (class) + P (class) P (class) * P (word class) Returns max {P(pos), P(neg)} Evaluaton of Algorthm The measures used for algorthm evaluaton are: Accuracy Precson Recall Relevance Contngency table for analyss of algorthm: Relevant Irrelevant Detected Opnons True Postve (tp) False Postve (fp) Undetected Opnons False Negatve (fn) True Negatve (tn) Now, Precson = tp tp +fp Accuracy = tp + tn, F tp + tn + fp + fn = 2*Pr ecson*re call Pr ecson + Re call ; Recall = tp tp +fn 1 The concept s to actually a fact that each & every word n a label s already referred n learnng set.

4 88 Computer Scence & Informaton Technology (CS & IT) Accuracy [1] Ion SMEUREANU, Crstan BUCUR, tranng the naïve gauss algorthm on 5000 sentences and get accuracy where no of groups (n) s Advantages of Naïve Bayes Classfcaton Methods 1. Model s easy to nterpret 2. Effcent computaton Dsadvantage of Naïve Bayes Classfcaton Methods Assumptons of attrbutes beng ndependent, whch may not be necessarly vald. 3.2 Support Vector Machne (SVM) The basc goal of support vector machne s to search a decson boundary between two classes that s excellently far away from any pont n the tranng data 2. SVM develops a hyper planes or a set of hyper planes n nfnte dmenson space. Ths dstance from decson surface to closest data pont determnes the margn of classfer. So the hyper planes act as decson surface whch act as crtera to decde the dstance of any data pont from t. The margn of classfer s calculated by the dstance from the closest data pont. Ths successfully creates a classfcaton but a slght error wll not cause a msclassfcaton. φ Input Space Feature Space Separatng Hyperplane For a tranng data set D, a set of n ponts: D = n p {( x,c ) x ε R,c ε{ 1,1 }...(1) =1 Where, x s a p-dmensonal real vector. Fnd the maxmum-margn hyper plane.e. splts the ponts havng c = 1 from those havng c = -1. Any hyperplane can be wrtten as the set of ponts satsfyng: w x - b =1...(2) Fndng a maxmum margn hyperplane, reduces to fndng the par w and b, such that the dstance between the hyperplanes s maxmal whle stll separatng the data. These hyperplanes (fg. 1) are descrbed by: w x -b= 1 and w x -b = -1 Fgure 3: Prncple of Support Vector Machne b The dstance between two hyperplanes s and therefore w needs to be mnmzed. Or we w can say that mnmze w n w, b subect to c (w.x - b) 1 for any = 1,, n. Usng Lagranges multplers (α ) ths optmzaton problem can be expressed as:

5 Computer Scence & Informaton Technology (CS & IT) 89 mn w, b max α 1 { 2 w - n 2 α [c (w.x - b) -1] }...( 3) 2 = Extensons of SVM To make SVM more robust and also more adaptable to real world problems, t has some extensons, some of whch nclude followng: 1. Soft Margn Classfcaton For very hgh dmensonal problems, common n text classfcaton, sometmes, data are lnearly separable. For multdmensonal problems lke n classfcaton of text, data are lnearly separable sometmes. But n most cases, the opnon soluton s the one that classfes most of the data and gnore some outlers or nosy data. If the tranng set D cannot be separated clearly then the soluton s to have fat decson classfers and make some mstake. Mathematcally, a stack varable ξ are ntroduced that are not equal to zero whch allow x to not meet the margn requrements wth a cost.e., proportonal to ξ. 2. Non-lnear Classfcaton The basc lnear SVM was gven by Vapnk (1963) later on Bernhard Boser, Isabelle Guyon and Vapnk n 1992 pavd a way for non-lnear classfers by usng kernel to max. margn hyper planes. The only dfference from th kernel trck gven by Azerman s that every dot product s replaced by non-lnear kernel functon. The effectveness of SVM n ths case les n the selecton of the kernel and soft margn parameters. 3. Multclass SVM Generally SVM s applcable for two class tasks. But multclass problems can be deal wth multclass SVM. In ths case labels are desgned to nstances whch are drawn from a fnte set of varous elements. These bnary classfers can be bult by two classfers lke by ether dstngush one versus all labels or between every par of classes one versus one Accuracy Usenet revews were classfed by pang by usng numercal ratngs whch were accompaned as basc truth. Varous learnng methods are used but ungrams gave best outputs n a presence based frequency model run by SVM. The accuracy undergone s the process was 82.9% Advantages of Support Vector Machne Method 1. Very good performance on expermental results 2. Low dependency on data set dmensonalty Dsadvantages of Support Vector Machne Method 1. Categorcal or mssng values need to be pre-processed. 2. Dffcult nterpretaton of resultng model. 2 Possbly dscountng some ponts as outlers or nose.

6 90 Computer Scence & Informaton Technology (CS & IT) 3.3 Mult-Layer Perceptron (MLP) MLP s a neural network whch s feed forward wth one or more layers between nput and output. Feed forward mples that, data flows n one drecton.e., from nput layer to output layer (.e., n forward drecton). Ths ANN whch multlayer perceptron starts wth nput layer where each node or neuron means a predcator varable. Input neurons are connected wth each neuron n hdden layers. The neurons n hdden layer are n turn connected to neuron n other hdden layers. The output layer s made up of one neuron n case of bnary predcton or more than one neuron n case of non-bnary predcton. Such arrangement makes a streamlned flow of nformaton from nput layer to output layer. MLP technque s qute popular owng to the fact that t can act as unversal functon approxmator. A back propagaton network has at least one hdden layer wth many non-lnear unts that can learn any functon or relatonshp between group of nput varable whether dscrete and for contnuous and output varable whether dscrete and for contnuous. Ths makes the technque of MLP qute general, flexble ad non-lnear tools. When output layers s to be classfed that has total number of nodes as total number of classes and the node havng hghest value then t gves the output.e., estmate of a class for whch an nput s made. If there s a specal case of two classes than, generally there s a node n output layer and classfcaton s carred between two classes and s done by applyng cut off pont to node value. An advantages of ths technque, compared to classcal modellng technques, s that t does not mpose any sort of restrcton wth respect to the startng data (type of functonal relatonshp between varables), nether does t usually start from specfc assumptons (lke the type of dstrbuton the data follow). Another vrtue of the technque les n ts capacty to estmate good models even despte the exstence of nose n the nformaton analyzed, as occurs when there s a presence of omtted values or outler values n the dstrbuton of the varables. Hence, t s a robust technque when dealng wth problems of nose n the nformaton presented; however, ths does not mean that the cleanng crtera of the data matrx should be relaxed Accuracy Ludmla I. Kuncheva, Member, IEEE on health care data calculated accuracy of Multlayer perceptron (MLP) as % Advantages of MLP 1) Capable of actng as a unversal functon approxmator. 2) Capablty to learn almost any relatonshp between nput and output varables Dsadvantages of MLP 1) Flexblty les n the need to have suffcent tranng data and that t requres more tme for ts executon than other technques. 2) It s somewhat consdered as complex black box. 3.4 Clusterng Classfer To dentfyng the promnent features of human and obects and recognzng them wth a type one can requre the obect clusterng. Bascally clusterng s unsupervsed learnng technque.

7 Computer Scence & Informaton Technology (CS & IT) 91 The obectve of clusterng s to determne a fresh or dfferent set of classes or categores, the new groups are of concern n themselves, and ther valuaton s ntrnsc. In ths method, data obects or nstances groups nto subset n such a method that smlar nstances are grouped together and varous dfferent nstances belong to the dfferent groups. Clusterng s an unsupervsed learnng task, so no class values representng a former combnaton of the data nstances are gven (stuaton of supervsed learnng). Fgure 4 Clusterng bascally assembles data obects or nstances nto subset n such a method that smlar obects are assembled together, whle dfferent obects belong to dfferent groups. The obects are thereby prepared or organzed nto effcent representatons that characterze the populaton beng sampled. Clusterng organzaton s denoted as a set of subsets C = C 1... C k of S, such that: k S= U C and C C = φ for. Therefore, any obect n S related to exactly one and only one subset. = 1 For example, consder the fgure 4 where data set has three normal clusters. Now consder the some real-lfe examples for llustratng clusterng: Example 1: Consder the people havng smlar sze together to make small and large shrts. 1. Talor-made for each person: expensve 2. One-sze-fts-all: does not ft all. Example 2: In advertsng, segment consumers accordng to ther smlartes: To do targeted advertsng. Example 3: To create a topc herarchy, we can take a group of text and organze those texts accordng to ther content matches. There are two man types of measures used to estmate ths relaton: dstance measures and smlarty measures. Bascally followng are two knds of measures used to guesstmate ths relaton 1. Dstance measures and 2. Smlarty measures Dstance Measures To conclude the smlarty and dfference between the par of nstances, many clusterng approaches usage the dstance measures. It s convenent to represent the dstance between two nstances let say x and x as: d (x,x ). A vald dstance measure should be symmetrc and gans ts mnmum value (usually zero) n case of dentcal vectors. Dstance measures method s known as metrc dstance measure f follow the followng propertes:

8 92 Computer Scence & Informaton Technology (CS & IT) 1.Tranglenequaltyd(x, x ) d(x, x ) + d(x, x ) 2.d(x, x ) = 0 x = x k x, x, x x, x k S S There are varatons n dstance measures dependng upon the attrbute n queston Clusterng Methods k A number of clusterng algorthms are gettng popular. The basc reason of a number of clusterng methods s that cluster s not accurately defned (Estvll-Castro, 2000). As a result many clusterng methods have been developed, usng a dfferent nducton prncple. Farley and Raftery (1998) gave a classfcaton of clusterng methods nto two man groups: herarchcal and parttonng methods. Han and Kamber (2001) gave a dfferent categorzaton: densty-based methods, model-based clusterng and grdbased methods. A totally dfferent categorzaton based on the nducton prncple s gven by (Estvll-Castro, 2000). 1. Herarchcal Methods Accordng to ths method clusters are created by recursve parttonng of the nstances n ether a top-down or bottom-up way. 2. Parttonng Methods Here, startng from the frst (startng) partton, nstances are rearranged by changng the poston of nstances from one cluster to another cluster. The basc assumpton s that the number of clusters wll be set by the user before. In order to get global optmalty n parttoned-based clusterng, an exhaustve enumeraton process of all possble parttons s done. 3. Densty-based Methods The bass of ths method s probablty. It suggests that the nstances that belong to each cluster are taken from a specfc probablty dstrbuton. The entre dstrbuton of the data s supposed to be a combnaton of numerous dssemnatons. The obectve of these methods s to dentfy the clusters and ther dstrbuton parameters. Ths partcular method s desgned n order to dscover clusters of arbtrary shape whch are not necessarly convex. 4. Grd-based Methods In grd-based method all the operatons for clusterng are performed n a grd structure and for performng n the grd structure all the avalable space s dvded nto a fxed number of cells. The basc man advantage of ths method s ts speed (Han and Kamber, 2001). 5. Soft-computng Methods Ths ncludes technques lke that of neural networks Evaluaton Crtera Measures for Clusterng Technque Generally, these crtera are splt nto two groups named Internal and External. 1. Internal Qualty Crtera Ths crtera generally measures compactness of clusters usng smlarty measures. It generally takes nto consderaton ntra-cluster homogenety, the nter-cluster separablty or a combnaton of these two. It doesn t use any exteror nformaton besde the data tself.

9 2. External Qualty Crtera Computer Scence & Informaton Technology (CS & IT) 93 They are useful for examnng the structure of the clusters match to some already defned classfcaton of the obects Accuracy Several dfferent classfers were used and the accuracy of the best classfer vared from 99.57% to 65.33%, dependng on the data Advantages of Clusterng Method The man advantage of ths method s that t offers the groups that (approxmately) fulfl an optmalty measure Dsadvantages of Clusterng Method 1. There s no learnng set of labelled observatons. 2. Number of groups s usually unknown. 3. Implctly, users already chooses the approprate features and dstance measure. 4. CONCLUSION The mportant part to gather nformaton always seems as what the people thnk. The rsng accessblty of opnon rch resources such as onlne analyss webstes and blogs rses as one can smply search and recognze the opnons of some other one. One can precse hs/her deas and opnons concernng goods and facltes. These vew and thoughts are subectve fgures whch sgnfes someone opnons, sentments, emotonal state or evaluaton. In ths paper, we present dfferent methods for data (feature or text) extracton and every method have some benefts and lmtatons and one can use these methods accordng to the stuaton for feature and text extracton. Based on the survey we can fnd the accuracy of dfferent methods n dfferent data set usng n- gram feature shown n table 2. Ngram Feature Table 2: Accuracy of Dfferent Methods Move Revews Product Revews NB MLP SVM NB MLP SVM Accordng to our survey, accuracy of MLP s better than other three methods when we use Ngram feature. The four methods dscussed n the paper are actually applcable n dfferent areas lke clusterng s appled n move revews and SVM technques s appled n bologcal revews & analyss. Although n the feld of opnon mnng s new, but stll the dverse methods avalable to provde a way to mplement these methods n varous programmng languages lke PHP, Python etc. wth an outcome of nnumerable applcatons. From a convergent pont of vew Naïve Bayes s best sutable for textual classfcaton, clusterng for consumer servces and SVM for bologcal readng and nterpretaton.

10 94 Computer Scence & Informaton Technology (CS & IT) ACKNOWLEDGEMENTS Every good wrtng requres the help and support of many people for t to be truly good. I would take the opportunty of thankng all those who extended a helpng hand whenever I needed one. I offer my heartfelt grattude to Mr. Mohd. Shahd Husan who encouraged, guded and helped me a lot n the proect. I extent my thanks to Mss. Ratna Sngh (fancee) for her ncandescent help to complete ths paper. A vote of thanks to my famly for ther moral and emotonal support. Above all utmost thanks to the Almghty God for the dvne nterventon n ths academc endeavour. REFERENCES [1] Ion SMEUREANU, Crstan BUCUR, Applyng Supervsed Opnon Mnng Technques on Onlne User Revews, Informatca Economcă vol. 16, no. 2/2012. [2] Bo Pang and Lllan Lee, Opnon Mnng and Sentment Analyss, Foundatons and TrendsR_ n Informaton Retreval Vol. 2, Nos. 1 2 (2008). [3] Abbas, Affect ntensty analyss of dark web forums, n Proceedngs of Intellgence and Securty Informatcs (ISI), pp , [4] K. Dave, S. Lawrence & D. Pennock. \Mnng the Peanut Gallery: Opnon Extracton and Semantc Class_caton of Product Revews." Proceedngs of the 12th Internatonal Conference on World Wde Web, pp , [5] B. Lu. \Web Data Mnng: Explorng hyperlnks, contents, and usage data," Opnon Mnng. Sprnger, [6] B. Pang & L. Lee, \Seeng stars: Explotng class relatonshps for sentment categorzaton wth respect to ratng scales." Proceedngs of the Assocaton for Computatonal Lngustcs (ACL), pp ,2005. [7] Nlesh M. Shelke, Shrnwas Deshpande, Vlas Thakre, Survey of Technques for Opnon Mnng, Internatonal Journal of Computer Applcatons ( ) Volume 57 No.13, November [8] Ndh Mshra and C K Jha, Classfcaton of Opnon Mnng Technques, Internatonal Journal of Computer Applcatons 56 (13):1-6, October 2012, Publshed by Foundaton of Computer Scence, New York, USA. [9] Oded Z. Mamon, Lor Rokach, Data Mnng and Knowledge Dscovery Handbook Sprnger, [10] Bo Pang, Lllan Lee, and Shvakumar Vathyanathan. Sentment classfcaton usng machne learnng technques. In Proceedngs of the 2002 Conference on Emprcal Methods n Natural Language Processng (EMNLP), pages [11] Towards Enhanced Opnon Classfcaton usng NLP Technques, IJCNLP 2011, pages , Chang Ma, Thaland, November 13, 2011 Author Pravesh Kumar Sngh s a fne blend of strong scentfc orentaton and edtng. He s a Computer Scence (Bachelor n Technology) graduate from a renowned gurukul n Inda called Dr. Ram Manohar Loha Awadh Unversty wth excellence not only n academcs but also had flagshp n choreography. He mastered n Computer Scence and engneerng from Integral Unversty, Lucknow, Inda. Currently he s actng as Head MCA (Master n Computer Applcatons) department n Thakur Publcatons and also workng n the capacty of Senor Edtor.