Specialized Weighted Majority Statistical Techniques in Robotics (Fall 2009)

Transcription

1 Statstcal Technques n Robotcs (Fall 09) Keywords: classfer ensemblng, onlne learnng, expert combnaton, machne learnng Javer Hernandez Alberto Rodrguez Tomas Smon javerhe@andrew.cmu.edu albertor@andrew.cmu.edu tsmon@andrew.cmu.edu Abstract The problem that we address s that of forecastng results by combnng expert predctons. Standard ensemble methods do not explctly consder the performance of experts to be varable across the problem doman. In contrast, we propose Specalzed Weghted Majorty. Specalsts for each area of the feature space are created by augmentng experts wth a classfer that chooses on whch samples to vote. Our method can be seen as tradng off complexty of the obtaned solutons and the amount of tranng data requred. We compare the proposed method to Weghted Majorty and SVMs on synthetc and real data. 1. Introducton The problem that we to address s that of expert or classfer combnaton, especally for applcatons n whch the experts are humans provdng forecasts. For ths reason, n the followng dscusson we wll use the word experts ndstnctly when referrng to both experts and classfers. We are nterested n dervng a method that provdes constructve nterference between the experts n order to do a better job at predctng than any sngle expert, but also takes advantage of the characterstcs of our problem doman to solve a smpler problem. One can thnk of learnng methods as a soluton to the trade-off between complexty of soluton and speed of learnng. On one sde of the spectrum we have smple, fast learnng, and fast adaptng algorthms such as Wnnow and Verson of the paper for Statstcal Technques n Robotcs. Weghted Majorty (WM) votng varants whch compete aganst the best of the component experts. On the other end we have more complex but more powerful algorthms such as SVM or AdaBoost, whch compete aganst combnatons of classfers. Ths added complexty often comes at the cost of ncreasng the amount of tranng examples requred to learn ths complex soluton. We propose an n-between soluton. Specalzed Weghted Majorty s a method that leverages Weghted Majorty-lke algorthms by modelng the performance of experts across the dmensons/features of the problem. It s known that when Weghted Majorty 1 combnes specalsts (experts that are aware of ther capabltes and can decde whether to vote or not), t competes aganst the best set of specalsts rather that the best expert. In ths paper we propose turnng experts nto specalsts by drectly learnng ther ndvdual areas of expertse. Intutvely, the exstence of these areas s a reasonable assumpton n many cases, for example, when combnng human experts. Our goal s to retan some of the advantages of smple learnng algorthms such as Weghted Majorty, but at the same tme beng capable of fndng more complex solutons. By drectly learnng the specaltes, we hope to develop a method that trades off between both extremes of the learnng spectrum. 2. Prevous Work Prevous work on both types of learnng methods (smple vs complex) s extensve. Smple methods have been shown to have good behavor aganst rrelevant features or nose and to be able to adapt n a tmely manner to dynamcal problems. Weghted Majorty case. 1 The Wnnow weght-update rule s actually used n ths

2 Votng (Lttlestone & Warmuth, 1994) and Wnnow (Lttlestone, 1988; Lttlestone, 1991) are the most representatve algorthms n that category. Both are onlne multplcatve weght updatng methods that try to mnmze the total number of mstakes wthn the learnng process. The weght adjustment process n both methods s desgned so that they compete n performance wth the best expert. On the other sde of the spectrum, we have ensemble methods such as AdaBoost (Freund, 1997; Schapre et al., 1998) and general purpose learners such as SVM (Burges, 1998). In a more offlne fashon, both methods combne experts by optmzng the weghts of each expert rather than n an ncremental way wth every new sample, n a more optmal and global sense wth respect to all prevously seen samples. By dong so, they are able to compete aganst lnear combnatons of experts. In ths study, we am to do better than any of the experts. At the same tme, we do not want to renounce to the good behavor and smplcty of Weghted Majorty. Wth ths n mnd, we have devsed a method that uses the concept of specalsts to separate the learnng process nto two stages. In the frst stage, a specalst s constructed from an expert by addng a classfer that predcts when the expert should vote or not. The second and ensemblng stage uses a small modfcaton to Weghted Majorty (the Wnnow weght-update rule) to combne specalsts. Ths rule has been shown to compete aganst the best set of specalsts (Blum, 1995; Blum, 1996), choosng the best expert for each area of the nput space. 3. Creatng specalsts from experts The problem settng that concerns us s that of emttng a predcton ŷ t (from the set of classes C) for each sample x t R D (D features assocated to each sample). The global predcton s based on the predctons made by our N experts 2, e (x t ) C. Our ground truth conssts of correctly labeled pars (x t, y t ), and we want to combne the predctons of the experts so as to mnmze the number of mstakes. In standard Weghted Majorty votng, ŷ t s assgned such that: ŷ t = arg max c C e (x t)=c w t (1) The weghts w t are updated multplcatvely accordng to the Wnnow algorthm (Lttlestone, 1988; Lttle- 2 We can assume that the experts have much more nformaton at ther dsposal than we can encode n our features x t. Makng e a functon of x t s for notatonal convenence only. stone, 1991): f an expert makes a mstake on a sample then ts weght s multpled by a factor β (0..1), and f (and only f) the global algorthm makes a mstake,.e., ŷ y, the weghts of those experts that voted correctly are dvded by β. The same weght-update algorthm can be appled to specalsts nstead of experts. In the followng sectons we dscuss two methods to create specalsts from experts. Secton 3.1 descrbes a nave approach, whle Sec. 3.2 abstracts ths concept to learn a more general category of areas of expertse Nave approach: Feature Specalsts (FS) Let us assume that the features x t descrbng each sample are bnary. Suppose that t s also reasonable to assume that for each state of a gven feature, one of the experts wll perform better than all others. In ths scenaro, one way to construct specalsts s as the Cartesan product of features and experts: for each expert and each feature, two new specalsts are created; one casts a vote (the same vote as the orgnal expert) only when the feature s actve, the other only when the feature s nactve. The set of N experts s transformed then nto 2DN specalsts. Ths can be thought of as the nave approach, attemptng to fnd the best expert for each feature ndependently : Explct dscovery of specaltes (SWM) In the prevous method, the search for the best expert for each feature can be seen as an attempt to fnd under what condtons each one of the orgnal experts has a good performance. Ths abstracton allows for the separaton of the learnng problem nto two smaller sub-problems. In, we frst learn under what condtons the expert performs well. In a posteror stage we combne the predctons of only those experts whose learned areas of expertse nclude the specfc sample under consderaton. The prevous decson functon s modfed as follows: ŷ t = arg max c C e (x t)=c f w t (2) Here, the flter f {0, 1} selects those experts that should vote, and wll be a functon of the features x t. In our approach, ths functon wll be a classfer traned on prevously seen examples, and wll am to predct whether the expert wll be correct or not. In partcular, we choose the flter to be a lnear SVM decson functon. See Algorthm 1 for an outlne of the process.

3 Algorthm 1 1: Intalze w 1 = 1 for = 1... N 2: for t = 1... M do 3: for = 1... N do 4: model = tran(x 1:t 1,e (x 1:t 1 ) == y 1:t 1 ) 5: f = test(x t,model ) 6: end for 7: ŷ t = arg max c C e (x t)=c f w t 8: Penalze classfers wth false postves: for all f = 1 s.t. e (x t ) y t, w t+1 w t β. 9: f msclassfcaton then 10: Reward classfers that predcted correctly: for all f = 1 s.t. e (x t ) = y t, w t+1 w t 1 β. 11: end f 12: end for In the pseudo-code, the functon tran(...) represents tranng a classfer model on prevous data wth the desred output, whle f = test(x t,model ) evaluates ths classfer on the current sample. The nave procedure (Feature Specalsts) assumes that the features cleanly partton the areas of expertse. By drectly learnng the expertse we are removng ths assumpton, and we can hope for a more accurate descrpton of these areas Dmensonalty Analyss In posteror results, as n Fgure 1, we see that Specalzed Weghted Majorty acheves better results than learnng methods on both ends of the learnng spectrum. Although the dmensonalty of smple methods such as Weghted Majorty s small, the performance s constraned by the best expert. On the other end, complex methods such as SVM can potentally obtan better solutons, however they are hndered by the dmensonalty of the problem they try to solve. From a dmensonalty pont of vew, t s far to compare SWM both wth the FS approach and wth SVM, representatve of both categores of learnng methods: Features Specalsts Although the algorthm remans purely as a weght multplcatve update method and the learnng speed should be fast, the dmensonalty of the problem jumps from N to 2DN. Ths ncreases greatly the amount of data requred to fnd a satsfyng soluton. SVM Suppose we try to fnd an optmal combnaton of the experts by nputtng to a lnear SVM both the experts and the orgnal features as features. The dmenson of the problem s then N + D. SWM Assumng the local classfers for each one of the experts s mplemented by a lnear SVM, the number of parameters to learn s (D + 1)N. However, the separaton of the problem nto two stages reduces the complexty of the learnng problem by decomposng t nto N smaller sub-problems of dmenson D. The decomposton of the problem succeeds n creatng smaller, easer sub-problems of lower dmensonalty that can hopefully be solved n a qucker fashon. 4. Expermental Results 4.1. Synthetc Data In order to evaluate and compare several methods n an deal scenaro, we have created synthetc data where the areas of expertse of each one of the experts can be specfed Data Generaton The dataset has the followng parameters: N - The number of experts K - The number of output classes to predct M - The number of trals/samples. D - The number of features assocated wth each sample (dmenson of the nput space). For ths data, we have chosen bnary features. The value of the features and the output classes are generated n a purely random fashon, enforcng the absence of correlaton between features x t and output y t. The predctons of the experts for each tral are randomly generated, allowng for a certan probablty of beng correct for each expert. Ths probablty depends on two parameters: ɛ g (capturng the general knowledge of the expert) and ɛ s (capturng the specfc expertse, dependng on the area of the nput space x t ). For any gven expert, the value of ts general knowledge ɛ g s drawn unformly from the nterval [0, ˆɛ g ]. Its specfc knowledge along each one of the features of the nput space s drawn unformly form the nterval [0, ˆɛ s ]. Then, for any gven sample (x, y), the probablty of gvng a correct predcton condtoned on feature s defned as: P g {}}{ P [correct x 1 = 1] = K + ɛ g +ɛ s P [correct x = 0] = P g ɛ s

4 We further use condtonal ndependence between features n order to calculate the probablty of beng correct on each sample as: P [x corr]p [corr] P [correct x] = P [x] = P [corr] D P [x corr] P [x] 1 = P [x]p [corr] D 1 P [x corr]p [corr] 1 = P [x]p [corr] D 1 P [corr x ]P [x ] 1 P [correct x] = 2 D P [x]p [corr] D 1 P [corr x ] Wth a smlar dervaton, the overall probablty of beng correct (not condtoned) can be shown to be ndependent of ɛ s and equal to: Results P [correct] = P g = 1 K + ɛ g In ths secton we compare performance wth four methods: Best Expert, Weghted Majorty, Feature Specalsts and SVMs. For the SVM classfer, we use the LIBSVM (Chang & Ln, 01) mplementaton wth an RBF kernel, and all of the hyper-parameters (e. kernel wdth, C) were tuned usng cross-valdaton. In ths case, both sample features x 1:t 1 and expert predctons e 1:N (x 1:t 1 ) are used as tranng data. Fgure 1 shows the performance measured as percentage of accumulated mstakes for a synthetc dataset wth the followng parameters: N = (experts), M = 0 (samples), D = 10 (features), and K = 2 (classes). The y-axs vares the global knowledge of each expert (ɛ g ), whle the x-axs shows varatons n the amount of specalzaton of each expert ɛ s. As expected, we can see that Weghted Majorty s unable to learn expert specaltes and ts amount of mstakes s roughly algned wth the x-axs. More complcated methods (second row) are capable of takng advantage of the specaltes, as can be seen n the decreasng number of mstakes across the x-axs. The complexty of SVMs requres larger values of ɛ s to perform as well as the proposed methods (or alternatvely, a larger number of tranng samples). Lkewse, the Feature Specalsts are not well suted for ths data due to the large number of actve features n each sample. As hoped, our method s able to capture expert specaltes and outperform the best expert Applcaton: Sports Bettng An example problem on whch to apply the proposed method s that of sports bettng. The result to be predcted s the wnner of a match, and the experts wll be a group of sports fans offerng ther predcton. The man assumpton s that certan people mght be better at predctng the outcome of certan games, by vrtue of trackng the performance of the partcular teams more closely. Addtonally, fans of specfc teams mght have unrealstc expectatons or bases when ther preferred team s playng. The parameters of ths dataset are: N = (sport fans), M = 273 (matches), and D = (teams), and K = 3 (team A wns, team B wns, or te). Each feature wll be x t {0, 1}, where 1 ndcates the team s playng and 0 otherwse. Therefore, any gven sample contans 2 features set to one and 18 features set to zero. Experments on ths dataset have shown that all algorthms perform smlarly to the best expert after the 273 matches (142 ± 3 mstakes). Although, far from the worst expert (170 mstakes), the dfferences between algorthms are not sgnfcant enough to draw strong conclusons. In order to understand these results, we produced a smlar synthetc dataset wth the same sparsty pattern (see results n Fgure 2). There s a consderable decrease n the performance of SWM and SVMs, whle there s an ncrease n the learnng of FS. Ths can be nterpreted as SVMs havng more dffculty n learnng from hghly sparse data (due to there beng less nformaton n each sample). Ths drectly affects the performance of our method snce SVMs are used to represent the specaltes. On the other hand, the smplcty of the Feature Specalsts (combned wth the fact that very few features are actve on each sample) allows better learnng. Because the performance on the real data does not reflect the dfferences observed on sparse synthetc data, we beleve ths dataset presents a lack of clear specaltes (.e. users predct at random more often than not). 5. Conclusons We have shown how turnng a set of experts nto specalsts can lead to a sgnfcant gan. Ths was accomplshed by augmentng experts wth knowledge about ther performance n dfferent areas of the feature space. Compared to smpler combnaton methods such as Weghted Majorty, our method offers the

5 P(correct) = 1/K + epslon g epslon s P(correct) = 1/K + epslon g epslon s P(correct) = 1/K + epslon g epslon s (a) Best Expert (b) Average Expert (c) Weghted Majorty P(correct) = 1/K + epslon g Feature Specalsts epslon s P(correct) = 1/K + epslon g epslon s P(correct) = 1/K + epslon g epslon s (d) Feature Specalsts (e) (f) Support Vector Machnes Fgure 1. The fgures show the percentage of accumulated mstakes. advantage of beng able to express a larger set of target concepts. Compared to more general learnng methods commonly used n stackng or cascadng, our method smplcty and use of doman knowledge results n faster learnng (n the sense of requrng less tranng examples). These conclusons have been extracted from plausbly generated (albet stll contrved) artfcal data. Emprcal support for our conclusons was not found n the real-data applcaton on whch we tested. Due to the small amount of avalable data and the smlar performance of all tested methods on ths set (and ndeed, the smlar performance of all avalable experts), we cannot draw strong conclusons from ths experment. It mght be the case that our ntal hypothess for ths data was wrong that there are no areas of expertse n sports bettng users, but t s just as plausble that not enough tranng data was avalable to learn these dfferences for any of the consdered methods. There are two lnes of future work: Explorng technques to mnmze the total number of mstakes n a more global fashon. Although ths would ncrease the dmensonalty of the problem, methodologes for combnng global results wth local ones have proven useful n the past. Our formulaton shows a herarchy that would easly allow such combnaton. Testng the proposed method n dfferent applcatons, wth more complete and sutable datasets. We have consdered for other applcatons are weather and stock market predcton. References Blum, A. (1995). Emprcal Support for Wnnow and Weghted-Majorty Algorthms: Results on a Calendar Schedulng Doman. Machne Learnng (pp ). Morgan Kaufmann. Blum, A. (1996). On-lne algorthms n machne learnng. In Proceedngs of the Workshop on On-Lne Algorthms, Dagstuhl (pp. 6 3). Sprnger. Burges, C. J. (1998). A tutoral on support vector machnes for pattern recognton. Data Mnng and Knowledge Dscovery, 2, Chang, C.-C., & Ln, C.-J. (01). LIBSVM: a lbrary for support vector machnes. Software avalable at

6 P(correct) = 1/K + epslon g epslon s P(correct) = 1/K + epslon g epslon s P(correct) = 1/K + epslon g epslon s (a) Feature Specalsts (b) (c) Support Vector Machnes Fgure 2. The above fgures compare performance of the algorthms on sparse synthetc feature data, where only two features are set to 1 on any gven sample. Freund, Y. (1997). A Decson-Theoretc Generalzaton of On-Lne Learnng and an Applcaton to Boostng,. Journal of Computer and System Scences, 55, Gangardwala, A., & Polkar, R. (05). Dynamcally weghted majorty votng for ncremental learnng and comparson of three boostng based approaches. Proceedngs. 05 IEEE Internatonal Jont Conference on Neural Networks, 05., Lttlestone, N. (1988). Learnng quckly when rrelevant attrbutes abound: A new lnear-threshold algorthm. Machne Learnng (pp ). Lttlestone, N. (1991). Redundant nosy attrbutes, attrbute errors, and lnear-threshold learnng usng wnnow. Annual Workshop on Computatonal Learnng Theory. Lttlestone, N., & Warmuth, M. (1994). The weghted majorty algorthm. th Annual Symposum on Foundatons of Computer Scence, 108, Schapre, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boostng the margn: a new explanaton for the effectveness of votng methods. The Annals of Statstcs, 26,