Coevolutionary Computation for Synthesis of Recognition Systems

Transcription

1 Coevolutonary Computaton for Synthess of Recognton Systems Krzysztof Krawec 1 and Br Bhanu Center for Research n Intellgent Systems Unversty of Calforna, Rversde, CA , USA {kkrawec,bhanu}@crs.ucr.edu Abstract Ths paper ntroduces a novel vsual learnng method that nvolves cooperatve coevoluton and lnear genetc programmng. Gven exclusvely tranng mages, the evolutonary learnng algorthm nduces a set of sophstcated feature extracton agents represented n a procedural way. The proposed method ncorporates only general vson-related background knowledge and does not requre any task-specfc nformaton. The paper descrbes the learnng algorthm, provdes a frm ratonale for ts desgn, and proves ts compettveness wth the human-desgned recognton systems n an extensve expermental evaluaton, on the demandng real-world task of object recognton n synthetc aperture radar (SAR) magery. 1. Introducton Currently, most real-world recognton systems are handcrafted and make ntense use of task-specfc doman knowledge to attan a compettve performance level. Ths makes ther desgn resource-demandng and prohbts the transferablty of knowledge between dfferent applcatons. In ths paper, we propose a general-purpose vsual learnng method that requres only general background knowledge related to Computer Vson (CV) and Pattern Recognton (PR). The method automates the desgn process and does not make assumptons about the tranng data. To cope wth the complexty of synthess task, we break t down nto components and employ cooperatve coevoluton [15] to perform teratve co-desgn, as t does not requre the explct specfcaton of objectves for each component. 2. Motvatons 2.1. Ratonale for learnng n CV Despte the enormous body of lterature produced n the Computer Vson feld durng the past two decades, the progress n overall understandng of desgn prncples, reducton of desgn costs, and ncrease of performance for real-world CV systems for 3D object recognton has been lmted. The proposed methods are often over-specalzed, make demandng assumptons, and are non-transferable between dfferent applcaton areas. The tedous, costly, and hghly subjectve tral-and-error approach, s n fact stll the domnant method for desgnng real-world object recognton systems. We attrbute ths state of affars to the fact that not enough attenton has been pad to hgh-level learnng methods for synthess and desgn support of object recognton systems. Few nterestng contrbutons (see Secton 3) that pursued ths drecton, are rather exceptons. Ths negatve trend may be overturned only by ncorporatng adaptve learnng methodologes, whch meet the demands of real world tasks, do not assume much about the nput data, and focus on optmalty of the entre recognton system. Ths becomes a necessty as the complexty of real-world tasks ncreases and/or the tasks themselves change wth tme Ratonale for evolutonary learnng n CV/PR The paradgm of Evolutonary Computaton (EC) has found a sgnfcant number of applcatons n mage processng and analyss. It has been found effectve for ts ablty to perform global parallel search n hghdmensonal spaces and to resst the local optma problem. However, n most approaches the adaptaton s lmted to parameter optmzaton; here, we take a further step and synthesze entre feature extracton procedures. The proposed approach follows the paradgm of feature-based recognton, so the evolutonary search proceeds n the space of mage features. In partcular, f we defne the feature extracton module as a graph of nterconnected buldng blocks, whch encapsulate basc operatons, then the task of synthess (learnng) of recognton system conssts n manpulatng such graphs. Gven no other extra source of knowledge, the structure of ths exponental-szed search space s unknown a pror 1 On a temporary leave from Insttute of Computng Scence, Poznań Unversty of Technology, Poznań, Poland. 1

2 and t may be effectvely searched only by means of metaheurstcs. Evolutonary computaton s nowadays the most ntensvely nvestgated and state-of-art metaheurstcs, wth a long record of success stores n optmzaton and learnng, ncludng attanng humancompettve performance and outperformng patented solutons [9] Ratonale for coevolutonary learnng n CV/PR It s wdely recognzed that the scalablty of basc learnng methods wth respect to the complexty of the task s usually lmted; e.g., one can successfully tran a neural network to recognze characters, but ths result cannot be extrapolated to, for nstance, nterpretaton of outdoor scenes. Ths observaton motvated many recent research trends n related dscplnes, e.g., mult-agent systems n Artfcal Intellgence, ensembles of classfers n Machne Learnng, multple classfers n Pattern Recognton, etc. Results obtaned there clearly ndcate that further progress cannot be made wthout resortng to some hgher-level learnng methodologes that nherently nvolve modularty (see, e.g., [4]). To provde modularty, we use Cooperatve Coevoluton (CC) [15], whch, besdes beng appealng from the theoretcal vewpont, has been reported to yeld nterestng results n some experments [19]. As opposed to standard EC whch mantans a sngle populaton of ndvduals, each beng a complete soluton to the problem, n CC n>1 populatons are co-evolved, wth each ndvdual encodng only a part of the soluton. To undergo evaluaton, ndvduals have to be (temporarly) combned wth ndvduals from the remanng populatons to form a soluton. Ths jont evaluaton scheme forces the populatons to cooperate. In [2], we provded an expermental evdence for the superorty of CC-based feature constructon over EC approach n the standard machne learnng settng; here, we extend ths dea to vsual learnng. Accordng to Wolpert s No Free Lunch theorem [21], the choce of coevoluton as a partcular search method s rrelevant, as the average performance of any metaheurstc search over a set of all possble ftness functons s the same. In real world, however, not all ftness functons are equally probable. Most real-world problems are characterzed by some features that make them specfc. The practcal utlty of a search/learnng algorthm depends, therefore, on ts ablty to detect and beneft from those features. The decomposable nature of vsual learnng task s such a feature, and CC seems to ft t well, as t provdes an elegant and general framework for teratve co-desgn of components. It allows for breakng up a complex problem nto components wthout specfyng explctly the objectves for them. The manner n whch the ndvduals from populatons cooperate emerges as the evoluton proceeds. Ths makes CC, n our opnon, a perfect match for CV/PR, where t s very often the case that defnng an objectve (functon) for partcular ntermedate stage of processng s dffcult. 3. Related work and contrbutons The area of vsual learnng, especally hgh-level learnng amed at synthess of recognton systems, receved only scant attenton n manstream research of CV/PR. Nevertheless, some past contrbutons ponted out nterestng research drectons, nvolvng blackboard archtecture [5], case-based reasonng, renforcement learnng [13][14], and evolutonary computaton [7][16] [18], to menton the most predomnant. In partcular, [16] descrbes an approach that s comparable to that presented n ths paper (however, wthout engagng coevoluton) and tests t on a smlar data. Ths paper reports the human-compettve performance of a syntheszed recognton system on a non-trval recognton task. The major contrbuton s a general method that, gven exclusvely a set of tranng mages, performs vsual learnng and yelds a complete featurebased recognton system. Its novelty conssts mostly n () procedural representaton of features for recognton, () utlzaton of coevolutonary computaton for nducton of mage representaton, and () learnng process that syntheszes features pror to classfer nducton. 4. Techncal approach 4.1. Coevolutonary synthess of features Followng the feature-based recognton paradgm, we splt the object recognton process nto two modules: feature extracton and decson makng. The algorthm learns from a fnte set of tranng examples (mages) D n a supervsed manner,.e. requres D to be parttoned nto mutually dsjont decson classes D. The vsual learnng conssts here n a CC-based search performed n the space of feature extracton procedures. The cooperaton may be characterzed as takng place at feature level, as t ams at buldng the complete mage representaton (feature vector Y), wth each populaton responsble for evolvng some of those features. In partcular, each ndvdual I from gven populaton encodes a sngle feature extracton procedure that, gven an nput mage X, produces vector of features y(i,x). For clarty, detals of encodng of a sngle feature extracton procedures are provded n Secton

3 The coevolutonary search proceeds n all populatons ndependently, except for the evaluaton phase, shown n Fg. 1. To evaluate an ndvdual I j from populaton #j, we frst provde for the remanng part of the representaton. For ths purpose, representatves I are selected from all the remanng populatons j. A representatve I of th populaton s defned here n a way that has been reported to work best [19]: t s the best ndvdual w.r.t. the prevous evaluaton. In the frst generaton of evolutonary run, snce no pror evaluaton data s gven, t s a randomly chosen ndvdual. Populaton #1 Representatve I 1 Populaton # Indvdual I Representatve I 1 Populaton #n Representatve I n S f(s,d) Soluton Evaluaton LGP program nterpreter Cross-valdaton experment Tranng mages D Basc mage processng operatons Feature vectors Y(X) for all tranng mages X D Fast classfer C ft Fg. 1. The evaluaton of an ndvdual I from th populaton. Subsequently, I j s temporarly combned wth representatves of all the remanng populatons to form the soluton S = I1, K, I j 1, I j, I j+ 1, K, I. n Then, the feature extracton procedures encoded by ndvduals from S are run (see Secton 4.2) for all mages X from the tranng set D. The feature values y computed by them are concatenated, buldng the compound feature vector Y: Y( X ) = y( I1, X ), K, y( I j, X ), K, y( I n, X ). Feature vectors Y(X), computed for all tranng mages X D, together wth the mages decson class labels consttute the dataset: Y ( X ), : X D, D. Fnally, cross-valdaton,.e. multple tran-and-test procedure s carred out on these data. For the sake of speed, we use here a fast classfer C ft that s usually much smpler than the classfer used n the fnal recognton system. The resultng predctve recognton rato becomes the evaluaton of the soluton S, whch s subsequently assgned as the ftness value f ( ) to the ndvdual I j, concludng ts evaluaton process: f ( I j, D) = f ( S, D) =. Y( X ),, D, X D : C ( Y( X )) = { }/ D ft Usng ths evaluaton procedure, the coevolutonary search proceeds untl some stoppng crteron (usually consderng computaton tme) s met. The fnal outcome of the coevolutonary run s the best found soluton/representaton S Representaton of feature extracton procedures In the proposed approach, each ndvdual encodes a feature extracton procedure that s a sequence of predefned basc operatons. For ths encodng, we adopt a varety of Lnear Genetc Programmng (LGP) [1], a hybrd of genetc algorthms (GA) and genetc programmng (GP) [7]. The ndvdual s genome s a fxed-length strng of bytes, representng a sequental program composed of (possbly parameterzed) basc operatons that work on mages and scalar data. Ths representaton combnes advantages of both GP and GA, beng both procedural and more resstant to the destructve effect of crossover that occurs n regular GP. A feature extracton procedure accepts an mage X as nput and yelds a vector y of scalar values as the result. Its operatons are effectvely calls to mage processng and feature extracton functons, whch work on regsters, and may use them for both nput as well as output arguments. Image regsters store processed mages, whereas realnumber regsters keep ntermedate scalar results and features. Each mage regster has sngle channel (grayscale), the same dmensons as the nput mage X, and mantans a rectangular mask that, when actvated by an operaton, lmts the processng to ts area. For smplcty, the numbers of both types of regsters are controlled by the same parameter m. Each chunk of 4 consecutve bytes n genome encodes a sngle operaton wth the followng components: (a) operaton code, (b) mask flag decdes whether the operaton should be global (work on the entre mage) or local (lmted to the mask), (c) mask dmensons (gnored f the mask flag s off ), and (d) arguments: references to regsters to fetch nput data and store the result. For parametrc operatons, selected arguments stored n scalar regsters are nterpreted as parameters. An exemplary encoded operaton could be: morphologcal openng, appled locally to the mask of sze to the mage fetched from mage regster ponted by argument #1, and storng the result n mage regster ponted by argument #2. Ths way of encodng gves the learnng process the possblty of manpulatng the procedure at three levels of abstracton: () choce of operatons, () parameter settng, and () data flow. There are currently 70 operatons mplemented n the system. They mostly consst of calls to functons from Intel Image Processng [6] and OpenCV 3

4 [12] lbrares, and encompass general-purpose mage operatons (processng, basc features, geometrcal moments, mage statstcs), mask-related operatons, and elementary arthmetc and logc. The processor-optmzed mplementaton of lbrares provdes short processng tmes of operatons and good scalablty w.r.t. mage sze. Operaton #1 Intal contents: copes of the nput mage X wth masks set to dstnctve features Genome of ndvdual I (LGP program) op code R 1 R 2 R m Image regsters Operaton #2 arguments Lbrary of basc mage processng procedures Workng memory Operaton #3 r 1 r 2 r m Real-number regsters Feature values y (X), =1,,m fetched from here after executon of entre LGP program Fg. 2. Decodng and executon of LGP code contaned n ndvdual s I genome (for a sngle mage X). The processng of a sngle nput mage X D by the LGP procedure encoded n an ndvdual I starts wth ntalzaton of each of the m mage regsters wth X (Fg. 2). The masks of mages are set to m most dstnctve local features (here: brght blobs ) found n the mage. Real-number regsters are set to the center coordnates of correspondng masks. Then, the operatons encoded by I are carred out one by one, and ntermedate results are stored n regsters. At the end of executon, the scalar values y j (I,X), j=1,,m, contaned n the m real-value regsters are nterpreted as the output yelded by I for mage X. The values are gathered to form an ndvdual s output vector y ( I, X ) = y1( I, X ), K, y ( I, X ), that s subject to further processng (see Secton 4.1) Archtecture of the recognton system The overall recognton system conssts of () best feature extracton procedures S constructed usng the approach descrbed n Sectons 4.1 and 4.2, and () classfers traned usng those features. Facng the suboptmal character of representatons elaborated by the evolutonary process, we ncorporate mult-agent methodology that ams to compensate for that mperfectness and allows us to boost the overall performance. The basc prerequste for the agents fuson m to become benefcal s ther dversfcaton. Ths may be ensured by usng homogenous agents wth dfferent parameter settng, homogenous agents wth dfferent tranng data, heterogeneous agents, etc. Here, the dversfcaton s naturally provded by the random nature of the genetc search. In partcular, we run many genetc searches that start from dfferent ntal states (ntal populatons). The best representaton S evolved n each run becomes a part of a sngle subsystem n the recognton system s archtecture. Each subsystem has two major components: () a representaton S, and () a classfer C traned usng that representaton. As ths classfer tranng s done once per subsystem, a more sophstcated classfer C may be used here (as compared to the classfer C ft used n the evaluaton functon). The subsystems process the nput mage X ndependently and output recognton decsons that are further aggregated by a smple majorty votng procedure nto the fnal decson. The subsystems are therefore homogenous as far as the structure s concerned; they only dffer n the features extracted from the nput mage and the decsons made. 5. Expermental results The prmary objectve of the experment was to synthesze, usng the descrbed methodology, an object recognton system for a non-trval CV/PR task. Secondary goals conssted n testng the scalablty of the approach wth respect to the number of decson classes and the number of subsystems n sub used n the votng procedure. As expermental testbed, we chose the demandng task of object recognton n synthetc aperture radar (SAR) mages. The dffcultes that make recognton n ths modalty extremely hard nclude poor vsblty of objects (usually only promnent scatterng centers are vsble), low persstence of features under rotaton (azmuth change), and hgh levels of nose. The source of mages was MSTAR publc database [17] contanng real mages of several objects taken at dfferent poses (azmuths) and at 1-foot spatal resoluton. From the orgnal complex (2- channel) SAR mages, we extracted the magntude component and cropped t to pxels. No other form of preprocessng has been appled. The followng parameter settngs have been used for each coevolutonary run: selecton operator: tournament selecton wth tournament pool sze = 5; mutaton operator: one-pont, probablty 0.1; crossover operator: one-pont, probablty 1.0; number of regsters (mage and numerc) m: 2; number of populatons n: 4; genome length: 40 bytes (10 operatons); sngle populaton sze: 200 ndvduals. The remanng parameters have been set to defaults used n ECJ software package [11]. 4

5 BRDM T72 ZSU T62 ZIL 2S1 BMP2 BTR70 Fg. 3. Selected objects and ther SAR mages used n the learnng experment. To estmate the performance of the learnng process n a lmted perod of tme, each evolutonary run has been lmted to 4000 seconds on PC wth Pentum 1.4 GHz processor. Decson tree nducer C4.5 (as mplemented n WEKA [20]) has been used as classfer C ft for feature set evaluaton durng evolutonary run. Ths choce was motvated by the low computatonal complexty of C4.5 s nducton and queryng. However, the fnal syntheszed recognton system uses support vector machne wth polynomal kernels of degree 3 as classfer C, wth complexty parameter set to 10 [20]. Ths classfer s much more tme-consumng n learnng, but provdes sgnfcantly better predctve accuracy. True postve rate subsystems 5 subsystems 3 subsystems 1 subsystem # of decson classes Fg. 4. Test set recognton rato as a functon of number of decson classes for dfferent number of subsystems nsub. To nvestgate the scalablty of the proposed approach wth respect to the problem sze, we prepared several datasets wth ncreasng number of decson classes, usng 15-deg. depresson (elevaton) angle data. The smallest problem contans l=2 decson classes: BRDM and ZSU. Consecutve tasks were created by addng the decson classes up to l=8 n the followng order: T62, ZIL, T72, 2S1, BMP2, and BTR70 (see Fg. 3). For th decson class, ts representaton D n the tranng data D conssts of 120 mages, sampled unformly wth respect to azmuth from the orgnal MSTAR database. For l-class task, total sze of tranng set s therefore 120l. The correspondng test set T contans all the remanng mages (for gven decson classes and depresson angle) from the orgnal MSTAR collecton. Fgure 4 presents the recognton rato (true postve rate) as a functon of the number of decson classes l represented n the tranng and test data, for dfferent numbers of votng subsystems n sub. It can be observed that the overall recognton performance of the system s relatvely hgh for all the range of problems consdered. When only one syntheszed subsystems s used (n sub =1), the recognton performance decreases sgnfcantly as new decson classes are added to the problem. However, usng more voters rapdly mproves the scalablty, and wth n sub =10 voters, new decson classes do not affect the performance much. Note that n the test, we have not used confusers,.e. test mages from dfferent classes that those present n the tranng set. 6. Conclusons In ths contrbuton, we provded evdence for the possblty of syntheszng, wthout or wth lttle human nterventon, a feature-based recognton system whch recognzes 3D objects n dffcult radar magery at the performance level that s close to handcrafted, modelbased solutons (cf. [3]). Let us also note that ths approach attans compettve results also for recognton of object varants [10], whch we, however, do not report here due to lmted space. These encouragng results have been obtaned wthout resortng to the database of object models, wthout explct estmaton of object pose, wthout any hand-tunng of basc operatons specfcally to ths applcaton, and, n partcular, wthout ntroducng any SAR-specfc concepts or features lke scatterng center. The learnng process requres a mnmum amount of tranng data,.e. mages and ther class labels only, and does not rely on domanspecfc knowledge, usng only general vson-related knowledge encoded n basc operatons. The graph of processng that s carred out by feature extracton procedure emerges from the evolutonary process and may be used as an nterestng source of nformaton. In partcular, the evolved procedures are often novel and sometmes very dfferent from expert s ntuton, as may be seen from example shown n Fgure 5. The objectvty of the learnng process make the results free from subjectve flaws/bases, whch the humandesgned solutons are prone to: n many cases, the assumpton that features desgned by an expert are the optmal ones has not much scentfc ratonale. 5

6 Input mage X Standard devaton Intal regster contents Laplacan 3x3 Prewtt vert. 3x3 Computed feature value y 1 =18.15 Absolute dfference Fg. 5. An example of syntheszed feature extracton procedure for BRDM mage (azmuth 2.3 ) as an nput. As there s no need for, usually tme-consumng, matchng of the recognzed mage wth models from the database, the recognton speed of the syntheszed systems s very hgh. The average tme requred by the entre recognton process for a sngle mage, startng from the raw mage and endng up at the decson, ranged on the average from 2.2 to 20.5 ms for systems usng n sub =1 and n sub =10 subsystems, respectvely. We clam that ths mpressve recognton speed makes our approach sutable for real-tme applcaton. Acknowledgements Ths research was supported by the grant F C The contents of the nformaton do not necessarly reflect the poston or polcy of the U. S. Government. The frst author s supported by the KBN grant no. 8T11F We thank the authors of software packages: ECJ [11] and WEKA [20]. References [1] W. Banzhaf, P. Nordn, R. Keller and F. Francone, Genetc Programmng: An Introducton. On the automatc Evoluton of Computer Programs and ts Applcaton. Morgan Kaufmann, [2] B. Bhanu and K. Krawec, Coevolutonary constructon of features for transformaton of representaton n machne learnng, Proc. Genetc and Evolutonary Computaton Conference, AAAI Press, New York, pp , [3] B. Bhanu and G. Jones, "Recognzng Target Varants and Artculatons n SAR Images," Optcal Engneerng, vol. 39, no. 3, pp , [4] T. Detterch, Machne learnng research: four current drectons, Artfcal Intellgence, vol. 18, no. 4, pp , [5] B. Draper, A. Hanson and E. Rseman. "Knowledge- Drected Vson: Control, Learnng and Integraton," Proceedngs of the IEEE, vol. 11, no. 84, pp , [6] Intel Image Processng Lbrary: Reference Manual. Intel Corporaton, [7] M.P. Johnson, Evolvng vsual routnes. Master s Thess, Massachusetts Insttute of Technology, [8] J.R. Koza, Genetc programmng - 2. MIT Press, Cambrdge, [9] J.R. Koza, Human-compettve Applcatons of Genetc Programmng, Advances n Evolutonary Computng, A. Ghosh and S. Tsutsu, eds., pp , Sprnger, [10] K. Krawec and B. Bhanu, Coevoluton and Lnear Genetc Programmng for Vsual Learnng, Proc. Genetc and Evolutonary Computaton Conference, 2003 (n press). [11] S. Luke, ECJ Evolutonary Computaton System [12] Open Source Computer Vson Lbrary: Reference Manual. Intel Corporaton, [13] J. Peng and B. Bhanu, Closed-Loop Object Recognton Usng Renforcement Learnng, IEEE Trans. on PAMI, vol. 20, no. 2, pp , Feb [14] J. Peng and B. Bhanu, "Delayed Renforcement Learnng for Adaptve Image Segmentaton and Feature Extracton," IEEE Trans. on Systems, Man and Cybernetcs, vol. 28, no. 3, pp , Aug [15] M.A. Potter and K.A. De Jong, Cooperatve Coevoluton: An Archtecture for Evolvng Coadapted Subcomponents, Evolutonary Computaton, no. 8, pp. 1-29, [16] M. Rzk, M. Zmuda, and L. Tamburno, Evolvng pattern recognton systems, IEEE Trans. Evol. Comp., vol. 6, no. 6, pp , Dec [17] T. Ross, S. Worell, V. Velten, J. Mossng, and M. Bryant, Standard SAR ATR Evaluaton Experments usng the MSTAR Publc Release Data Set, SPIE Proceedngs: Algorthms for Synthetc Aperture Radar Imagery V, vol. 3370, pp , Aprl [18] A. Teller and M.M. Veloso, PADO: A new learnng archtecture for object recognton, Symbolc Vsual Learnng, K. Ikeuch and M. Veloso, eds., pp , Oxford Press, 1997 [19] R.P. Wegand, W.C. Lles, and K.A. De Jong. An Emprcal Analyss of Collaboraton Methods n Cooperatve Coevolutonary Algorthms. Proc. Genetc and Evolutonary Computaton Conference, Morgan Kaufmann, pp , [20] I.H. Wtten and E. Frank, Data Mnng: Practcal Machne Learnng Tools and Technques wth Java Implementatons. Morgan Kaufmann, [21] D. Wolpert and W.G. Macready, No free lunch theorems for optmzaton, IEEE Trans. Evol. Comp., vol.1, no. 1, pp , Aprl