GROUPING OBJECTS BASED ON THEIR APPEARANCE

Transcription

1 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 GROUPING OBJECTS BASED ON THEIR APPEARANCE Altamrano Robles Lus Carlos, Zacarías Flores Fernando 2, De León Gómez Oscar Antono 3 and Sánchez López Abraham 4,2,3,4 ComputerScence, AutonomousUnversty of Puebla (BUAP), Puebla, Méxco fzflores@yahoo.com.mx, altamrano@cs.buap.mx, oleongomez@gmal.com asanchez@cs.buap.mx ABSTRACT The use of clusterng algorthms for partton to establsh a herarchcal structure n a lbrary of object models based on appearance s deployed. The man contrbuton corresponds to a novel and ntutve algorthm for clusterng of models based on ther appearance, closer to human behavor. Ths dvdes the complete set nto subclasses. Immedately, dvdes each of these n a number of predefned groups to complete the levels of herarchy that the user wants. Whose man purpose s to obtan a compettve classfcaton compared to what a human would perform. KEYWORDS Vson, Appearance, Models,Clusterng and Herarchy. INTRODUCTION At the begnnng of object recognton through dgtal mages, was used the well-known geometrc paradgm. Ths was manly to the wde avalablty of algorthms for detectng edges and corners. Despte the good results obtaned, s shown that usng only the way to recognze objects has ts lmtatons. For nstance, f we have two equal cubes n shape and sze but one red and the other blue, all systems based on geometrc characterstcswll say that t s the same object because both objects have the same shape. Ths happens because there s a substantal nformaton loss,nherent to the object n queston. Other sde, appearance-based approach has been proposed as an alternatve to geometrcal approaches. However, ths approach also has advantages (ncreases recognton rates, prncpally) and dsadvantages (depends on llumnaton condtons, pose and other parameters, y of course, requres a lot of mages n order to buld models). But, snce appearance models offer a promses alternatve to tradtonal models, they are been studyng wdely[0, 3, 4]. Because t s not easy to establsh a herarchy among models learnt by the machne, most authors prefer to works n dfferent drectons, such that: mprove recognton rate, mprove lost of nformaton, compresson methods (PCA, for example); one excepton are works made by Nelson and others [, 2]. However, they use just contour of objects for establsh any knd of herarchy no clear and depends of prmtves prevously selected. In ths paper, we present a novel and ntutve approach to establsh a herarchy whch does not depend of any prmtve or uses just contour of objects. Instead, we use all vsual nformaton about object appearance: of course contour, but also, color, texture, pose, etc. Our approach s ntutve because s closer to human behavor : our proposal uses a not supervsed approach n DOI : 0.52/jaa

2 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 order to buld several dfferent clusters and each one has an explanaton n terms of human behavor. They establsh a herarchy begnnng wth objects and fnalzng wth, 2, or the number of trees that we gve a pror, lke we want to dvde a group between men and women: we establsh the only parameter for our algorthm equal to 2 and we can obtan ths groupng wth 2 classes, but whch classes? Answer: several dfferent 2 classes, one par for each possble dualty: men/women, black/whte, short/tall, etc., lke human does. So, we developeda more natural object classfer that consders the followng aspects: Approprate ways to dfferentate between objects modeled by sets of dgtal mages (measures of smlarty/dssmlarty). Clusterng algorthms able to put objects models n class, Classfcaton based not only on ponts, as the tradtonal algorthms. Dfferent experments (shown below) shows that our approach exhbts results close to human results, n the sense above explaned. 2. PRELIMINARIES The followng subsectons present basc concepts about processng objects. 2.. Models based on appearance The object appearance s gven by: the combnaton of shape, ts reflectance propertes, and ts poston n front the camera n scene and the llumnaton present n ths [6]. Should be noted that the frst two are nherent to the object, however, n the thrd, the object s poston n front of the sensor and llumnaton may vary from scene to scene[6].lkewse, s clear that ths defnton mples that the appearance of an object at a partcular tme can be derved from a dgtal mage n whch that element s captured n a specfc scene. Smlarly, then may be used a set of mages to generate a model of the object under dfferent crcumstances. In fgure, you can see a graphcal representaton of an object's appearance[6]. Fgure. An object and ts model Usually, appearance-based approach works as follows: are taken a number of mages of the object you want to buld the model,an mage for each of the possble ways n whch the object s ntended that may appear n an mageand combned wth the varous lghtng condtons that mpnge upon t.next, these mages are stored n a format chosen (gf, tff, bmp, etc.),for later use; these mages themselves can form the model of the object or, they can extract some features that you thnk descrbe the object accurately, thus, these consttute the model. Fnally, the recognton process tself,s to compare the model wth a new mage that s lkely to contan the object (of course, ths latter process requres some extra processng on the new mage, such as object segmentaton of nterest n the mage, nterpolatng between mages, etc.) Creatng models usng Hotellng transform Due to the need for a large number of mages to a sngle object model and n practce t s common to have large quanttes of dfferent object, we need to fnd a compressed representaton 78

3 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 of all the mages. Normally, It s possble to use a common technque n pattern recognton named Hotellng transform or commonly known as prncpal component analyss [6]. Fgure. Egen-mage and a reconstructed mage The general procedure s as follows: [6], [4]:. Each mage s stacked by columns 2. Each mage s normalzed by dvdng by ther respectve norm 3. We obtan the average vector of all mages 4. Each mage s subtracted from the average 5. Matrx P s obtaned, where each column s one of the vectors prevously calculated 6. We obtan the covarance matrx mpled. T Q = P * P 7. Are obtaned the egenvalues and egenvectors of whch are obtaned correspondng to the covarance matrx, λ = λ e = λ /2 Pe Next, we obtan the set of egenvectors or egenmages, these vectors and egenvalues form a low- number of dmensonal egenspace. The dmenson of the egenspace correspondng to the prncpal components you want to use. 20 are normally suffcent as maxmum. We consder that ths should nclude all the mages of all objects. Immedately, we solve the followng system of equatons ea = P We obtan the matrx A contanng necessary coeffcents to reconstruct any mage of any pattern.ths compresson process shows nsgnfcant losses of nformaton for problem that concerns us. In fgure 3 the result of the manfolds extracton s shown for two objects and, n fgure 2 some of the egenmages can be observed obtaned by means of ths method Smlarty measures Typcally when you have a set of models and a test object but wth dfferent vews that contan some overlap n these, global data can dffer greatly the strategy s to fnd the common vew and measure the smlarty of those parts of the data. Then, we defne the dstance between manfolds as follows: d( M, M 2) = mn d( C, M 2) () 79

4 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 where = r, s the number of patches that make up the frst manfold. Thus, the dstance between manfolds s calculated as the best patch smlarty adjusted. For ths calculaton t s necessary to obtan other measures presented below Measure dstance between subspaces The measure of dstance between subspaces s denoted by d( S, S2) and currently there s not a unfed defnton. In ths paper we used the concept of prncpal angles for good performance [].Wth these concepts are defned the followng measures dstance. Man dea s that the nonlnear manfolds are composed of lnear subspaces and the dstance of thesee components defnes the smlarty between two manfolds. Formally denoted as: M = C, C2,, Cm where M s a manfold consstng of lnear subspaces. Once parttoned manfolds, t s only necessary to calculate the dstance among subspaces to proceed. A way to make t s to calculate the man angles that reflect the varatons among the subspaces wthout takng the dataa nto account []. Other methods take nto account the smlarty through the averages of each local pattern n a smlarty measure among exemplary. It s easy to understand that to use the coalton of both measures s better for the problem than we want to solve Measure dstance between subspaces The dstance from a subespacee to a manfold s denoted as d( S, M ) and s defned n the followng way d( S, M ) = mn d( S, C ) Ths means to fnd the mnmum dstance between a subspace and the lneal components of manfold Dstance from a manfold to a manfold Fgure 3. Examples calculated C M The dstance of a manfold to a manfold s denoted as d( M, M 2) and s defned n the followng way: d( M, M ) = mn d( C, M ) 2 2 C M = mn mn d( C, C ) C M C j M 2 j Clearly, the smlarty between M and M 2 s calculated as smlarty of local models that better they adapt. In [Wang2008] ths dstance s used n recognton of faces through sets of mages. In ths paper we wll use t to get groupng of models of objects based apparently. 80

5 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July Dstance among manfolds As t was already defned prevously, Matrx A contans manfolds of the objects n the lbrary. k These defne a trajectory parameterzed n R where k s the number of selected man components. Man problem of groupng models of objects based on apparently s that t should be treated wth ths compact representaton. It s essental to fnd a measure or set of measures of smlarty are necessary to make a classfcaton of objects smlar to what a human would. In the lterature t s common to fnd smlarty measures between objects represented as a vector (column orrow) wthd characterstcs, stored n database of N objects (dataset) [2]. The strength of the relatonshp between two ponts of a dataset s ndcated by a smlarty coeffcent, the larger ths coeffcent, sad objects are related more. Next, we defne the dstance between manfolds as a measure of smlarty between models based on appearance Maxmal Lneal Patch After obtanng the Manfolds, as mentoned n prevous secton, s necessary that each manfold s dvded nto a seres of subspaces called maxmal lneal patches.the concept of maxmal lneal patche (MLP) s defned as a set of ponts defnng a maxmal lnear subspace, where the lnear perturbaton of these ponts s gven naturally as the deflecton between the Eucldean dstance and geodesc dstance. Fgure 4. Examples of Manfolds obtaned from COIL-20 The man dea s to splt the data set nto several MLPs such that sets of mages contanng very smlar to each other. These patches are calculated from a random pont of dataset untl the condton of lnearty s lost. Ths procedure guarantees the local lnearty of models and adaptvely controls the number of models calculated. Formally, [7] be X = x,..., xn a data set, n our case are ponts that make up manfolds, where x D R and Ns the number of ponts n X. we dvde X nto a dsjont collecton of MLPs X = N = C, C C = ( j,, j =,2,, m) j m ( ) ( ) ( ) 2 N = C = x, x,, x ( N = N) n where m s the number of patches and N s the number of ponts n the patch. U C such that: 8

6 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 Fgure 5. Parttonng of a model based on appearance For parttonng s necessary to calculate the Eucldean dstance matrx DE and the matrx of geodesc dstances D G. Calculaton of the MLPs s used as an ntermedate step to fnd the dstance between two manfolds. Ths dstance has been used n face recognton through mage sets [7]. As mentoned above, the dea s to splt the set of mages n several patches whch are embedded n a lnear subspace. The followng shows the algorthm developed. Intally =, C =, X T =, X T =, X R = X.. Whle XT 2. Randomly select a pont n X R as seed ( ) ( ) 2.. C = { x }, X = X { x } R R 2.2. For x C m x, Identfy each of k nearest neghbors xc as canddate ( ) ( ) If x c X R and ( DG ( xc, xn ) / DE ( xc, xn )) θ C = CU { xc} X R = X R { xc} EndIf EndFor 3 XT = UCj, XR = X XT; +, C = EndWhle j= ( ) After calculatng a lnear patch, ts average s calculated. Ths s taken as a representatve of set and summarzes nformaton of selected patch that s formed by numercally smlar mages. The mportance of these representatves s reflected n the reducton of the number of mages requred to represent the object appearance and s taken nto account for the calculaton of the smlarty of a complete model wth other Measure dstance between subspaces by varatons For two subspaces C and C j we defne a dstance measure by varatons as ther average canoncal correlatons as follows: d ( C, C ) = r / σ V j k k = r 82

7 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 Where σ σ r are the canoncal correlatons and are obtaned by the method descrbed n [] and r s the mnmum szes of lnear subspaces Measure dstance between copes To complement the above measure s defned as measured between copes of the duplcate correlaton e and e j : (, ) Smlarty between models based on appearance Fnally, the dstance between subspaces s obtaned as the weghted average of the two prevous measures as follows: d( C, C ) = ( α) d ( C, C ) + αd ( C, C ) j E j V j When comparng two sets of mages the frst shows how smlar the appearance of objects, whle the other s so close that common varaton trends of the two sets. Thus, we can easly defne the dstance between manfolds as n equaton () Clusterng algorthms The K-means algorthm s one of the most used due to ts smplcty and speed. The core dea s to group the data repeatedly around the centrods calculated from an ntal allocaton of the data n a fxed number of k groups. The class membershp changes accordng to the evaluaton of an functon error untl ths error does not change sgnfcantly or data belongng ether. The error functon s defned as follows [5]: Let D be a data set wth n nstances, and be C, C2,, Ck k cluster dsjont from D. The error functon s calculated as: k = = x C E d( x, µ ( C )) where µ ( C ) s the centrod of cluster C and d (.,.) s the Eucldean dstance typcally. The Keywords secton begns wth the word, Keywords n 3 pt. Tmes New Roman, bold talcs, Small Caps font wth a 6pt. spacng followng. There may be up to fve keywords (or short phrases) separated by commas and sx spaces, n 0 pt. Tmes New Roman talcs. An 8 pt. lne spacng follows Spectral clusterng Graph theory can be used to partton data. In these technques we use a measure of smlarty between the data to form a weghted adjacency matrx. Intutvely, we want fnd a partton of the graph where the edges of the nodes that are n dfferent groups have low weght and have hgh weghts when they are n the same group [3], [8]. Let G = ( V, E) be anundrected graph wth a set of vertcesv = v,, vn. We assume that each edge has an assocated postve weght. The heavy adjacency matrx W = w,, j =,, n, where f w j = 0, vertces and j are not connected. Snce G s not addressed must be w j = w j. Furthermore, degree of vertex s defned as follow: j 83

8 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 d n = j= The dagonal matrx havng the degrees of the vertces s called degree matrx and s denoted as D. Lkewse defne heavy adjacency matrx and degree matrx. Next, we defne the unnormalzed Laplacan matrx of a graph as: L = D W Ths matrx has several nterestng propertes and. s postve sem-defnte and the number of egenvectors of ths matrx s drectly related to the number of connected components of the graph [3]. Besdes, ths method s closely related to the method kmeans kernelzado and cuts of graphs normalzed [9] Normalzed Laplacan Matrces In the lterature there are two standard Laplacan matrces. Each one s closely related to each other and s defned as: /2 /2 L = D LD L sym rw = D L Matrx L sym s denoted so because t s symmetrcal and matrx L rw s ntmately related to random walks. As n the case of the non-normalzed matrx, the multplcty of the frst Laplacan egenvalue of the matrx s related to the number of connected components of the graph of heavy adjacences [8].In ths study we used the non-standard matrx and matrx L for the calculaton of the groupngs. 3. Our proposal The ntenton of usng dfferent clusterng methods for parttonng s to acheve an ntal dvson of the set of models and then apply the same algorthm agan, dvdng the classes obtaned n the prevous step n another herarchcal level. Besdes usng the smlarty measure manfolds also be transformed so that they can be manpulated as a sngle object and not as a set of ponts. Ths was acheved by calculatng the centrod of each manfold and calculatng the standard devaton.ths defnes a sphere that contans almost all ponts of the manfold. Ths defnes then a sphere that contans almost all ponts of the manfold. Immedately show the method generates a herarchy of a set of object models. The proposed method can be summarzed as follows:. It creates an ntal partton of the orgnal set. 2. Each class obtaned s dvded k nto groups, the parameter k s user defned for each level of the herarchy 3. Results are stored 4. Repeat from step 2 to complete the requred levels Fnally, we obtaned a tree structure that shows how have been dvded every class obtaned,n each of the teratons calculated. The rates of each object dsplayed n the tree nodes. 4. TEST AND RESULTS Tests were conducted on the classc-lookng set of models suppled by Columba Unversty, COIL-20. The results obtaned were compared wth those valdated by a user vsually, ths n w j sym 84

9 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 order to check that the classfcaton obtaned s smlar or not. The results obtaned are presented n a tree structure where nodes tags are nterpreted accordng to the followng scheme: Fgure 6. scheme of object ndex The system was executed several tmes for each of the algorthms and wth dfferent numbers of levels and Parttons by level, found emprcally that n the partcular case of the lbrary used, three levels and no more than three parttons per group are suffcent to acheve good rankngs. In fgure 7, you can see one of the groups calculated by the algorthm. Note that the classfcaton of objects 4.8 and 6 places n the same class and ths wll not change usng other methods proposed, change only the objects that are grouped nto hgher levels of the herarchy Ths s because the calculaton of the dstance between manfolds gves greater weght to objects appearance so those n whch the domnant gray tone s very smlar are grouped n the same class as evdenced n the case of the wood peces. Ths measure can also reflect some specal characterstcs as how objects n the case of fgure 7 the majorty shares (objects are hgher than wde, almost the same gray and have almost the same knd of symmetry) whch may be an ndcaton that the results can be mproved somewhat by mxng these results to those obtaned usng only borders. Make a change of models representaton calculatng the centrods allows us to use unmodfed exstng algorthms but at the same tme preserves all the nnate drawbacks of these same. To avod the partcular algorthms derved usng k-means, we wll use the spectral clusterng on the centrods. 4.. Spectral clusterng by manfolds centrods Below are shown some of the results obtaned n testng algorthm. For ths partcular test we used three levels and a vector (3,3,2) v =. Fgure 7. One group obtaned wth the proposal. 85

10 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July CONCLUSIONS AND FUTURE WORK Although algorthm s the type of dvde and conquer, ts complexty s polynomal snce the algorthms used (kmeans, spectral) assure us that the groups formed are very smlar to each other and very dfferent from the other groups thereby avodng combnatoral check what s the best partton. However, when an object has been msclassfed, s mpossble to assgn another class and n the best case can be segregated from the group to the next level calculated. On the other hand,usng a measure of smlarty between objects, results were very smlar to those obtaned by other methods snce the model lbrary allows. In addton, t was found that the system s able to match the results obtaned by the user and also s able to detect smlartes not evdent at a glance.in fgure 8 we show some results obtaned wth our proposal. The results have demonstrated the usefulness of smlarty measures alternatve to classc and the data processng as a means to obtan better results. However, much work remans to be done n ths feld n whch ths research ams to poneer.some thngs to ncorporate nto future research are: Fgure 8. An unexpected group?. Use models based on appearance only for objects borders. Wth the am of usng as a complementary method. Select dfferent algorthms to apply to each level. The results may be mproved f dfferent algorthms are used at each level of the herarchy. Testng wth other models lbrares and compare results. It s essental to experment wth dfferent lbrares to delmt the scope of the proposal developed n ths work and and modfy t to make t applcable to more general problems. Automatcally analyze the results. To quanttatvely compare the classfcatons obtaned by the system wth those made by a human. Improvng the representaton of herarchcal structure. Explore the possblty of calculatng the tree dendrogram. Investgate possble applcatons of ths proposal. All these ponts can be taken as a bass for further work n order to fnd new applcatons, detect problems andand generate other proposals that serve to extend knowledge currently hasn the areas of computer vson and clusterng algorthms. ACKNOWLEDGEMENTS The authors would lke to thank to Autonomous Unversty of Puebla for ther fnancal support.óscar De León Gómez would lke to thank to CONACyT by the scholarshp No REFERENCES [] Å.BjörckandG.H.Golub.Numercal methods forcomputng angles between lnear subspaces. Mathematcs of Computaton, 27:579594,

11 Internatonal Journal of Artfcal Intellgence & Applcatons (IJAIA), Vol. 4, No. 4, July 203 [2] Clusterng. IEEE Press, [3] Ulrke Luxburg. A tutoral on spectral clusterng. Statstcs and Computng, 7(4):395 46, December [4] Taylor C.J. Cootes,T.F. Statstcal model of appearance for computer vson. Techncal report, Imagng Scence Bomedcal Engneerng, Unversty of Manchester,March, [5] Janhong Wu Guojun Gan, Chaoqun Ma. Data Clusterng: Theory, Algorthms, and applcatons. ASA-SIAM Seres on Statstcs and Appled Probablty, SIAM, [6] Nene,Nayar, Murase. Parametrc appearance representaton. Nayar, S. K., Poggo, T., (Eds.) Early Vsual Learnng,OxfordUnversty Press. NewYork, :3 60, 996. [7] Xln Chen WenGao Rupng Wang, Shguang Shan. Manfold-manfold dstance wth applcaton to face recognton based on mage set. IEEE,2008. [8] Francs R. Bach y Mchael I. Jordan.Learnng spectral clusterng.techncal Report UCB/CSD , EECS Department, Unverstyof Calforna, Berkeley, Jun [9] Inderjt S. Dhllon. Kernel k-means, spectral clusterng and normalzed cuts. In pp ACM Press, [0] Mundy, J., Lu, A., Pllow, N., et. al., An expermental comparson ofappearance and geometrc model based recognton, Object Representatonn Computer Vson II, n Proceedngs of ECCV96 Internatonal Workshop,Cambrdge U.K., pp Aprl, 996. [] Andrea Selnger and Randal C. Nelson, ``A Perceptual Groupng Herarchy for Appearance-Based 3D Object Recognton'', Computer Vson and Image Understandng, vol. 76, no., October 999, pp [2] Randal C. Nelson and Andrea Selnger ``A Cubst Approach to Object Recognton'', Internatonal Conference on Computer Vson (ICCV98), Bombay, Inda, January 998, [3] Zaman Khan and Adnan Ibraheem. Hand Gesture recognton: A lterature revew. Internatonal journal of artfcal Intellgence & Applcatons, Vol 3, No. 4, July 202. [4] Bm Jan, M.K. Gupta and JyotBhart.Effcent rs recognton algorthm usng method of moment.internatonal journal of artfcal Intellgence & Applcatons, Vol 3, No. 5, September 202. Authors Bography Dr. Fernando Zacaras s full tme professor n the Computer Scence Department at Autonomous Unversty of Puebla. He has drected and partcpated n research projects and development n ths area from 995, wth results that they have been reported n more than 50 publcatons of nternatonal level. And one of the most mportant s Answer set programmng and applcatons. Conacyt project, Reference number: A. He has been a member of several nternatonal commttees such as: The IEEE Latn Amerca Transacton, IAENG Internatonal journal of Computer Scence, Internatonal conference on Advances n Moble Computng and Multmeda, etc. Dr. Lus Carlos Altamrano receved hs Ph.D. degree at Natonal Polytechnc Insttute (Méxco) n Hs nterest areas nclude: Computer Vson, Image Processng, Remote Sensng and Artfcal Intellgence. He has partcpated n several dfferent projects ncludng: Ol projects and Computer Vson Projects. Currently, he s full tme professor n the Computer Scence Department at Autonomous Unversty of Puebla and head of Postgraduate Department at the same Insttuton. Dr. Abraham Sánchez López s full tme professor n the Computer Scence Department at Autonomous Unversty of Puebla. He was member of SNI (Natonal System of Researchers) from 2003 untl 202 (Level ). Hs man areas nclude: Artfcal Intellgence, Robot plannng and smulaton. He receved hs Ph. D. degree n M.C. Óscar Antono De León Gomez receved hs Master s degree at Computer Scence Department atautonomous Unversty of Puebla (202). He has been developng several dfferent projects on Computer Vson at INAOE (Natonal Insttute of Astrophyscs, Optcal and Electroncs), where currently he s workng 87