SEMANTIC REGION LABELLING USING A POINT PATTERN ANALYSIS

Transcription

1 6th European Sgnal Processng Conference (EUSIPCO 008), Lausanne, Swtzerlan, ugust 5-9, 008, copyrght by EURSIP SEMTIC REGIO LBELLIG USIG POIT PTTER LYSIS Sahb Bahroun, Za Belha, ozha Bouemaa École Supéreure es Communcatons e Tuns (SUP COM), Unté e Recherche en Imagere Satelltare et ses pplcatons (URIS), Cté Technologque es Communcatons Rte e Raoue Km 3,5. rana, Tunsa. Phone: + (6) , fax: + (6) , Emal: za.belha@supcom.rnu.tn, nozha.bouemaa@nra.fr BSTRCT Several recent stues an researches focuse on the combnaton of global an fne local regon escrpton usng ponts of nterest. The man beneft of global approach s that homogenous parts of the mage can easly be escrbe by means of global attrbutes whereas small etals are gnore. In the other han, as the regon gets smaller an wth hgh photometrc varablty, ponts of nterest are more approprate to carry local escrpton. In ths paper, we propose a new semantc labellng of regons usng ther nterest pont spatal sperson. We ntrouce a pont-base crteron to label regons nto homogeneous an texture classes. Our pont base crteron s base on a pont pattern analyss stuy an has been valate on a multspectral satellte mage atabase. In our work, we combne the regon an pont escrpton by constructng a escrptor prove by a new semantc labellng of regons for boostng the obect recognton.. ITROUCTIO Global query s consere when retrevng entre mage base on the overall vsual aspect. On the other han, when the user focuses on a small part of the mage or a relevant etal, several local approaches are use []. We can stngush ual schemas: Regon-base an Pont-base escrptons. Regon-base queres ten to focus on vsually homogenous patches an large regons; whle pont-base queres rely on features extracton by means of nterest pont etecton [, 3]. The latter escrpton s carre out when searchng precse obects or escrbng locally varable characterstcs. regon can be efne as a large set of pxels sharng almost smlar vsual attrbutes. In other wors, a regon s assmlate to a large homogenous area n the mage that can be well escrbe by global photometrc features. t the opposte, as the regon gets smaller an wth hgh photometrc varablty, local escrpton s more approprate because t catches small salent areas wth characterstc etals. Unfortunately, a smple vsual escrptor wll classfy regons wthout any stncton between homogenous an texture ones. To o so, local an global escrptons are taken nto account urng the nexng process. Thus our approach tens to a semantc knowlege nformaton on low level features by labellng regons nto homogenous an texture classes. Ths ual regon/ponts escrpton s evaluate on sngle coarse regons usng Harrs colour ponts etector that catches the local photometrc varablty on small stes. The use of ponts of nterest was motvate by ther ablty to fnely escrbe coarsely segmente regon. In ths paper, we use the technque of pont pattern analyss [4, 5]. Ths technque s base on pont of nterest sperson. Ths technque leas to label regons through topologcal an spatal sperson of ponts of nterest. Hstorcally, Pont Pattern nalyss was frst note n the works of botansts an ecologsts n the 930s. However, n the followng years, many fferent fels have also starte to use pont pattern analyss, such as archaeology, epemology, astronomy, an crmnology. In general, Pont Pattern nalyss can be use to escrbe any type of ncent ata. Ths paper s structure as follows. In Secton, we explan the vsual content escrpton. Frst the clusterng process for regon generaton an the Harrs colour ponts extracton metho for regon escrpton. In secton 3, we present the pont pattern analyss technque an the methos that we use n our work. Secton 4 ntrouces our propose technque that labels regons accorng to ther spatal nterest pont topology. Some scusse evaluatons are presente n the expermental setup.. COHERET REGIO ETECTIO FIE ESCRIPTIO The constructon of vsual escrptors assumes that the regons are relevant an also that ther escrpton s confent to ther effectve vsual content. ccorngly, two crucal steps gue escrptors generaton: coherent regons an fne local regon escrpton. The frst step mples that the clusterng algorthm coul relably catch the vsual photometrc content of the regon. In the secon step, the regons have to be fnely escrbe n orer to be well categorze.. Coherent regon etecton The Compettve gglomeraton (C) fuzzy parttonal algorthm [6] has the maor avantage of etermnng the optmal number of fnal classes accorng to a stablty crteron by mnmzng the followng obectve functon: J C C u ( x, β ) α u () Subect to C u,,...,

2 6th European Sgnal Processng Conference (EUSIPCO 008), Lausanne, Swtzerlan, ugust 5-9, 008, copyrght by EURSIP In (), X { x,,..., } mage, (, β ) s a set of ata representng the x represents the stance from feature vector x to the cluster prototype β, membershp of feature pont x n cluster u represents the egree of β an [ u ] s a C* matrx calle constrane fuzzy C-partton matrx. It shoul be note that the number of clusters C n () s ynamcally upate n the C algorthm. The weghtng factor α reflects the compacty of each cluster an balances the relatve nfluence of each term n (). It has a ecreasng exponental an aaptve expresson.. Fne regon escrpton Once the regons are efne by [6], our purpose s to escrbe them as nformatvely as possble n orer to generate a relable vsual escrptor that best summarzes the regon content. Thus, we use the Harrs nvarant ponts of nterest etector [3] to have a pont-base local escrpton of regons because t s more approprate to catch tny etals. Gouet an Bouemaa [] propose a color verson of the Harrs etector [3] for mage escrpton. The ea of the Harrs etector s that nterest ponts are local maxma relate to the secon moment matrx µ ( x,, ) () an the graent strbuton n a local neghbourhoo of a pont x. L ( ) ( ) x x, LxLy x, µ ( x,, ) g( )* () LxLy ( x, ) Ly ( x, ) where s the ntegraton scale, s the ervaton scale, g the Gaussan (3) an L the mage smoothe by a Gaussan (4) for each channel { R, G, B}. T x ( ) x (3) g exp π et L ( x, ) g ( ) I ( x ) (4) * The Harrs color extractor s efne as the postve local extrema of the followng operator: et( µ ( x,, )) α * trace µ ( x,, ) (5) Ths escrpton s more accurate than wth global features (mean color, color strbuton...) snce t catches the local an slght graent varatons aroun a pxel. 3. POIT PTTER LYSIS Pont Pattern nalyss s a class of technques that nvolves the ablty to escrbe patterns of locatons of pont events an test whether there s a sgnfcant occurrence of clusterng of ponts n a partcular area. In general, a spatal ata set m takes the form: X { xk / xk R, m Ν}. Our nterest wll le n quantfyng the sperson of nterest ponts wthn a confne regon. We apply a pont pattern analyss technque to unsupervse label regons nto homogenous or texture classes. In fact, the maor property of ths technque s that ponts contrbute to a global sperson crteron that encapsulates the followng cases: If a regon has few extracte ponts of nterest, or concentrates some ponts on the contours ue to the coarse segmentaton an rough contours, t s more lkely to be labelle as homogenous (Fgure (a)). If a regon s covere wth hgh number of nterest ponts, more or less regularly sperse n the regon, t s more lkely to be labelle as texture. The regon s characterze by an mportant photometrc varablty (Fgure (b)). (a) Fgure - Many ponts are locate on the contours an confne n a small part of the regon n (a). The ponts n (b) are sperse n the regon. The pont pattern analyss s base on comparng the strbuton of nterest ponts etecte n a gven regon wth ranomly strbute ponts n the same regon. The Complete Spatal Ranomness (CSR) moel assumes that ponts are strbute at ranom, whch often splay features that look lke clumpng. In aton, ths moel assumes that the mean ensty of ponts per unt area s known. There are several methos an algorthms to escrbe pattern for a collecton of ponts. The most popular an best establshe mathematcal an statstcal methos use n the lterature are the Quarant Count Metho [7] an the earest eghbour stance [8]. 3. Homogeneous Posson Process The ranom moel that wll serve as our stanar of comparson s the Complete Spatal Ranomness (CSR) moel [5, 9, 0]. The CSR moel has two basc characterstcs: () The number of ponts n any planar regon s wth area follows a Posson strbuton wth mean λ. () Gven there are n ponts n, those ponts are nepenent an form a ranom sample from a unform strbuton on. The constant λ s the mean number of ponts per unt area. lso, by (), the ntensty of ponts oes not vary over the plane. ccorng to (), CSR also mples the events are nepenent of each other an there s no nteracton between them. The mathematcal construct that we wll use to smulate a CSR moel s the homogenous Posson process. Please recall that the Posson strbuton was preferre to the Bnomal an the Geometrc strbutons. Mathematcally, ths s expresse n the fact that for a Posson strbuton, the varance of the sample s equal to ts mean. The geometrc strbuton has a varance to mean rato VMR >, whle the bnomal strbuton has VMR <.0. In probablty theory an statstcs, the varance-to-mean rato (VMR) s a goo measure of the egree of ranomness of a gven phenomenon. Therefore, to assess f a set of ponts are strbute or f there s some clusters n a gven regon we wll compare the VMR of our strbuton of nterst ponts to the VMR of the homogeneous posson strbuton. It s well (b)

3 6th European Sgnal Processng Conference (EUSIPCO 008), Lausanne, Swtzerlan, ugust 5-9, 008, copyrght by EURSIP known that for a Posson strbuton, the varance an the mean are the same. For ths reason, ranom varables wth a varance to mean rato greater then one are calle over sperse an those wth a varance to mean rato less then one are uner sperse [0]. If the set of nterest ponts are ranomly strbute then, the VMR s about Quarant Count Metho The Quarant Count Metho can be escrbe smply as parttonng the ata set nto n equal sze sub regons; we wll call these sub regons quarants. In each quarant we wll be countng the number of nterest ponts that occur an t s the strbuton of quarant counts that wll serve as our ncator of pattern. fter parttonng the ata set nto quarants (see Fgure ), the frequency strbuton of the number of ponts per quarant has been constructe. The Varance (6) an Mean are then compute to calculate the Varance-to-Mean Rato (VMR). The followng s the way we wll nterpret the VMR: If VMR >.0, ths mples that the set of nterest ponts has one or more groups of ponts n clusters an large quarants wthout ponts. Ths mples that the regon can be labelle as homogeneous. If VMR <.0, ths mples that the set of nterest ponts are strbute more or less regularly over the regon. Ths mples that the regon can be labelle as texture. If VMR.0, ths mples that the set of nterest ponts are ranomly sperse. They have no omnant tren towars clusterng or sperson. Fgure Example of applyng the Quarant Count Metho. Bulng of quarants wth the same sze The varance an Mean was compute as follows: Varance n n Q Q (6) Where Q s the number of quarants an n s the number of nterest ponts n quarant. The Mean s the number of nterest ponts n the regon ve by the number of quarants. The VMR compares the number of ponts n each quarant wth the average of the ponts on all the quarants. The man savantage of ths metho s the choce of the quarant sze because t can greatly affect our analyss. We test fferent quarant szes; we notce the smaller number of quarants correspons to larger varance. But as we ve the regon nto smaller quarants, the varance starts ecreasng to one. We choose the followng quarant sze because we want to have a quarant sze proportonal to the regons sze: Sze ( n / rea) *, where n s the number of all nterest ponts etecte n a gven regon. nother savantage of ths metho s where we have VMR close to.0. Then, we can t be confent about the label that we wll attrbute to the regon. Moreover, t has been mentone n [7] that the Quarant Count Metho s not the most accurate metho for entfyng clusterng of ata ponts. The earest eghbor nalyss woul help us to be more confent about the label that we wll attrbute to all regons. 3.3 earest eghbour stance The earest eghbour metho was ntally ntrouce by J. G. Skellam [] where the rato of expecte an observe mean value of the nearest neghbour stances s use to etermne f a ata set s clustere. Further work was one by P. J. Clark an F. C. Evans [8] to ntrouce a statstcal test of sgnfcance of the nearest neghbour statstc n orer to quantfy the eparture of the pattern from ranom. Ths test s of great mportance because even ranomly generate ata coul be labelle as clustere, but the sgnfcance test woul reveal f the evence for ths classfcaton s lackng. ot surprsngly, the nearest neghbour metho s base on comparng the strbuton of the stances that occur from a ata pont to ts nearest neghbour n a gven ata set wth the ranomly strbute ata set. The Complete Spatal Ranomness (CSR) moel assumes that nterest ponts are strbute at ranom, whch often splay features that look lke clumpng. In aton, ths moel assumes that the mean ensty of nterest ponts per unt area s known. fter revewng the ervaton of the nearest neghbour metho we wll calculate the test statstc whch etermnes whether the strbuton s ranomly strbute, unformly strbute or clustere. fter computng the test statstc we woul lke to know whether our result s ue to a samplng error, so we woul lke to see how relable our result s. The basc ea from Skellam [] s to evse a test that woul compare the strbuton of the nearest neghbour stances for a gven ata set wth nearest neghbour stances of a ranomly strbute ata set n a CSR moel. The Posson strbuton wll be use to evelop a CSR moel for a ranomly strbute set of ata ponts. For a set of nterest ponts strbute n a gven regon, we calculate the average stance from each pont to hs nearest neghbour [8]: (7) For a ranom strbuton of ponts generate by the homogeneous process the average stance s equal to []: posson Fnally, we compute the rato R: R posson (9) The rato R s nterprete as follows: R <.0: Exstence of clusters. The ponts are more groupe than the ponts generate by the ranom strbuton. Therefore, the regon can be labelle as homogeneous. R >.0: the ponts are more sperse than the ponts generate by the ranom strbuton. The regon can be labelle as texture. (8)

4 6th European Sgnal Processng Conference (EUSIPCO 008), Lausanne, Swtzerlan, ugust 5-9, 008, copyrght by EURSIP R.0: the ponts have no tren to be clustere or sperse. For values of R very close to.0, t s necessary to o statstc test of sgnfcance to uge the robustness of the result. 3.4 Test of Sgnfcance Clark an Evans [8] propose a test to ncate whether the observe average nearest neghbour stance was sgnfcantly fferent from the mean ranom stance. The test s between the observe nearest neghbour stance an that expecte from a ranom strbuton an s gven by: E T, (0) T s the sgnfcance of eparture from ranom expectaton. s a stanar evaton of expecte mean stance to the nearest neghbour + posson () Where s the number of nterest ponts n the gven regon. There have been other suggeste tests for the nearest neghbour stance as well as correctons for ege effects (for more etals see [0]). However, equatons (0) an () are use most frequently to test the average nearest neghbour stance. fter havng carre out the test of total sgnfcatvty, t s nterestng to carry out the test of partal sgnfcatvty. We have to choose a egree of confence whch correspons to a probablty n the table of Stuent, P0.05 for example. The value T observe s compare wth the value from the table of Stuent. If T observe s greater than the corresponng value n the table of stuent, the fference between the two means s to 95% sgnfcant. The number of egrees of freeom for the strbuton of ponts s OUR PROPOSE PPROCH In ths paragraph, we present the ecson algorthm that we ntrouce to label regons nto homogeneous or texture. We have use both the Quarant Count Metho an the earest eghbour stance. We have also use the sgnfcance test f we obtan fferent labels by these two methos. To label regons nto homogeneous an texture, we have to follow these steps: ) pply the quarant Count Metho an attrbute a label L QCM to the gven regon. ) pply the earest eghbour Metho an attrbute a label L M to the gven regon. 3) If L M L QCM then fnal label L Fnal L M L QCM Else apply the Fsher Stuent s test If T observe > T Stuent then L Fnal L M Else L Fnal L QCM 5. EXPERIMETL SETUP In our evaluaton, we frst segmente a generc atabase contanng about 500 Multspectral satellte mages wth fferent spectral resolutons an spatal resolutons rangng between m an 0m. Ths process proves about 7 regons per mage, an then we apply our crteron on about 0000 regons. Ponts are etecte usng the Colour Harrs etector []; each mage wth sze 5*5 pxels s escrbe by about 000 ponts of nterest. Many types of areas (urban, Mountan, sea, lake ) are present n our atabase (see Fgure 3). ccorngly, two crucal steps gue our metho: coarse an coherent regons; an fne local regon escrpton. The frst step mples that the segmentaton coul be rough regarng contours sharpness but relable enough to catch the vsual photometrc content of the regon. When applyng the Compettve gglomeraton algorthm [6], we tre to elmnate the small regons. In the secon step, the regons etecte have to be fnely escrbe by nterest ponts n orer to be well labelle. We have use Harrs etector [] for regon escrpton: startng from a multspectral mage, we extract ponts of nterest. Gven the segmentaton mask obtane by the C algorthm, for each regon, we extract ponts. We present n Fgure 3 an example of 4 mages where we tre to label respectvely the followng regons: sea n Fgure 3.a (blue colour), mountan area n Fgure.b (re colour), lake n Fgure 3.c (yellow colour) an urban area n Fgure 3. (whte colour). (a) sea (c) lake (b) mountan () urban Fgure 3 Example of fferent regons (sea, mountan, lake, urban) respectvely n fgures (a), (b), (c) an () In Fgure 3, we have Examples of homogenous labelle regons: they encapsulate perceptually homogenous regons as well as those resultng from pont s accumulaton over borers an groupng on small etals. We also have Examples of texture labelle regons: they gather texture patches an fully covere regons enotng of an mportant photometrc varaton. We compute n Table the VMR, R an the label etermne by our propose algorthm for the regons presente n Fgure 3. otce, from Table that for regon an regon 4 whch represent Fgure 3.a an Fgure 3., we have

5 6th European Sgnal Processng Conference (EUSIPCO 008), Lausanne, Swtzerlan, ugust 5-9, 008, copyrght by EURSIP got the same label by the QCM an the M. For all regons n our atabase where we obtane the same label by the two methos we have mae test of sgnfcance to uge the robustness of the pont 3) of our propose approach an t was foun that n 9% of the regons teste we obtane the goo label. umber of VMR R Label nterest ponts Regon homogeneous Regon texture Regon texture Regon , texture Table : VMR, R an label compute for the regons presente n Fgure 3 For regon an regon 3 whch represent Fgure 3.b an Fgure 3.c, we have got fferent labels. The values of VMR an R were close to.0. In orer to see whether ths result s statstcally sgnfcant we conuct a test of sgnfcance For regon an regon 3 we obtane T observe > T Stuent. Then regons were labelle texture. It s the label gven by the earest eghbour Metho. For regon, the sze of the quarant was 6*6 pxels an for regon 3 8*8 pxels. Combnng these two methos allowe us to attrbute the label texture to these two regons wth 95% of confence. For vsual escrptor constructon, we use a colour hstogram for homogeneous labelle regons an correlogram [] for texture labelle regons. [] V. Gouet an. Bouemaa, Obect-base queres usng color ponts of nterest, In Workshop on Content-Base ccess of Image an Veo Lbrares (CVPR), 00. [3] C. Harrs an M. Stephens. combne corner an ege etector. In lvey Vson Conference, pages 47 5,988. [4] ale, Mark R. T., Spatal Pattern nalyss n Plant Ecology, Cambrge Unversty Press, 999. [5] ggle, Peter J. Statstcal nalyss of Spatal Pont Patterns. ew York: Oxfor Unversty Press Inc., 003. [6] Hchem Frgu, ew approaches for robust clusterng an for estmatng the optmal number of clusters, thése e octorat, ecember 997. [7] Cresse, oel., Statstcs for Spatal ata (revse eton), John Wley & Sons, 993. [8] Clark, P. an F. Evans. stance to earest eghbour as a Measure of Spatal Relatonshps n Populatons, Ecology, 35, [9] Mles, R. E. (970). On the homogeneous planar Posson pont process. Mathematcal Boscences, Vol. 6, pp, [0] aley,.j, Vere-Jones,., (988) Introucton to the theory of posson process. Sprnger-Verlag. [] Skellam, J.G. (95). Stues n statstcal ecology. I. Spatal pattern, Bometrca, 39, [] Bahroun S., Belha Z., Bouemaa., Correlogram n multspectral satellte Image Inexng. In: Frst Internatonal Conference of E-mecal Systems. Fez, Morocco, October COCLUSIO In ths paper, we presente a new approach base on pont pattern analyss to label regons nto homogeneous or texture. The Complete Spatal Ranomness (CSR) moel s one way to smulate ata ponts that occur at ranom an the Posson strbuton proves a convenent tool for ths moel. The computaton of a test statstc was use for classfyng the strbuton of nterest ponts as beng ranomly strbute, unformly strbute or clustere. test of sgnfcance was foun that prove a measure of how far remove the observe result s from the ranom moel. We propose a new algorthm to label regons. It was then apple to a set of multspectral satellte mages wth fferent spatal an spectral resolutons. Our new metho encapsulates the coarse escrpton of regons an the fne localzaton of nterest ponts to label regons accorng to ther structure epcte by spatal topology an sperson of nterest ponts. Many rectons can erve from the couple use of ponts/regons escrpton. One possblty s boostng technque for obect recognton. REFERECES [] H. Houssa an. Bouemaa, Regon labellng usng a pont-base coherence crteron IST/SPIE conference on Multmea Content nalyss, Management an Retreval, 006.