Unsupervised Texture Segmentation Using Feature Distributions

Similar documents
FEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

Parallelism for Nested Loops with Non-uniform and Flow Dependences

A PATTERN RECOGNITION APPROACH TO IMAGE SEGMENTATION

A Binarization Algorithm specialized on Document Images and Photos

Comparison Study of Textural Descriptors for Training Neural Network Classifiers

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning

ROTATION-INVARIANT TEXTURE CLASSIFICATION USING FEATURE DISTRIBUTIONS. Abstract

Local Quaternary Patterns and Feature Local Quaternary Patterns

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 15

TN348: Openlab Module - Colocalization

S1 Note. Basis functions.

Edge Detection in Noisy Images Using the Support Vector Machines

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

Unsupervised Learning

A Fast Content-Based Multimedia Retrieval Technique Using Compressed Data

Scale Selective Extended Local Binary Pattern For Texture Classification

CS 534: Computer Vision Model Fitting

Color Image Segmentation Using Multispectral Random Field Texture Model & Color Content Features

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching

Unsupervised Learning and Clustering

6.854 Advanced Algorithms Petar Maymounkov Problem Set 11 (November 23, 2005) With: Benjamin Rossman, Oren Weimann, and Pouya Kheradpour

An Image Fusion Approach Based on Segmentation Region

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

Machine Learning: Algorithms and Applications

Brushlet Features for Texture Image Retrieval

Unsupervised Learning and Clustering

Determining the Optimal Bandwidth Based on Multi-criterion Fusion

Skew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach

A Gradient Difference based Technique for Video Text Detection

Support Vector Machines

Combination of Color and Local Patterns as a Feature Vector for CBIR

A Gradient Difference based Technique for Video Text Detection

A Robust Method for Estimating the Fundamental Matrix

Applying EM Algorithm for Segmentation of Textured Images

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:

Fitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

Fuzzy Filtering Algorithms for Image Processing: Performance Evaluation of Various Approaches

Fuzzy C-Means Initialized by Fixed Threshold Clustering for Improving Image Retrieval

Feature Reduction and Selection

Proper Choice of Data Used for the Estimation of Datum Transformation Parameters

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

Local Tri-directional Weber Rhombus Co-occurrence Pattern: A New Texture Descriptor for Brodatz Texture Image Retrieval

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline

Object-Based Techniques for Image Retrieval

IMAGE FUSION TECHNIQUES

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1

SHAPE RECOGNITION METHOD BASED ON THE k-nearest NEIGHBOR RULE

12. Segmentation. Computer Engineering, i Sejong University. Dongil Han

COMPLEX WAVELET TRANSFORM-BASED COLOR INDEXING FOR CONTENT-BASED IMAGE RETRIEVAL

Hierarchical clustering for gene expression data analysis

Problem Set 3 Solutions

An Entropy-Based Approach to Integrated Information Needs Assessment

EYE CENTER LOCALIZATION ON A FACIAL IMAGE BASED ON MULTI-BLOCK LOCAL BINARY PATTERNS

Lecture 5: Multilayer Perceptrons

Cluster Analysis of Electrical Behavior

Classifying Acoustic Transient Signals Using Artificial Intelligence

Feature Extractions for Iris Recognition

A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception 1

Support Vector Machines

X- Chart Using ANOM Approach

Detection of an Object by using Principal Component Analysis

Histogram of Template for Pedestrian Detection

NAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics

USING GRAPHING SKILLS

A Background Subtraction for a Vision-based User Interface *

Steps for Computing the Dissimilarity, Entropy, Herfindahl-Hirschman and. Accessibility (Gravity with Competition) Indices

Classifier Selection Based on Data Complexity Measures *

Maximum Variance Combined with Adaptive Genetic Algorithm for Infrared Image Segmentation

Fitting: Deformable contours April 26 th, 2018

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

Gender Classification using Interlaced Derivative Patterns

An efficient method to build panoramic image mosaics

Corner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity

Image Alignment CSC 767

Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

MOTION PANORAMA CONSTRUCTION FROM STREAMING VIDEO FOR POWER- CONSTRAINED MOBILE MULTIMEDIA ENVIRONMENTS XUNYU PAN

Problem Definitions and Evaluation Criteria for Computational Expensive Optimization

MOTION BLUR ESTIMATION AT CORNERS

A Modified Median Filter for the Removal of Impulse Noise Based on the Support Vector Machines

Programming in Fortran 90 : 2017/2018

A Novel Adaptive Descriptor Algorithm for Ternary Pattern Textures

Active Contours/Snakes

Colour Image Segmentation using Texems

TECHNIQUE OF FORMATION HOMOGENEOUS SAMPLE SAME OBJECTS. Muradaliyev A.Z.

A MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS

Editorial Manager(tm) for International Journal of Pattern Recognition and

User Authentication Based On Behavioral Mouse Dynamics Biometrics

Wishing you all a Total Quality New Year!

A Statistical Model Selection Strategy Applied to Neural Networks

Pictures at an Exhibition

A B-Snake Model Using Statistical and Geometric Information - Applications to Medical Images

A fast algorithm for color image segmentation

Feature Selection for Target Detection in SAR Images

IMAGE MATCHING WITH SIFT FEATURES A PROBABILISTIC APPROACH

Face Tracking Using Motion-Guided Dynamic Template Matching

Sequential search. Building Java Programs Chapter 13. Sequential search. Sequential search

Transcription:

Unsupervsed Texture Segmentaton Usng Feature Dstrbutons Tmo Ojala and Matt Petkänen Machne Vson and Meda Processng Group, Infotech Oulu Unversty of Oulu, FIN-957 Oulu, Fnland ojala@ee.oulu.f, mkp@ee.oulu.f Abstract Ths paper presents an unsupervsed texture segmentaton method, whch uses dstrbutons of local bnary patterns and pattern contrasts for measurng the smlarty of adjacent mage regons durng the segmentaton process. Nonparametrc log-lkelhood test, the G statstc, s engaged as a pseudo-metrc for comparng feature dstrbutons. A regon-based algorthm s developed for coarse mage segmentaton and a pxelwse classfcaton scheme for mprovng localzaton of regon boundares. The performance of the method s evaluated wth varous types of test mages. Texture segmentaton Feature dstrbuton G statstc Spatal operator Local Bnary Pattern Contrast Introducton Segmentaton of an mage nto dfferently textured regons s a dffcult problem. Usually one does not know a pror what types of textures exst n an mage, how many textures there are, and what regons have whch textures (). In order to dstngush relably between two textures relatvely large samples of them must be examned,.e., relatvely large blocks of the mage. But a large block s unlkely to be entrely contaned n a homogeneously textured regon and t becomes dffcult to correctly determne the boundares between regons. Many dfferent approaches to mage and texture segmentaton have been proposed (2,3,4). Segmentaton methods are usually classfed as regon-based, boundary-based or as a hybrd of the two. The segmentaton can be supervsed or unsupervsed. In unsupervsed segmentaton no a pror nformaton about the textures present n the mage s avalable. Ths makes t s a very challengng research problem n whch only lmted success has been acheved so far. Early methods proposed for unsupervsed regon-based texture segmentaton nclude approaches based on splt-and-merge methods (5), pyramd node lnkng (6), selectve feature smoothng wth clusterng (7), and a quadtree method combnng statstcal and spatal nformaton (8). Examples of more recent approaches are methods based on local lnear transforms and multresoluton feature extracton (9), feature smoothng and probablstc relaxaton (), autoregressve models (,2), Markov random feld models (3,4,5,6), multchannel flterng (7,8,9), neural network based generalzaton of the multchannel approach (2), wavelets (2,22), fractal dmenson (23), and hdden Markov models (24). A method for unsupervsed segmentaton of color textures usng Markov random felds and a splt-and-merge type algorthm was proposed by Panjwan and Healey (25). Some of the exstng methods perform reasonably well for a small set of fne-graned texture mosacs, but they usually need some pror knowledge of the mage contents to acheve satsfactory results, lke the number of textures or regons. The choce of proper parameters for dfferent types of mages may also be dffcult and the methods typcally perform poorly for natural mages contanng nonunform textures. Unsupervsed segmentaton of mages contanng texture prmtves at very dfferent scales may even be unrealstc, because t s hard to dscrmnate small mage regons from large texture prmtves wthout any pror knowledge. The choce of hghly dscrmnatng texture features s the most mportant factor for a success n texture segmentaton, but ths has been neglected n most earler approaches. The features should easly dscrmnate varous types of textures and the wndow sze used for computng textural features should be small enough to be useful for small mage regons and to provde small error rates at regon boundares. Our recent studes show that excellent texture dscrmnaton can be obtaned wth local texture operators and nonparametrc statstcal dscrmnaton of sample and prototype dstrbutons. Texture classfcaton results obtaned by usng dstrbutons of local bnary patterns (LBP) or gray scale dfferences have been better than those obtaned wth the exstng methods (26,27,28,29). Our method can be easly generalzed to utlze multple texture features, multscale nformaton, color features and combnatons of multple features usng the new multchannel approach to texture descrpton (29). Ths paper presents an effcent method for unsupervsed texture segmentaton based on texture descrpton wth feature dstrbutons. A regon-based algorthm s developed for coarse mage segmentaton and a pxelwse classfcaton scheme for mprovng the localzaton of regon boundares.

2 Texture Descrpton The texture contents of an mage regon are characterzed by the jont dstrbuton of Local Bnary Pattern (LBP) and Contrast (C) features(27). The orgnal 3x3 neghborhood (Fg. a) s thresholded by the value of the center pxel. The values of the pxels n the thresholded neghborhood (Fg. b) are multpled by the bnomal weghts gven to the correspondng pxels (Fg. c) and obtaned values (Fg. d) are summed for the LBP number (69) of ths texture unt. By defnton LBP s nvarant to any monotonc gray scale transformaton. LBP descrbes the spatal structure of the local texture, but t does not address the contrast of the texture. For ths purpose we combne LBP wth a smple contrast measure C, whch s the dfference between the average gray level of those pxels whch have value and those whch have value (Fg. b). 6 5 2 7 6 9 3 7 4 6 8 32 64 28 32 8 2 LBP = +8+32+28 = 69 28 C = (6+7+9+7)/4 - (5+2++3)/4 = 4.5 (d) Fg.. Computaton of Local Bnary Pattern (LBP) and contrast measure C. The LBP/C dstrbuton s approxmated by a dscrete two dmensonal hstogram of sze 256xb, where b s the number of bns for C. Choosng b s a trade-off between the dscrmnatve power and the stablty of the texture transform. If b s too small, the hstogram wll lack resoluton and feature C wll add very lttle dscrmnatve nformaton to the process. However, snce the mage regon contans a fnte number of pxels, t does not make sense to go to the other extreme, for then the hstogram becomes sparse and unstable. Based on the results of our past texture classfcaton experments wth the LBP/C transform we elected to use 8 bns, although we expect to acheve comparable results wth 4 or 6 bns as well. See Ojala et al.(27) for a detaled descrpton of the mappng from the contnuous C space to the dscrete bn ndex. A log-lkelhood-rato, the G statstc(3), s used as a pseudo-metrc for comparng LBP/C dstrbutons. The value of the G statstc ndcates the probablty that the two sample dstrbutons come from the same populaton: the hgher the value, the lower the probablty that the two samples are from the same populaton. We measured the smlarty of two hstograms wth a two-way test of nteracton or heterogenety: n n n n n n f f log f log f f + f log G = 2 f log f s, m = s, m s, m = s, m = s, m = = = s, m () where s, m are the two sample hstograms, n s the number of bns and f s the frequency at bn. The more alke the hstograms s and m are, the smaller s the value of G. 3 Segmentaton Algorthm The segmentaton method conssts of three phases: herarchcal splttng, agglomeratve mergng and pxelwse classfcaton. herarchcal splttng agglomeratve mergng (d) pxelwse classfcaton Fg. 2. Texture mosac #; the man sequence of the proposed segmentaton algorthm.

Frst, herarchcal splttng s used to dvde the mage nto regons of roughly unform texture. Then, agglomeratve mergng procedure merges smlar adjacent regons untl a stoppng crteron s met. At ths pont we have obtaned rough estmates of the dfferent textured regons present n the mage and complete the analyss by a pxelwse classfcaton to mprove the localzaton. Fg. 2 llustrates the progress of the segmentaton algorthm on a 52x52 mosac contanng fve dfferent Brodatz (3) textures. 3. Herarchcal Splttng A necessary prerequste for the agglomeratve mergng to be successful s that the ndvdual mage regons are unform n texture. For ths purpose we apply the herarchcal splttng algorthm, whch recursvely splts the orgnal mage nto square blocks of varyng sze. The decson whether a block s splt to four subblocks s based on a unformty test. We measure the sx parwse G dstances between the LBP/C hstograms of the four subblocks. If we denote the largest of the sx G values by G max and the smallest by G mn, the block s found to be nonunform and s thus splt further nto four subblocks, f a measure of relatve dssmlarty wthn regon s greater than a threshold R G max = ------------ > X (2) G mn Regardng the proper choce of X, one should rather choose a too small value for X nstead of a too large one. It s better to splt too much than too lttle, for the followng agglomeratve mergng procedure s able to correct errors, where an unform block of a sngle texture has been needlessly splt. But error recovery s not possble, f segments contanng several textures are assumed to be unform. Threshold X was expermentally set to value.2. We computed G max and G mn (Eq. 2) for numerous blocks wth varyng texture contents, and a 2% dfference generally ndcated a devaton n the local texture. To begn wth, we dvde the mage nto rectangular blocks of sze S max. If we appled the unformty test on arbtrarly large mage segments, we could fal to detect small texture patches and end up treatng regons contanng several textures as unform. The next step s to use the unformty test. If a block does not satsfy the test, t s dvded nto four subblocks. Ths procedure s repeated recursvely on each subblock untl a predetermned mnmum block sze S mn s reached. It s necessary to set a mnmum lmt for the block sze, for the block has to contan a suffcent number of pxels for the LBP/C hstogram to be relable. Snce t s fundamental to make relable merges n the early stages of the mergng process, we decded to use the relatvely large value of 6 for S mn. Comparable results were obtaned wth value 8, whereas hstograms of 4x4 blocks turned out to be too nosy n some cases. The choce of S max s less crucal, and we chose to use 64. Fg. 2b llustrates the result of the herarchcal splttng algorthm. As expected, the splttng goes deepest around the texture boundares. Note that the herarchcal splttng phase s not mandatory, but we could skp t by dvdng the nput mage drectly to blocks of sze S mn and the successve agglomeratve mergng phase would stll succeed. Ths s partcularly true for easer problems of homogeneous and clearly dstnct textures. However, our experments have shown that fndng larger areas of unform texture wth the herarchcal splttng method mproves the convergence of the agglomeratve mergng algorthm. 3.2 Agglomeratve Mergng Once the mage has been splt nto blocks of roughly unform texture, we apply an agglomeratve mergng procedure, whch merges smlar adjacent regons untl a stoppng crteron s satsfed. At a partcular stage of the mergng, we merge that par of adjacent segments, whch has the smallest Merger Importance (MI) value. MI s defned as MI = p G (3) where p s the number of pxels n the smaller of the two regons and G s the dstance measure defned n Eq.. In other words, at each step the procedure chooses that merger of all possble mergers, whch ntroduces the smallest change n the segmented mage. Once the par of adjacent segments wth the smallest MI value has been found, the regons are merged and the two respectve LBP/C hstograms are summed to be the hstogram of the new mage regon. Before movng to the next merger we compute the G dstances between the new regon and all adjacent regons to t. Mergng s allowed to proceed untl the stoppng rule MI cur MIR = ---------------- > Y (4) MI max

trggers. Mergng s halted f MIR, the rato of MI cur, Merger Importance for the current best merge, and MI max, the largest Merger Importance of all precedng mergers, exceeds a preset threshold Y. In theory, t s possble that the very frst merges have a zero MI value (.e. there are adjacent regons wth dentcal LBP/C hstograms), whch would lead to a premature termnaton of the agglomeratve mergng phase. To prevent ths the stoppng rule s not evaluated for the frst % of all possble merges. The value of threshold Y was determned expermentally. We appled the algorthm to numerous texture mages and examned the values of MIR durng the mergng process. The concluson was that for well-defned homogeneous textures MIR values up to.5 or.6 were stll acceptable, whle values over 2. generally were due to a percevable dfference n the texture contents of the adjacent regons of the current best merge. Based on ths observaton we chose value 2. for Y. In more general terms, threshold Y can be nterpreted as the scale of texture dfferences we want to dscrmnate, and thus the value of Y may be a very subjectve decson. Ths s partcularly the case wth outdoor scenes and rregular textures, where the number of dstnct regons n the segmentaton result strongly correlates wth the threshold. Fg. 2c shows the result of the agglomeratve mergng phase after 74 merges. The MIR of the 75th merge (MIR stop ) s 9.5 and the mergng s halted. The hghest MIR value up to that pont (MIR h ) had been.2 (Fg. 3). The relatonshp between MIR stop, MIR h and threshold Y reflects the relablty of the result of the agglomeratve mergng phase. The very large value of MIR stop and very small value of MIR h underlne the easness wth whch the rough estmate of the texture regons s obtaned for mosac #. Note that the segmentaton error of.4% after the agglomeratve clusterng phase (ERR a ) s somewhat based n ths problem, for the horzontal and vertcal texture boundares are accdentally algned wth the ntal blocks. Fg. 3. Plot of MIR. 3.3 Pxelwse Classfcaton To mprove the localzaton of the boundares a smple pxelwse classfcaton algorthm s used. If the herarchcal splttng and agglomeratve mergng phases have succeeded, we have obtaned qute relable estmates of the dfferent textured regons present n the mage. Treatng the LBP/C hstograms of the mage segments as our texture models we swtch nto a texture classfcaton mode. If an mage pxel s on the boundary of at least two dstnct textures (.e. the pxel s 4-connected to at least one pxel wth a dfferent label), we place a dscrete dsc wth radus r on the pxel and compute the LBP/C hstogram over the dsc. We compute the G dstances between the hstogram of the dsc and the models of those regons, whch are 4-connected to the pxel n queston. We relabel the pxel, f the label of the nearest model s dfferent from the current label of the pxel and there s at least one 4-connected adjacent pxel wth the tentatve new label. The latter condton mproves smooth adapton of texture boundares and decreases the probablty of small holes occurrng nsde the regons. If the pxel s relabeled,.e. t s moved from an mage segment to the adjacent segment, we update the correspondng texture models accordngly, hence the texture models become more accurate durng the process. Only those pxels at whch the dsc s entrely nsde the mage are examned, hence the fnal segmentaton result wll contan a border of r pxels wde. In the next scan over the mage we only check the neghborhoods of those pxels, whch were relabeled n the prevous sweep. The process of pxelwse classfcaton contnues untl no pxels are relabeled or maxmum number of sweeps s reached. Ths s set to be two tmes S mn, based on the reasonng that the boundary estmate

of the agglomeratve mergng phase can be at most ths far away from the true texture boundary. Settng an upper lmt for the number of teratons ensures that the process wll not wander around endlessly, f the dsc s not able to capture enough nformaton of the local texture to be stable. Accordng to our experments the algorthm generally converges quckly wth homogeneous textures, whereas wth locally stochastc natural scenes maxmum number of sweeps may be consumed. We dd not apply any postprocessng method to mprove the fnal segmentaton result, e.g. by smoothng the texture boundares or removng small regons as many exstng algorthms do. The relatonshp between the radus r of the dsc and the fnal segmentaton result s obvous. A very small dsc s unstable, producng ragged texture boundares and holes nsde regons, whereas a very large dsc s stable and produces smooth boundares, but may fal n locatng the boundares accurately. We used the fnal segmentaton error as a gudelne n choosng the value of r. Mosacs #4 (Fg. 7) and #5 (Fg. 8) were processed wth the pxelwse classfcaton algorthm, wth r rangng from to 2. The segmentaton errors are plotted as a functon of r n Fg. 4. As expected, the error frst decreases wth ncreasng dsc sze, reaches the mnmum and then slowly ncreases as the dsc becomes too large to locate the boundares accurately. Based on ths result we chose value for radus r. Fg. 4. Fnal segmentaton error as a functon of the radus r of the dsc. The sold lne corresponds to mosac #5 and the dashed lne corresponds to mosac #6. Fg. 2d shows the fnal segmentaton result after the pxelwse classfcaton phase, where 6 sweeps were needed. The fnal segmentaton error (ERR p ), computed over the area processed by the dsc whch excludes the border of r pxels, s.7%. 4 Expermental Results The segmentaton results for four addtonal texture mosacs and two natural scenes are presented. The same set of parameter values was used for all texture mosacs to demonstrate the robustness of the approach: b=8, S max =64, S mn =6, X=.2, Y=2., and r=. In each case we provde the orgnal mage, the rough segmentaton result after the agglomeratve mergng phase and the fnal segmentaton result after the pxelwse classfcaton phase. The segmentaton results are superpostoned on the orgnal mage. Parameters descrbng segmentaton of texture mosacs are gven n Table. Mosac #2 (Fg. 5a) s a 52x52 mage contanng four textures made by a GMRF process and a crcle of panted surface n the mddle (32). The more dffcult nature of ths problem shows n the values of MIR stop (5.2) and MIR h (.6), whch are clearly closer to threshold Y than what was the case wth mosac #. Nevertheless, the rough segmentaton result (Fg. 5b) wth segmentaton error of 4.2% s qute decent. The fnal segmentaton result (Fg. 5c) after 23 sweeps wth segmentaton error of.2% s excellent. Mosac #3 (Fg. 6a) s a 52x52 mage wth a background made by a GMRF process and four dstnct regons; the square and the crcle are panted surfaces wth dfferent surface roughnesses and the ellpse and the trangle are made by a fractal process (32). As we can see from the values of MIR stop (8.) and MIR h (.2), the rough estmate (Fg. 6b) of the texture regons s obtaned relatvely easly. The fnal segmentaton result (Fg. 6c) after 3 sweeps contans only.9% msclassfed pxels.

Fg. 5. Texture mosac #2. Fg. 6. Texture mosac #3. Mosacs #4 (Fg. 7a) and #5 (Fg. 8a) are composed of textures taken from outdoor scenes(2). In ther study Jan and Karu tackled the problem of texture segmentaton wth a neural network generalzaton of the tradtonal multchannel flterng method, usng varous flter banks for feature extracton. For mosac #4, whch s 256x256 pxels n sze, they obtaned a segmentaton error of 3.3% wth learned masks n unsupervsed mode. Ther method was not strctly unsupervsed, though, because the number of clusters was manually set to fve. Our method acheves a smaller msclassfcaton error of.4% (Fg. 7c). The dfference between MIRstop (4.) and MIRh (.3) s consderable, whch reflects the relablty of the analyss. Fg. 7. Texture mosac #4.

For mosac #5, whch s 384x384 pxels n sze, Jan and Karu reported a labelng error of 6% wth Laws flters n supervsed mode. Our unsupervsed method gves a clearly better segmentaton result of 2.%. Note that the pxelwse classfcaton clearly mproves the result of the agglomeratve mergng phase (7.8%). The dfference between MIRstop (2.8) and MIRh (.2) s stll notceable, but by far the smallest n the three cases, reflectng the nherent dffculty of ths problem. Fg. 8. Texture mosac #5. Table : Parameters descrbng segmentaton of mosacs #-#5. mosac MIRstop MIRh ERRa(%) ERRp(%) sweeps # 9.5.2.4.7 6 #2 5.2.6 4.2.2 23 #3 8..2 4.6.9 3 #4 4..3 2..4 9 #5 2.8.2 7.8 2. 24 LBP/C transform s by defnton rotaton-varant. Fg. 9 demonstrates how the segmentaton algorthm works n the case of an edge between two dfferent orentatons of a drectonal texture. Fg. 9. Segmentaton of a mosac whch contans two dfferent orentatons of a drectonal texture. The orgnal texture (D2 from the Brodatz album) was rotated 3 degrees n both clockwse and counterclockwse drecton usng cubc nterpolaton, and the rotated textures were merged nto the 2x2 mosac shown n

Fg. 9a. Ths sze guarantees that the edge between the two orentatons s not accdentally algned wth the ntal blocks whch could bas the result. The same set of parameter values was used as wth texture mosacs #-#5. The fnal segmentaton result n Fg. 9b contans 66 mslabelled pxels along the edge. We also appled the texture segmentaton method to natural scenes. The scenes were orgnally n RGB format(25), but we converted them to gray level ntensty mages. As an example, scene # (Fg. a) s a 384 x 384 mage of rocks n the sea and scene #2 (Fg. a) s a 92x92 mage of a beach, water and folage. As we can observe from the mage, the textures of natural scenes are generally more nonunform than the homogeneous textures of the test mosacs. Also, n natural scenes adjacent textured regons are not necessarly separated by welldefned boundares, but the spatal pattern smoothly changes from one texture to another. Further, we have to observe the nfnte scale of texture dfferences present n natural scenes; choosng the rght scale s a very subjectve matter. For these reasons there s often no correct segmentaton for a natural scene, as s the case wth texture mosacs. Fg.. Natural scene #. The parameters X and Y prmarly control the scale of texture dfferences that wll be detected. Wth values X=. and Y=.5 the rough segmentaton results after the agglomeratve mergng phase are presented n Fg. b and Fg. b, and the fnal segmentaton results are shown n Fg. c and Fg. c, respectvely. If we decreased Y further, the segmentaton result would contan an ncreasng number of regons. The nvarance of the LBP/C transform to average gray level shows n the bottom part of the mage, where the sea s nterpreted as a sngle regon despte the shadows. The results obtaned for these natural scenes are very satsfactory, consderng that mportant color or gray scale nformaton s not utlzed n the segmentaton. Fg.. Natural scene #2.

5 Dscusson In the presented method texture s descrbed by jont occurrences of LBP and C. Obvous generalzatons are to use other texture features or feature domans (e.g. color) and scale. Although LBP/C s a very powerful texture transform, we expect to acheve better results by combnng a larger number of features n the analyss. Other powerful texture measures, lke dstrbutons based on gray level dfference hstograms or co-occurrence matrces, can be easly ncorporated nto our algorthm. In Petkänen et al. (33) we demonstrated that a method based on comparson of feature dstrbutons can be used for hgh-accuracy color measurements. Ths suggests that dstrbutons of color features could be easly used to fnd small color dfferences between neghborng regons n segmentaton. Color features should make our method effcent for segmentng mages contanng color textures, lke the orgnal color mages used n the experments (25). Further, we could consder a partcular feature at multple scales, by straghtforwardly computng the desred feature for sutably symmetrcal dscrete neghborhoods of any sze, such as dsks or boxes of odd or even sze. A smple way to defne a multresoluton LBP would be to choose the eght neghbors of the center pxel from the correspondng postons n dfferent neghborhoods (3x3, 5x5, 7x7, etc.). The remanng queston s how to combne the multple feature channels obtaned wth several features and/or scales. We can hardly expect to relably estmate jont dstrbutons for a large number of features. Also, multdmensonal hstograms wth large numbers of bns are very computatonally ntensve and consume very much memory. An alternatve s to use an approxmaton wth margnal dstrbutons and to employ each ndependent feature separately, as a -D hstogram, to compute a smlarty score such as G for each feature, and then ntegrate ndvdual scores nto an aggregate smlarty score. Ths approach has gven very promsng results n our texture classfcaton experments (29) and more recently n color classfcaton as well (33). Combnng t wth a carefully chosen set of non-redundant complementary features we expect to mprove the performance of our segmentaton method consderably. In a smlar way, jont pars of features, lke LBP/C, can be combned wth other sngle features or feature pars. It would also be possble to use sngle features or jont features one by one, by e.g. frst comparng the unformty of regons wth respect to texture and then wth respect to color. The hstogram comparson approach based on the G test could be replaced wth some other related method, lke hstogram ntersecton (34) or a statstcal ch-square test. Accordng to our experence the choce of proper texture measures s usually a much more mportant factor n texture dscrmnaton than the partcular method used for hstogram comparson (29). However, t would be nterestng to study the performances of dfferent types of approaches n the case of very small mage wndows. In recent texture classfcaton studes (35) we have compared the performance of LBP and other operators to that of GMRF and Gabor energy features. The mage data ncluded both Brodatz textures and the many dfferent texture mages avalable at MeasTex (36) whch s an ndependent texture classfcaton algorthm evaluaton ste accessble n WWW. LBP dd well n these classfcaton experments whch suggests that t should be sutable for texture segmentaton as well. 6 Concluson We proposed a soluton to unsupervsed texture segmentaton, n whch a method based on comparson of feature dstrbutons s used to fnd homogeneously textured mage regons and to localze boundares between regons. Texture nformaton s measured wth a method based on local bnary patterns and contrast (LBP/C) that we have recently developed. A regon-based algorthm s developed for coarse mage segmentaton and a pxelwse classfcaton scheme for mprovng the localzaton of regon boundares. The method performed very well n experments. It s not senstve to the selecton of parameter values, does not requre any pror knowledge about the number of textures or regons n the mage, and seems to provde sgnfcantly better results than exstng unsupervsed texture segmentaton approaches. The method can be easly generalzed, e.g., to utlze other texture features, multscale nformaton, color features, and combnatons of multple features. Acknowledgements The fnancal support provded by the Academy of Fnland, the Technology Development Center of Fnland, and the Graduate School n Electroncs, Telecommuncatons and Automaton s gratefully acknowledged. The authors also wsh to thank followng persons for provdng mages used n ths study; Rchard C. Dubes, Anl K. Jan, John Lees and Kalle Karu from the Mchgan State Unversty and Glenn Healey and Davd Slater from the Unversty of Calforna at Irvne. References

. M. Tuceryan and A.K. Jan, Texture analyss, Chapter 2. n Handbook of Pattern Recognton and Computer Vson, C.H. Chen, L.F. Pau, P.S.P. Wang (eds.), World Scentfc, Sngapore, pp. 235-276 (993). 2. R.M. Haralck and L.G. Shapro, Image segmentaton technques, Computer Vson, Graphcs, and Image Processng 29, pp. -32 (985). 3. T.R. Reed and J.M.H. Du Buf, A revew of recent texture segmentaton and feature extracton technques, CVGIP Image Understandng 57, pp. 359-372 (993). 4. N.R. Pal and S.K. Pal, A revew on mage segmentaton technques, Pattern Recognton 26, pp. 277-294 (993). 5. P.C. Chen and T. Pavlds, Segmentaton by texture usng a co-occurrence matrx and a splt-and-merge algorthm, Computer Graphcs and Image Processng, pp. 72-82 (979). 6. M. Petkänen and A. Rosenfeld, Image segmentaton by texture usng pyramd node lnkng, IEEE Transactons on Systems, Man, and Cybernetcs, pp. 822-825 (98). 7. L.S. Davs and A. Mtche, Mtes: A model drven, teratve texture segmentaton algorthm, Computer Graphcs and Image Processng 9, pp. 95- (982). 8. M. Spann and R. Wlson, A quad-tree approach to mage segmentaton whch combnes statstcal and spatal nformaton, Pattern Recognton 8, pp. 257-269 (985). 9. M. Unser and M. Eden, Multresoluton feature extracton and selecton for texture segmentaton, IEEE Transactons on Pattern Analyss and Machne Intellgence, pp. 77-728 (989).. J.Y. Hsao and A.A. Sawchuk, Unsupervsed texture mage segmentaton usng feature smoothng and probablstc relaxaton technques, Computer Vson, Graphcs, and Image Processng 48, pp. -2 (989).. A. Khotanzad and A. Bouarfa, A parallel non-parametrc clusterng algorthm wth applcaton to mage segmentaton, Proc. 22nd Aslomar Conference on Sgnals, Systems, and Computers, pp. 35-39, Pacfc Grove, CA (988). 2. J. Mao and A.K. Jan, Texture classfcaton and segmentaton usng multresoluton smultaneous autoregressve models, Pattern Recognton 25, pp. 73-88 (992). 3. J.F. Slverman and D.B. Cooper, Bayesan clusterng for unsupervsed estmaton of surface and texture models, IEEE Transactons on Pattern Analyss and Machne Intellgence, pp. 482-495 (988). 4. C. Bouman and B. Lu, Multple resoluton segmentaton of textured mages, IEEE Transactons on Pattern Analyss and Machne Intellgence 3, pp. 99-3 (99). 5. B.S. Manjunath and R. Chellappa, Unsupervsed texture segmentaton usng Markov random feld models, IEEE Transactons on Pattern Analyss and Machne Intellgence 3, pp. 478-482 (99). 6. F.S. Cohen and Z. Fan, Maxmum lkelhood unsupervsed textured mage segmentaton, CVGIP: Graphcal Models and Image Processng 54, pp. 239-25 (992). 7. A. Bovk, M. Clark and W.S. Gesler, Multchannel texture analyss usng localzed spatal flters, IEEE Transactons on Pattern Analyss and Machne Intellgence 2, pp. 55-73 (99). 8. A.K. Jan and F. Farrokhna, Unsupervsed texture segmentaton usng Gabor flters, Pattern Recognton 24, pp. 67-86 (99). 9. H. Greenspan, R. Goodman, R. Chellappa and C.H. Anderson, Learnng texture dscrmnaton rules n a multresoluton system, IEEE Transactons on Pattern Analyss and Machne Intellgence 6, pp. 894-9 (994). 2. A.K. Jan and K. Karu, Learnng texture dscrmnaton masks, IEEE Transactons on Pattern Analyss and Machne Intellgence 8, pp. 95-25 (996). 2. M. Unser, Texture classfcaton and segmentaton usng wavelet frames, IEEE Transactons on Image Processng 4, pp. 549-56 (995). 22. A. Lane and J. Fan, Frame representatons for texture segmentaton, IEEE Transactons on Image Processng 5, pp. 77-78 (996). 23. B.B. Chaudhur and N. Sarkar, Texture segmentaton usng fractal dmenson, IEEE Transactons on Pattern Analyss and Machne Intellgence 7, pp. 72-77 (995). 24. J.-L. Chen and A. Kundu, Unsupervsed texture segmentaton usng multchannel decomposton and hdden Markov models, IEEE Transactons on Image Processng 4, pp. 63-69 (995). 25. D.K. Panjwan and G. Healey, Markov random feld models for unsupervsed segmentaton of textured color mages, IEEE Transactons on Pattern Analyss and Machne Intellgence 7, pp. 939-954 (995). 26. D. Harwood, T. Ojala, M. Petkänen, S. Kelman and L.S. Davs, Texture classfcaton by center-symmetrc auto-correlaton, usng Kullback dscrmnaton of dstrbutons, Pattern Recognton Letters 6, pp. -, (995). 27. T. Ojala, M. Petkänen and D. Harwood, A comparatve study of texture measures wth classfcaton based on feature dstrbutons, Pattern Recognton 29, pp. 5-59 (996). 28. T. Ojala, M. Petkänen and J. Nsula, Determnng composton of gran mxtures by texture classfcaton based on feature dstrbutons, Internatonal Journal of Pattern Recognton and Artfcal Intellgence, pp.

73-82 (996). 29. T. Ojala, Multchannel approach to texture descrpton wth feature dstrbutons, Techncal Report CAR-TR- 846, Center for Automaton Research, Unversty of Maryland (996). 3. R.R. Sokal and F.J. Rohlf, Introducton to Bostatstcs, 2nd ed. W.H. Freeman and Co, New York (987). 3. P. Brodatz, Textures: A Photographc Album for Artsts and Desgners. Dover Publcatons, New York (966). 32. P.P. Ohanan and R.C. Dubes, Performance evaluaton for four classes of textural features, Pattern Recognton 25, pp. 89-833 (992). 33. M. Petkänen, S. Nemnen, E. Marszalec and T. Ojala, Accurate color dscrmnaton wth classfcaton based on feature dstrbutons, Proc. 3th Internatonal Conference on Pattern Recognton, vol. 3, pp. 833-838, Venna, Austra (996). 34. M. Swan and D. Ballard, Color ndexng, Internatonal Journal of Computer Vson 7, pp. -32 (99). 35. T. Ojala, Nonparametrc texture analyss usng spatal operators, wth applcatons n vsual nspecton, Ph.D. thess, Acta Unv. Oul. C 5, Unversty of Oulu, Fnland (997). 36. G. Smth and I. Burns, MeasTex Image Texture Database and Test Sute, CSSIP, Unversty of Queensland, Australa, http://www.cssp.elec.uq.edu.au/~guy/meastex/home.html.