44 JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 A ew Color-Texture Approach for Industral Products Inspecton Moulay A. Akhlouf 1, Centre de Robotque et de Vson Industrelles and Computer Vson Lab, Laval Unversty, Quebec, Canada 1 Emal: moulay.akhlouf@crv.ca Xaver Maldague and Wael Ben Larb Computer Vson Lab, Laval Unversty, Quebec, Canada Emal: {akhlouf, maldagx, wblarb}@gel.ulaval.ca Abstract Ths work presents an approach for color-texture classfcaton of ndustral products. An extenson of Gray Level Co-occurrence Matrx (GLCM) to color mages s proposed. Statstcal features are computed from an sotropc Color Co-occurrence Matrx for classfcaton. The followng color spaces are used: RGB, HSL and La*b*. ew combnaton schemes for texture analyss are ntroduced. A comparson wth Local Bnary Patterns (LBP) s also performed. The tests were conducted n a varety of ndustral samples. The obtaned results are promsng and show the possblty of effcently classfyng complex ndustral products based on color and texture features. Index Terms Color vson, texture analyss, classfcaton, statstcal features extracton. I. ITRODUCTIO Texture and color analyss have been wdely studed n the lterature. Each of the two domans was studed ndependently. In recent years, we see an ncrease of nterest n the use of both color and texture n mage analyss. Most of these work done by the research communty n these areas used smple mages or avalable mage databases. The applcaton here s of ndustral nature and the obects to analyze are made of varous materals. The obectve of ths applcaton s to classfy these products usng color and texture cues. The problem s complex because the non homogeneous nature of the products at hand. For example, roofng shngles show a dfference n the spatal dstrbuton of color and texture even for smlar products. Many methods have been proposed n order to handle machne vson problems where color and texture features serve as a cue for classfcaton, segmentaton and recognton. Varous statstcal descrptors have been proposed for the measure of mage textures [1]-[7], [1]. Theses statstcal approaches use n-order statstcs to defne mage textures. Other approaches defne textures by means of mathematcal morphology operators [8], [9] or Flter Banks [1]-[13]. These technques were frst proposed for processng grayscale mages, and then were extended to color texture processng. Some authors proposed the use of a combnaton of color and texture features. Texture features were computed n grayscale and combned wth color hstograms and moments [14]-[17]. These combned features are then sent to a classfer for color-texture classfcaton. Other authors proposed the use of color quantzaton to reduce the number of colors and process the resultng mage as grayscale for texture extracton [18]-[4]. More sophstcated technques use a combnaton n between color bands for texture features computaton [5]-[9]. In ths work we present a new and effectve framework for color-texture classfcaton of ndustral products. The proposed Color Co-occurrence Matrx (CCM) approach uses statstcal features computed from an sotropc cooccurrence matrx extracted from color bands and combned wth color mage entropes. A comparatve study s also performed between the proposed approach [35] based on Gray Level Co-occurrence Matrx (GLCM) and Local Bnary Patterns (LBP) texture analyss ntroduced n [6]. Ths study permts to show that some technques are more sutable for the analyss of complex ndustral products where the color and texture dstrbuton are non homogenous and where slght varaton n color for smlar samples s present. These texture analyss technques were extended to process color mages. The comparson has been conducted n the followng color spaces: RGB, HSL and La*b* [3], [31] wth nterestng results. A varety of ndustral products provded by an ndustral partner were used n our experments. The remanng of ths paper s organzed as follows: In secton, we ntroduce the proposed framework for color-texture classfcaton. In secton 3, the theoretcal background for texture analyss s prsented. In sectons 4 and 5, we present the color-texture analyss and classfcaton scheme. Expermental results are dscussed n secton 6. The mage database and the expermental setup are presented n ths secton. Secton 7 concludes ths work.
JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 45 Fgure 1. Framework for color-texture classfcaton. II. COLOR-TEXTURE CALSSIFICATIO FRAMEWORK In ths work color-texture classfcaton s performed and comparson s conducted between the proposed Color Co-occurrence Matrx (CCM) and Local Bnary Patterns (LBP) texture analyss. The Color Co-occurrence Matrx (CCM) was frst proposed n [35]. Ths technque extends Gray Level Cooccurrence Matrx (GLCM) to process color mages and combne t wth color mage entropes for color-texture classfcaton. Local Bnary Patterns (LBP) texture analyss was ntroduced n [6]. Ths technque s wdely used by the research communty for texture classfcaton. In ths work we extend the LBP approach to process color mages. For each color-texture classfcaton technque we compare smlar and non smlar mages. The obtaned dstances are normalzed n order to show the dscrmnaton power of each method. ormalzaton s performed by dvdng each dstance wth the largest obtaned dstance. The classfcaton framework s gven n Fg 1. The algorthm steps are gven below: 1. Extract color bands from a color mage (R, G, B or H, S, L or L, a*, b*) [1].. Apply the texture analyss technque to each band. 3. Compute the texture features F for each processed color band. 4. Compute the Eucldan dstance between the same features n smlar bands of two dfferent mages: = ( f ) f. 5. The fnal dstance s the sum of the three bands dstances. A. Gray Level Co-occurrence Matrx (GLCM) Gray Level Co-occurrence Matrx was proposed n [] by Haralck and s wdely used for texture analyss. It estmates the second order statstcs related to mage propertes by consderng the spatal relatonshp of pxels. GLCM depcts how often dfferent combnatons of gray levels co-occur n an mage. The GLCM s created by calculatng how often a pxel wth the ntensty value occurs n a specfc spatal relatonshp to a pxel wth the value. The spatal relatonshp can be specfed n dfferent ways, the default one s between a pxel and ts mmedate neghbor to ts rght. However we can specfy ths relatonshp wth dfferent offsets and angles. The pxel at poston (,) n GLCM s the sum of the number of tmes the (,) relatonshp occurs n the mage. Fg descrbes how to compute the GLCM. It shows an mage and ts correspondng co-occurrence matrx usng the default pxels spatal relatonshp (offset = +1 n x drecton). For the par (,1) (pxel followed at ts rght by pxel 1), t s found tmes n the mage, then the GLCM mage wll have as a value n the poston correspondng to I =1 and I =. The GLCM matrx s a 56x56 matrx; I and I are the ntensty values for an 8bt mage. The GLCM can be computed for the eght drectons around the pxel of nterest (Fg 3). Summng results from dfferent drectons lead to the sotropc GLCM and help acheve a rotaton nvarant GLCM (Fg 4). The followng sectons show how to compute colortexture features for classfcaton. III. TEXTURE AALYSIS TECHIQUES Texture analyss s an area of computer vson that has attracted a lot of nterest. Many technques are avalable n the lterature. In ths work we ntroduce a new approach for color-texture classfcaton: Color Cooccurrence Matrx (CCM) [35] and compare ts performance wth the Local Bnary Patterns (LBP) texture analyss technque [6]. In the followng sectons we present each technque and the statstcal features representng texture attrbutes that can be extracted from the texture mage. Fgure. Descrpton of the Gray Level Co-occurrence Matrx. Fgure 3. Drectons used for computng sotropc GLCM.
46 JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 (a) Orgnal (b) GLCM Red (b) GLCM Green (d) GLCM Blue Fgure 4. Example of sotropc GLCM mages. B. Local Bnary Patterns (LBP) The local bnary pattern (LBP) texture analyss operator was ntroduced n [6]. It s a gray-scale nvarant texture measure computed from the analyss of a 3x3 local neghbourhood over a central pxel. The LBP s based on a bnary code descrbng the local texture pattern. Ths code s bult by thresholdng a local neghbourhood by the gray value of ts center. The eght neghbours are labelled usng a bnary code {, 1} obtaned by comparng ther values to the central pxel value. If the tested gray value s below the gray value of the central pxel, then t s labelled, otherwse t s assgned the value 1: ' f I(x,y ) < I(x,y P = 1 otherwse ' P s the obtaned bnary code, ) (1) P s the orgnal pxel value at poston and P s the central pxel value. Wth ths technque there s 56 ( 8 ) possble patterns (or texture unts). The obtaned value s then multpled by weghts gven to the correspondng pxels. The weght s gven by the value -1. Summng the obtaned values gves the measure of the LBP: 8 = ' 1 LBP P = 1 I () Fg 5 shows an example on how to compute LBP. The orgnal 3x3 neghbourhood s gven n Fg 5 (a). The central pxel value s used as a threshold n order to assgn a bnary value to ts neghbours. Fg 5 (b) shows Fgure 5. Computaton of LBP. the result of thresholdng the 3x3 neghbourhood. The obtaned values are multpled by ther correspondng weghts. The weghts kernel s gven by Fg 5 (c). The result s gven n Fg 5 (d). The sum of the resultng values gves the LBP measure (169). The central pxel s replaced by the obtaned value. A new LBP mage s constructed by processng each pxel and ts 3x3 neghbours n the orgnal mage. Fg 6 shows an example of the resultng LBP mages. C. Texture statstcal features From the texture mage we can compute dfferent statstcal measures. The followng are used n ths work: 1. Entropy:, = Entropy = ln( P ) P (3). Energy: ( P, = Energy = ) (4) 3. Contrast:, = Constrast = P ( ) (5) 4. Homogenety: P Homogenet y = (6), = 1+ ( ) 5. Correlaton: ( µ )( µ ) Correlaton = P (7), = σ Where: P s the pxel value n poston (,) n the texture mage. s the umber of gray levels n the mage., = 1 µ = P s the texture mage mean. σ, = = P ( µ) s the texture mage varance. D. Image entropy In the proposed approach [35], we add orgnal mage entropy n the computaton of color-texture dstance. Ths statstcal feature has shown to add more
JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 47 dscrmnaton power to the classfcaton scheme. The same approach s used n both texture analyss technques: GLCM and LBP. The entropy equaton s gven n (3) above. IV. COLOR-TEXTURE AALYSIS The texture analyss technques presented above (GLCM and LBP) have been defned for grayscale mages. We use a smple extenson of these technques to color mages. In color space the texture analyss technques and ther statstcal features are computed for each band. Comparsons can then be done between smlar bands from two dfferent mages for classfcaton. Color mages can be represented n dfferent color spaces. The color model s an abstract mathematcal model for color representaton as vector of numbers. Many color spaces are used n computer vson. In ths work, we compared the performance of these texture analyss technques n the followng color spaces: RGB, HSL and La*b*[1], [3]-[31]. RGB color space s the classcal color model for color mage representaton. It uses an addtve color mxng model of red, green and bleu colors. Ths color mage model s provded drectly by the cameras. HSL color model represent the hue, saturaton and lumnance obtaned form color mages. HSL s obtaned from RGB usng color converson equatons [1], [3]- [31]. Ths model s wdely used n computer vson for color processng. La*b* color model s a standard color space desgned by Internatonal Commsson on llumnaton. It s perceptually unform, and ts L component closely matches human percepton of lghtness. Ths model s usually used as a reference for color dfference computaton [1], [3]-[31]. V. TEXTURE CLASSIFICATIO In order to classfy the texture mages, a dstance s computed between ther extracted features. Ths dstance s obtaned by computng the Eucldan dstance between smlar features [35]: texture = ( F F ) (8) Where: (a) Orgnal (b) LBP Red (b) LBP Green (d) LBP Blue Fgure 6. Example LBP mages. [ Entropy, Energy, Contrast, Homogenety Correlaton] F =, s the features vector ( for mage 1 and for mage ). The global texture dstance for each color band s computed by addng to the GLCM and LBP features dstance, the square dfference between entropes of the orgnal band mage: = + Entrop Entrop ) (9) b texture ( Where: Entrop s the entropy of the orgnal band mage ( for mage 1 and for mage ). For color mage classfcaton, we extend the features dstance to the color space. The dstance s obtaned by computng the Eucldan dstance between smlar features for smlar bands. The global color-texture dstance s the sum of each color band dstance: = + + (1) Color Texture b1 b b3 VI. EXPERIMETAL RESULTS For Image acquston and processng, the followng setup was used: 1. JAI 3-CCD hgh performance camera wth 14x768 resoluton and a camera lnk nterface.. 16mm F1.4 lens; 3. Fber optc dffuse coaxal lghtng wth W lght source; 4. A Camera lnk frame grabber; 5. Intel Pentum P4 processor. GHz and GB RAM; 6. evson C++ lbrary for mage acquston; 7. Matlab 7.3 (Release 6B) and Image Processng Toolbox. The samples were placed at approxmately cm from the camera. The color-texture analyss algorthms were mplemented usng Matlab 7.3, the Image Processng Toolbox and C++. The followng ndustral products were used n our experments: Roofng shngles, wood, organc fbers and fabrc. About 65 mages were collected for our tests. Examples of these mages are gven n Fg 7. The frst tests were conducted n RGB, HSL and La*b* color spaces. RGB color space gave the best results, followed closely by HSL color space (Table I) [35]. La*b* color space performed the worst. The remanng tests were done n RGB color space.
48 JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 (a) Example of Wood samples (b) Example of Roofng shngles samples (c) Example of Organc fbers samples (d) Example of Fabrc samples Fgure 7. Example of ndustral products used n our experments. TABLE I PERFORMACE OF COLOR SPACES USIG GLCM FOR ROOFIG SHIGLES CCM RGB HSL La*b* Roofng shngles 94% 93% 71% Color-texture classfcaton experments were conducted n the avalable samples, success rates are gven n Table II. The obtaned results show that CCM outperforms LBP color-texture classfcaton technque. Tests conducted n other mages n the database lke lamnates and leather gave smlar nterestng results. Ths s very promsng for our color-texture classfcaton framework of complex ndustral products. TABLE II SUCCESS RATE OF COLOR-TEXTURE CLASSIFICATIO TECHIQUES I RGB COLOR SPACE CCM LBP Roofng shngles 94% 83% Wood 9% 75% Organc fbers 95% 85% Fabrc 91% 79% The Color sotropc Co-occurrence Matrx (CCM) performed the best n RGB and HSL color spaces. In the La*b* color space the results does not permt to effectvely dscrmnate between close color-texture samples. Classfcaton between mages was done successfully usng a sum of color bands dstances computed usng sotropc GLCM. For each channel a sum of the Eucldan dstance between GLCM features and the
JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 49 square dfference between entropes of the orgnal band mages was used. Also of mportance s the lghtng used durng the classfcaton. We used a fber optc coaxal dffuse lghtng that permts a unform dffuse lghtng over the nspected surface. Ths lghtng setup gave the best result n our tests. Other lghtng technques were also tested. Dffuse hgh frequency stablzed full spectrum lghtng provded the second best results. Poor results were obtaned wth fluorescent lghtng. VII. COCLUSIO In ths work we presented a new framework for colortexture classfcaton of ndustral products. A comparatve study was conducted between the proposed color co-occurrence matrx (CCM) approach and local bnary patterns (LBP) texture analyss approach. Also, new combnaton schemes for texture analyss were ntroduced. We conducted expermental tests wth a varety of ndustral samples. The obtaned results are promsng and show the possblty of effcently classfyng complex ndustral products where non homogenous color and texture dstrbutons are present. The proposed CCM approach for texture analyss provde an advantageous scheme for effectvely encodng complex non homogenous color and textures where rregular components are present and also textures lackng repettve patterns. ACKOWLEDGMET Ths work has been supported by a grant from Quebec Mnstry of Educaton - PART grant: PART635. Specal thanks to M. Trudel, for provdng product samples for our experments and to CRVI techncans J. Pouln and L.A. Marceau (http://www.crv.ca) who helped n buldng the mage database. REFERECES [1] R.C. Gonzalez, and R. E. Woods, Dgtal Image Processng nd Edton. Prentce Hall,. [] R.M. Haralck, K.Shanmugam, and I. Dnsten, Textural Features for Image Classfcaton, IEEE Trans. on Systems, Man and Cybernetcs, 1973, pp.61-61. [3] T. Caell, and D. Reye, On the classfcaton of mage regons by color, texture and shape, Pattern Recognton, 6 (4), 1996, pp. 461-47. [4] M. Unser Sum and Dfference Hstograms for Texture Classfcaton, IEEE Trans. on Pattern Analyss and Machne Intellgence, ew York, 1986. [5] T. Oala, and M Petkanen, Unsupervsed texture segmentaton usng feature dstrbutons, Pattern Recognton 3 (3), 1999, pp. 477-486. [6] T. Oala, M Petkanen., and D. Harwood, A comparatve study of texture measures wth classfcaton based on feature dstrbutons, Pattern Recognton 9 (1), 1996, pp. 51-59. [7] L.J. Van Gool, P. Dewaele and A. Oosterlnck, Texture analyss, Computer Vson, Graphcs, and Image Processng 9 (3), 1985, pp. 336-357. [8] A. Hanbury, U. Kandaswamy and D. A. Aderoh, Illumnaton-nvarant Morphologcal Texture Classfcaton, 7th Internatonal Symposum on Mathematcal Morphology, Pars, France 5. [9] S. Maumdar, and D.S. Jayad, Classfcaton of cereal grans usng machne vson: Combned morphology, color, and texture models, Trans. of the Amercan Socety of Agrcultural Engneers, vol. 43, 6,, pp. 1689-1694. [1] M. Varma, and A. Zsserman, Texture Classfcaton: Are Flter Banks ecessary?, IEEE Conference on Computer Vson and Pattern Recognton, 3. [11] C.J. Setchell and.w. Campbell, Usng Colour Gabor Texture Features for Scene Understandng, 7th. Internatonal Conference on Image Processng and ts Applcatons, 1999, pp. 37-376. [1] D. Denns, W.E. Hggns, and J. Wakeley, Texture segmentaton usng D Gabor elementary functons, IEEE Trans. on Pattern Analyss and Machne Intellgence, Vol. 16 (), 1994, pp. 13-149. [13] J.H. Husoy, T. Randen, and T.O. Gulsrud, Image texture classfcaton wth dgtal flter banks and transforms, Proc. SPIE, Applcatons of Dgtal Image Processng XVI, Vol. 8, 1993, pp. 6-71. [14] M. Mrmehd and M. Petrou, Segmentaton of Color Textures, IEEE Trans. on Pattern Analyss and Machne Intellgence, Vol.,,, pp.14-159. [15] M. Petkanen, T. Maenpaaand J. Vertola, Color Texture Classfcaton wth Color Hstograms and Local Bnary Patterns, Workshop on Texture Analyss n Machne Vson,, pp.19-11. [16] M. Petkanen, T. Maenpaaand J. Vertola, Separatng color and pattern nformaton for color texture dscrmnaton, Internatonal Conference on Pattern Recognton,, pp. 668-671. [17] A. Dmbarean, and P. Whelan, Experments n color texture analyss, Pattern. Recognton Letters. Vol., 1. p.1161. [18] C.C. Chang, and L.L.Wang, Color texture segmentaton for clothng n a computer-aded fashon desgn system, Image and Vson Compung, 14 (9), 1996, pp. 685-7. [19] P. Chang and J. Krumm, Obect Recognton wth Color Cooccurrence Hstograms, IEEE Conference on Computer Vson and Pattern Recognton, 1999, pp. 498-54. [] K. Chen, and S. Chen, Color texture segmentaton usng feature dstrbutons, Pattern Recognton Letters, 3, 7,, pp. 755-771. [1] Y. Deng, and B.S.Manunath, Unsupervsed segmentaton of color-texture regons n mages and vdeo, IEEE Trans. on Pattern Analyss and Machne Intellgence, 1. [] Y. Deng, B.S. Manunath and H. Shn, Color mage segmentaton, IEEE Conference on Computer Vson and Pattern Recognton, 1999, pp.446-51. [3] Y. Deng and B.S. Manunath, An effcent lowdmensonal color ndexng scheme for regon based mage retreval, IEEE Internaonal Conference on Acoustcs, Speech and Sgnal Processng, 1999. [4] M. Hauta-Kasar, J. Parkknen, T. Jääskelänen,and R. Lenz, Generalzed cooccurrence matrx for multspectral texture analyss, 13th Internatonal Conference on Pattern Recognton, 1996, pp. 785-789. [5] G. Van de Wouwer, P. Scheunders, S. Lvens and D. Van Dyck, Wavelet correlaton sgnatures for color texture characterzaton, Pattern Recognton, 3(3), 1999, pp. 443-451. [6] G. Van de Wouwer, B. Weyn and D. Van Dyck, Multscale, asymmetry sgnatures for texture analyss, Internatonal Conference on Image Processng, 4, pp. 1517-15
5 JOURAL OF MULTIMEDIA, VOL. 3, O. 3, JULY 8 [7] C.R. Dyer, T. Hong and A. Rosenfeld, Texture classfcaton usng gray level cooccurrence based on edge maxma, IEEE Trans. Systems, Man, and Cybernetcs, 1, 198, pp. 158-163. [8] A. Rosenfeld, W. Cheng-Ye, A. Wu, Multspectral Texture, IEEE Trans. on Systems, Man and Cybernetcs, 1, 198, pp. 79-84. [9] V. Arvs, C. Deban, M. Berducatand A. Benass, Generalzaton of the cooccurrence matrx for colour mages : applcaton to colour texture classfcaton, Journal of Image Analyss and Stereology, vol. 3, 4, pp. 63-7. [3] S. Westland, and A. Rpamont, Computatonal Colour Scence. John Wley, 4. [31] A. Tremeau, C. Fernandez-Malogne, and P. Bonton, Image numérque couleur. Dunod, 4. [3] K.I. Laws, Textured mage segmentaton, Report 94, Image Processng Insttute, Unv. of Southern Calforna, 198. [33] M.J. Chantler, The effect of varaton n llumnant drecton on texture classfcaton, Ph.D. thess, Dept. Computng and Electrcal Engneerng, Herot-Watt Unversty, August 1994. [34] R. Muñz and J.A. Corrales, Use of Band Ratonng for Color Texture Classfcaton, Frst Iberan Conference on Pattern Recognton and Image Analyss, 3, pp.66-615. [35] M.A. Akhlouf, W. Ben Larb and X. Maldague, Framework for color-texture classfcaton n machne vson nspecton of ndustral products, IEEE Internatonal Conference on Systems, Man, and Cybernetcs, 7, pp.167-171. Assocate Researcher wth the Percepton and Robotcs Laboratory of Ecole Polytechnque of Montreal n 1999. Then he oned Matrox Imagng, where he was n charge of Pattern recognton, OCR and LPR wthn the R&D group from tll 6. He s currently a Machne Vson Research Manager at Centre de Robotque & Vson Industrelle, Levs, Quebec. He s nvolved wth IEEE and IAPR organsatons. Xaver Maldague s a full professor and head of the Electrcal and Computng Engneerng Department of Unversty Laval, Quebec, Canada. He receved the M.Sc.A. and PhD from Unversty Laval n 1984 and 1989 respectvely. Hs research nterests are n nfrared thermography, nondestructve evaluaton (DE) technques and vson/dgtal systems for ndustral nspecton. He has authored or coauthored more than one hundred papers and several books on these topcs. He s also a Canada Char holder n the feld of nfrared vson. In 3 he was honoured wth the IEEE Canada Archambault Award for hs dedcaton to engneerng. Dr. Maldague s also actve on dfferent nternatonal commttees. He s also one of the drectors of Quebec s Provncal Chapter of the Canadan Socety for ondestructve Testng (CSDT) and vce-presdent of the Insttute of Electrcal and Electroncs Engneers (IEEE), Quebec Cty Secton. He s currently an Assocate Edtor of the Canadan Journal of Electrcal and Computer Engneerng. Before onng Unversty Laval n 1989, Dr. Maldague was wth the atonal Research Councl, Industral Materals Insttute, Bouchervlle, Quebec, Canada, workng on DE technques. Dr. Maldague s a senor member of IEEE. He s also a member of CARDE, CSDT, AST, and of Quebec's professonal engneers' assocaton (OIQ). Dr. Maldague receved the Canadan Governor General's Medal n 1978. Moulay A. Akhlouf receved the M.Sc.A. n Electrcal Engneerng from Ecole Polytechnque, Montreal, Canada, n 1999, and the MBA from Unversty Laval, Quebec, Canada, n 6. He s currently enrolled n the PhD n Electrcal Engneerng wth the Computer Vson Laboratory of Laval Unversty. Hs research nterests are n pattern recognton, bometrcs, ndustral nspecton and 3D vson. He was an Wael Ben Larb s enrolled n the M.Sc.A. n Electrcal Engneerng wth the Computer Vson Laboratory of Laval Unversty. Hs research nterests are n nfrared thermography ondestructve Testng and Evaluaton.