Scale Selectve Extended Local Bnary Pattern For Texture Classfcaton Yutng Hu, Zhlng Long, and Ghassan AlRegb Multmeda & Sensors Lab (MSL) Georga Insttute of Technology 03/09/017
Outlne Texture Representaton and Its Challenge Proposed Local Descrptor, SSELBP Expermental Results Concluson
Texture Defnton Defnton of texture [1] : The feel or shape of a surface or substance such as smoothness, roughness, and softness Texture s everywhere. [1]: https://en.wktonary.org/wk/texture 3
Why are Textures Important? Classfcaton/Retreval [1] Segmentaton [] Synthess [3] Transfer [4] [1] http://www.robots.ox.ac.uk/~vgg/research/texclass/ [] https://www.vs.un-stuttgart.de/nc/lehre/detals/typ/vorlesung/1767/98.html [3] http://cs.brown.edu/courses/c9/results/proj4/kgao/ [4] https://graphcs.stanford.edu/~mdfsher/texturesynthess.html 4
Ppelne for Texture Classfcaton Alumnum _fol Tranng Local Feature Extracton Image Encodng Lnen Testng Classfcaton Alumnum_fol Or Lnen Unknown texture Local Feature Extracton Image Encodng 5
Texture Representaton Local feature descrptors Handcrafted local descrptors Gray Level Co-occurrng Matrx (GLCM) Markov Random Feld (MRF) Flter Banks Scale-nvarant Feature Transform (SIFT) Speed-up Robust Features (SURF) Local Bnary Pattern (LBP) Orentated FAST and Rotated BRIEF (ORB) CNN local descrptors 6
Challenges n Texture Representaton Illumnaton, rotaton, and scale varatons Illumnaton Rotaton Scale [1]: Y. Xu, H. J, and C. Ferm uller, Vewpont nvarant texture descrpton usng fractal analyss, Internatonal Journal of Computer Vson, vol. 83, no. 1, pp. 85 100, 009. 7
Outlne Texture Representaton and Its Challenge Proposed Local Descrptor, SSELBP Expermental Results Concluson 8
The Framework of SSELBP Scale Space Generaton Mult-scale ELBP Feature Extracton ELBP( P1, R1) Jont Hstogram / NI P1, R1 ELBP ELBP( P, R ) ELBP _ NI ELBP _ RD P, R P, R Image I Normalzaton s 1 ELBP Feature Extracton ELBP( P, R) ELBP( P3, R3) Jont Hstogram Jont Hstogram / NI P, R / NI P3, R3 Hstogram Concatenaton 1 ( P, R ) Gaussan Flterng s Gaussan Flterng s 3 ELBP( P, R ) N N Jont Hstogram Mult-scale ELBP Feature Extracton Mult-scale ELBP Feature Extracton PN, RN s 1 ( P, R ) s3 1 ( P, R ) Gaussan Flterng s 4 Mult-scale ELBP Feature Extracton s4 1 ( P, R ) Maxmum Poolng SSELBP Feature 9
Scale Space Generaton Input Image Normalzaton D Gaussan D Gaussan D Gaussan Flter ( ) Flter ( ) Flter ( ) Scale Space s s s s 1 3 4 10
The Framework of SSELBP Scale Space Generaton Mult-scale ELBP Feature Extracton ELBP( P1, R1) Jont Hstogram / NI P1, R1 ELBP ELBP( P, R ) ELBP _ NI ELBP _ RD P, R P, R Image I Normalzaton s 1 ELBP Feature Extracton ELBP( P, R) ELBP( P3, R3) Jont Hstogram Jont Hstogram / NI P, R / NI P3, R3 Hstogram Concatenaton 1 ( P, R ) Gaussan Flterng s Gaussan Flterng s 3 ELBP( P, R ) N N Jont Hstogram Mult-scale ELBP Feature Extracton Mult-scale ELBP Feature Extracton PN, RN s 1 ( P, R ) s3 1 ( P, R ) Gaussan Flterng s 4 Mult-scale ELBP Feature Extracton s4 1 ( P, R ) Maxmum Poolng SSELBP Feature 11
Extended Local Bnary Pattern (ELBP) Feature Extracton Global Sgn Pattern Example: P = 8 Neghborng Intensty Pattern Radal Dfference Pattern Rotaton-nvarant and unform- ( ) Illumnaton and Rotaton Invarance [1]: L. Lu, L. Zhao, Y. Long, G. Kuang, and P. Feguth, Extended local bnary patterns for texture classfcaton, Image and Vson Computng, vol. 30, no., pp. 86 99, 01. 1
The Framework of SSELBP Scale Space Generaton Mult-scale ELBP Feature Extracton ELBP( P1, R1) Jont Hstogram / NI P1, R1 ELBP ELBP( P, R ) ELBP _ NI ELBP _ RD P, R P, R Image I Normalzaton s 1 ELBP Feature Extracton ELBP( P, R) ELBP( P3, R3) Jont Hstogram Jont Hstogram / NI P, R / NI P3, R3 Hstogram Concatenaton 1 ( P, R ) Gaussan Flterng s Gaussan Flterng s 3 ELBP( P, R ) N N Jont Hstogram Mult-scale ELBP Feature Extracton Mult-scale ELBP Feature Extracton PN, RN s 1 ( P, R ) s3 1 ( P, R ) Gaussan Flterng s 4 Mult-scale ELBP Feature Extracton s4 1 ( P, R ) Maxmum Poolng SSELBP Feature 13
Sngle-scale- and Mult-scale- ELBP Hstogram Jont Hstogram ELBP ELBP( P, R ) ELBP _ NI ELBP _ RD P, R P, R 1 0 1 0 ELBP 0 1 P 1 ELBP _ RD P R, Frequency / NI P1, R1 Frequency ELBP _ NI P R, Concatenated Hstogram P 1 H s 1 ELBP PR 1, 1 Frequency P, R [0 0 0] Frequency 0 1 ( P )( P ) 1 1 ( P, R ) ELBP( P, R ) 0 1 ( P )( P ) 1 0 1 ( P )( P ) 1 0 1 ( P )( P ) 1 4( P )( P ) 1 14
The Framework of SSELBP Scale Space Generaton Mult-scale ELBP Feature Extracton ELBP( P1, R1) Jont Hstogram / NI P1, R1 ELBP ELBP( P, R ) ELBP _ NI ELBP _ RD P, R P, R Image I Normalzaton s 1 ELBP Feature Extracton ELBP( P, R) ELBP( P3, R3) Jont Hstogram Jont Hstogram / NI P, R / NI P3, R3 Hstogram Concatenaton 1 ( P, R ) Gaussan Flterng s Gaussan Flterng s 3 ELBP( P, R ) N N Jont Hstogram Mult-scale ELBP Feature Extracton Mult-scale ELBP Feature Extracton PN, RN s 1 ( P, R ) s3 1 ( P, R ) Gaussan Flterng s 4 Mult-scale ELBP Feature Extracton s4 1 ( P, R ) Maxmum Poolng SSELBP Feature [1]: Z. Guo, X. Wang, J. Zhou, and J. You, Robust texture mage representaton by scale selectve local bnary patterns, Image Processng, IEEE Transactons on, vol. 5, no., pp. 687 699, 016. 15
Maxmum Poolng Frequency 1 ( P, R ) Frequency 0 1 ( P )( P ) 1 N( P )( P ) 1 s 1 ( P, R ) Frequency 1 ( P, R ) 0 1 ( P )( P ) 1 N( P )( P ) 1 0 1 ( P )( P ) 1 N( P )( P ) 1 16
Ppelne for Texture Classfcaton Alumnum _fol Tranng Local Feature Extracton Image Encodng Lnen Unknown texture Local Feature Extracton Testng Image Encodng Classfcaton Alumnum_fol Or Lnen 17
Nearest Neghbor Classfer (NNC) Ch-square dstance between hstogram T and M : T n and M n are the values of T and M at the n-th bn Nearest neghbor classfer (NNC): The class label of a test mage s determned by the tranng mage that has the mnmal ch-square dstance to the test mage. 18
Outlne Texture Representaton and Its Challenge Proposed Local Descrptor, SSELBP Expermental Results Concluson 19
Test Databases: KTHTIPS [1]: E. Hayman, B. Caputo, M. Frtz, and J. Eklundh, On the sgnfcance of real-world condtons for materal classfcaton, n European Conference on Computer Vson (ECCV), 004, pp. 53 66. 0
Expermental Results Table 1: Classfcaton accuracy (%) of the proposed SSELBP usng dfferent samplng schemes on the KTH-TIPS database. Number of Radus, N Maxmum Accuracy (%) Radus Selecton for Maxmum Mean Accuracy (%) Standard Dervaton Feature Dmenson 1 96.44 () 94.80 1.56 00 97.86 (1,6) 97.04 0.63 400 3 98.09 (, 5, 8) 97.51 0.43 600 4 98.11 (, 3, 4, 7) 97.71 0.30 800 5 98.10 (1,, 3, 4, 8) 97.84 0.0 1000 Table : Classfcaton accuracy (%) of the proposed SSELBP and typcal texture descrptors on the KTH-TIPS and UMD databases. The number n the bracket followng databases denotes the number of tranng samples used per class. Classfcaton Accuracy KTH-TIPS (40) UMD (0) CLBP (Guo et al.) 97.19 98.00 RP (Lu et al.) 97.71 99.13 MRELBP (Lu et al.) - 98.66 SSLBP (Guo et al.) 97.80 98.84 SSELBP (Proposed) 98.11 98.96 1
Outlne Texture Representaton and Its Challenge Proposed Local Descrptor, SSELBP Expermental Results Concluson
Concluson To characterze texture mages wth scale varatons, we extracted local scale varant mult-scale ELBP features and then appled a global transformaton. The maxmum poolng strategy of mult-scale ELBP hstograms generated from a scale space selected domnant scales and addressed scale varaton ssues for texture mages. SSELBP acheved hgh accuracy comparable to typcal texture descrptors on gray-scale-, rotaton-, and scale-nvarant texture classfcaton but uses only one thrd of the feature dmenson of CLBP or SSLBP. 3