Detection and Classification of Apple Fruit Diseases using Complete Local Binary Patterns

Transcription

1 2012 Third Iteratioal Coferece o Computer ad Commuicatio Techology Detectio ad Classificatio of Apple Fruit Diseases usig Complete Local Biary Patters Shiv Ram Dubey Departmet of Computer Egieerig ad Applicatios GLA Uiversity Mathura, Idia shivram1987@gmail.com Aad Sigh Jalal Departmet of Computer Egieerig ad Applicatios GLA Uiversity Mathura, Idia asjalal@gla.ac.i Abstract Diseases i fruit cause devastatig problem i ecoomic losses ad productio i agricultural idustry worldwide. I this paper, a solutio for the detectio ad classificatio of apple fruit diseases is proposed ad experimetally validated. The image processig based proposed approach is composed of the followig mai steps; i the first step K-Meas clusterig techique is used for the image segmetatio, i the secod step some state of the art features are extracted from the segmeted image, ad fially images are classified ito oe of the classes by usig a Multi-class Support Vector Machie. Our experimetal results express that the proposed solutio ca sigificatly support accurate detectio ad automatic classificatio of apple fruit diseases. The classificatio accuracy for the proposed solutio is achieved up to 93%. Keywords-K-Meas Clusterig; Local Biary Patter; Multiclass Support Vector Machie; Texture Classificatio; I. INTRODUCTION The classical approach for detectio ad idetificatio of fruit diseases is based o the aked eye observatio by the experts. I some developig coutries, cosultig experts are expesive ad time cosumig due to the distat locatios of their availability. Automatic detectio of fruit diseases is essetial to automatically detect the symptoms of diseases as early as they appear o the growig fruits. Apple fruit diseases ca cause major losses i yield ad quality appeared i harvestig. To kow what cotrol factors to take ext year to avoid losses, it is crucial to recogize what is beig observed. Some disease also ifects other areas of the tree causig diseases of twigs, leaves, ad braches. Some commo diseases of apple fruits are apple scab, apple rot, ad apple blotch [1], as show i Fig. 1. Apple scabs are gray or brow corky spots. Apple rot ifectios produce slightly suke, circular brow or black spots that may be covered by a red halo. Apple blotch is a fugal disease ad appears o the surface of the fruit as dark, irregular or lobed edges. Visual ispectio of apples is already automated i the idustry by machie visio with respect to size ad color. However, detectio of defects is still problematic due to atural variability of ski color i differet types of apple, high variace of defect types, ad presece of stem/calyx. Majority of the works performig defect segmetatio of apples are doe usig simple threshold approach ([2], [3]). A globally adaptive threshold method (modified versio of Otsu s algorithm) to segmet fecal cotamiatio defects o apples are preseted i [4]. Classificatio-based techiques attempt to partitio pixels ito several classes usig differet classificatio methods. Bayesia classificatio is the most used method by researchers ([5], [6]), where pixels are compared with a pre-calculated model ad classified as defected or healthy. Usupervised classificatio does ot beefit ay guidace i the learig process due to lack of target values. Such a approach was used by [7] for defect segmetatio. I [8], Ojala et al used the Local Biary Patter histogram for rotatio ivariat texture classificatio. Local Biary Patter is a simple yet very efficiet operator to defie local image patter, ad it has reported impressive classificatio outcomes o represetative texture databases [9]. Local Biary Patter has also bee adapted by other applicatios, such as face recogitio [10] dyamic texture recogitio [11] ad shape localizatio [12]. A Complete Local Biary Patter is preseted i [13] as the completed modelig of Local Biary Patter. We have proposed ad experimetally validated the sigificace of usig clusterig techique for the disease segmetatio ad Multi-class Support Vector Machie as a classifier for the automatic detectio ad classificatio of fruit diseases. I order to validate the proposed approach, we have cosidered three types of the diseases i apple; apple blotch, apple rot ad apple scab. (a) (b) (c) Figure 1. Three commo apple fruit diseases: (a) apple scab, (b) apple rot, ad (c) apple blotch. II. THE PROPOSED APPROACH The steps of the proposed approach are show i the Fig. 2. For the fruit disease classificatio problem, precise image segmetatio is required; otherwise the features of the oifected regio will domiate over the features of the ifected regio. I this approach K-Meas based image segmetatio is preferred to detect the regio of iterest which is the ifected part oly. After segmetatio, features are extracted from the segmeted image of the fruit. Fially, traiig ad classificatio are performed o a Multi-class SVM classifier /12 $ IEEE DOI /ICCCT

2 Data Set Preparatio Image Segmetatio Feature Extractio Traiig ad Testig by a Classifier processig time for the image segmetatio. I this experimet iput image are partitioed ito four segmets. From the empirical observatios it is foud that usig 3 or 4 clusters yields good segmetatio result. Fig. 4 demostrates the output of K-Meas clusterig for a apple fruit ifected with apple scab disease. Fig. 5 also depicts some more image segmetatio results usig the K-Meas clusterig techique. Iput Image objects i cluster 1 Figure 2. The basic procedure of the proposed approach. The framework of the proposed solutio is show i the Fig. 3. Each phase of the proposed method is described i the rest of this sectio. Traiig Apple Fruit Images Testig Apple Fruit Image (a) (b) objects i cluster 2 objects i cluster 3 Image Segmetatio Image Segmetatio Feature Extractio Feature Extractio Traiig by Multi-class SVM Classificatio by Multi-class SVM (c) (d) Recogized Disease of Apple objects i cluster 4 image labeled by cluster idex Figure 3. Framework of the proposed approach. A. Image Segmetatio K-Meas clusterig techique is used for the image segmetatio. Images are partitioed ito four clusters i which oe cluster cotais the majority of the diseased part of the image. K-Meas clusterig algorithm [14] was developed by J. MacQuee (1967). The k-meas clusterig algorithms classify the objects (pixels i our problem) ito K umber of classes based o a set of features. The classificatio is carried out by miimizig the sum of squares of distaces betwee the data objects ad the correspodig cluster. I this experimet, squared Euclidea distace is used for the K- meas clusterig. Algorithm for the K-Meas image segmetatio Step 1. Read iput image. Step 2. Trasform image from RGB to L*a*b* color space. Step 3. Classify colors usig K-Meas clusterig i 'a*b*' space. Step 4. Label each pixel i the image from the results of K- Meas. Step 5. Geerate images that segmet the image by color. Step 6. Select the segmet cotaiig disease. We have used L*a*b* color space because the color iformatio i the L*a*b* color space is stored i oly two chaels (i.e. a* ad b* compoets), ad it causes reduced (e) (f) Figure 4. K-Meas clusterig for a apple fruit that is ifected with apple scab disease (a) The ifected fruit image, (b) first cluster, (c) secod cluster, (d) third cluster, ad (e) fourth cluster, respectively, ad (f) sigle gray-scale image colored based o their cluster idex. (a) (b) Figure 5. Image segmetatio results (a) Images before segmetatio, (b) Images after segmetatio. 347

3 B. Feature Extractio I the proposed approach, we have used some state of the art color ad texture features to validate the accuracy ad efficiecy. The features used for the apple fruit disease classificatio problem are Global Color Histogram, Color Coherece Vector, Local Biary Patter, ad Complete Local Biary Patter. 1) Global Colour Histogram (GCH) The Global Color Histogram (GCH) is the simplest approach to ecode the iformatio preset i a image [15]. A GCH is a set of ordered values, for each distict color, represetig the probability of a pixel beig of that color. Uiform ormalizatio ad quatizatio are used to avoid scalig bias ad to reduce the umber of distict colors [15]. 2) Color Coherece Vector (CCV) A approach to compare images based o color coherece vectors are preseted i [16]. They defie color coherece as the degree to which image pixels of that color are members of a large regio with homogeeous color. These regios are referred as coheret regios. Coheret pixels are belogs to some sizable cotiguous regio, whereas icoheret pixels are ot. I order to compute the CCVs, the method blurs ad discretizes the image s color-space to elimiate small variatios betwee eighborig pixels. The, it fids the coected compoets i the image i order to classify the pixels i a give color bucket is either coheret or icoheret. After classifyig the image pixels, CCV computes two color histograms: oe for coheret pixels ad aother for icoheret pixels. The two histograms are stored as a sigle histogram. 3) Local Biary Patter (LBP) Give a pixel i the iput image, LBP [8] is computed by comparig it with its eighbors: = 1 1, x 0 LBPN, R s( v vc )2, s( x) = = 0 0, x < 0 Where, v c is the value of the cetral pixel, v is the value of its eighbors, R is the radius of the eighborhood ad N is the total umber of eighbors. Suppose the coordiate of v c is (0, 0), the the coordiates of v are (Rcos(2π / N), Rsi(2π / N)). The values of eighbors that are ot preset i the image grids may be estimated by iterpolatio. Let the size of image is I*J. After the LBP code of each pixel is computed, a histogram is created to represet the texture image: H ( k) I J = i= 1 j= 1 f ( LBP N, R ( i, j), k), k [0, K], 1, x = y f ( x, y) = 0, otherwise Where, K is the maximal LBP code value. I this experimet the value of N ad R are set to 8 ad 1 respectively to compute the LBP feature. 4) Complete Local Biary Patter (CLBP) LBP feature cosiders oly sigs of local differeces (i.e. differece of each pixel with its eighbors) whereas CLBP feature [13] cosiders both sigs (S) ad magitude (M) of local differeces as well as origial ceter gray level (C) value. CLBP feature is the combiatio of three features, amely CLBP_S, CLBP_M, ad CLBP_C. CLBP_S is the same as the origial LBP ad used to code the sig iformatio of local differeces. CLBP_M is used to code the magitude iformatio of local differeces: 1 = 1, x c CLBP N, R = t( m, c)2, t( x, c) = 0 0, x < c Where, c is a threshold ad set to the mea value of the iput image i this experimet. CLBP_C is used to code the iformatio of origial ceter gray level value: 1, x c CLBPN, R = t( gc, ci ), t( x, c) = 0, x < c Where, threshold c I is set to the average gray level of the iput image. I this experimet the value of N ad R are set to 8 ad 1 respectively to compute the CLBP feature. C. Traiig ad Classificatio Recetly, a uified approach is preseted i [17] that ca combie may features ad classifiers. The author approaches the multi-class classificatio problem as a set of biary classificatio problem i such a way oe ca assemble together diverse features ad classifier approaches customtailored to parts of the problem. They defie a class biarizatio as a mappig of a multi-class problem oto twoclass problems (divide-ad-coquer) ad referred biary classifier as a base learer. For N-class problem N (N-1)/2 biary classifiers will be eeded, where N is the umber of differet classes. Accordig to the author, the ij th biary classifier uses the patters of class i as positive ad the patters of class j as egative. They calculate the miimum distace of the geerated vector (biary outcomes) to the biary patter (ID) represetig each class, i order to fid the fial outcome. Test case will belog to that class for which the distace betwee ID of that class ad biary outcomes will be miimum. Their approach ca be uderstood by a simple three class problem. Let three classes are x, y, ad z. Three biary classifiers cosistig of two classes each (i.e., x y, x z, ad y z) will be used as base learers, ad each biary classifier will be traied with traiig images. Each class will receive a uique ID as show i Table 1. To populate the table is straightforward. First, we perform the biary compariso x y ad tag the class x with the outcome +1, the class y with 1 ad set the remaiig etries i that colum to 0. Thereafter, we repeat the procedure comparig x z, tag the class x with +1, the class z with 1, ad the remaiig etries i that colum with 0. I the last, we repeat this procedure for biary classifier y z, ad tag the class y with +1, the class z with -1, ad set the remaiig etries with 0 i that colum, where the etry 0 meas a Do t care value. Fially, each row represets uique ID of that class (e.g., y = [ 1, +1, 0]). 348

4 () TABLE I UNIQUE ID OF EACH CLASS x y x z y z x y z Each biary classifier will result a biary respose for ay iput example. Let s say if the outcomes for the biary classifier x y, x z, ad y z are +1, -1, ad +1 respectively the the iput example will belogs to that class which have the miimum distace from the vector [+1, -1, +1]. So the fial aswer will be give by the miimum distace of mi dist([+1, -1, +1],{[+1, +1, 0],[-1, 0, +1],[0, -1, -1]}). This paper uses Multi-class Support Vector Machie (MSVM) as a set of biary Support Vector Machies (SVMs) for the traiig ad classificatio. (a) (b) III. EXPERIMENTAL RESULT A. Data Set Preparatio To demostrate the performace of the proposed approach, we have used a data set of ormal ad diseased apple fruits, which comprises four differet categories: (104), Apple rot (107), Apple scab (), ad Normal Apple (120): totalizig 431 apple fruit images. Fig. 6 depicts the classes of the data set. Presece of a lot of variatios i the type ad color makes the data set more realistic. (d) Figure 6. Sample images from the data set of type (a) apple scab, (b) apple rot, (c) apple blotch, ad (d) ormal apple. (c) GCH_RGB CCV_RGB LBP_RGB CLBP_RGB Traiig Examples per Class (a) Usig RGB colour image. Accurcy (%) GCH_HSV CCV_HSV LBP_HSV CLBP_HSV (b) Usig HSV colour image. Figure 7. for the GCH, CCV, LBP, ad CLBP features derived from RGB ad HSV colour images cosiderig MSVM classifier. GCH_RGB GCH_HSV CCV_RGB CCV_HSV LBP_RGB LBP_HSV CLBP_RGB CLBP_HSV (a) GCH ad CCV feature. (b) LBP ad CLBP feature. Figure 8. Compariso of the accuracy achieved i RGB ad HSV colour space for the GCH, CCV, LBP, ad CLBP features cosiderig MSVM classifier. 349

5 B. Result Discussio I the quest for fidig the best categorizatio procedure ad feature to apple disease classificatio, this paper aalyzes some color ad texture based image descriptors derived from RGB ad HSV stored images cosiderig MSVM classifier. If we use N images per class for traiig the remaiig images are used for testig. The accuracy of the proposed approach is defied as, Total umber of images correctly classified Accuracy(%) = * Total umber of images used for testig Fig. 7 (a-b) shows the results for differet features i the RGB ad HSV color spaces respectively. The x-axis represets the umber of images per class i the traiig set ad the y-axis represets the accuracy for the test images. This experimet shows that GCH does ot perform well ad reported accuracy is lowest for it i both the color spaces. Oe possible explaatio is that, GCH feature have oly color iformatio, it does ot cosiders eighborig iformatio. GCH uses simply frequecy of each color; however CCV uses frequecy of each color i coheret ad icoheret regios separately so CCV performs better tha GCH i both color spaces. From the Fig. 7 (a-b), it is clear that LBP ad CLBP features yield better result tha GCH ad CCV features because both LBP ad CLBP uses the eighborig iformatio of each pixel i the image. Both LBP ad CLBP are robust to illumiatio differeces ad they are more efficiet i patter matchig because they use local differeces which are computatioally more efficiet. For istace, i HSV color space with traiig examples per class, the reported classificatio accuracy is.94% for GCH, 86.47% for CCV,.97% for LBP, ad 93.14% for CLBP feature. The LBP feature uses oly the sig iformatio of the local differeces, istead, LBP reasoably represet the image local features because sig compoet preserves the major iformatio of local differeces. The CLBP feature exhibits more accurate result tha LBP feature because CLBP feature uses both sig ad magitude compoet of local differeces with origial ceter pixel value (i.e. CLBP cosiders additioal discrimiat iformatio). We also observe across the plots that each feature performs better i the HSV color space tha the RGB color space as show i the Fig. 8 (a-b). For 45 traiig examples ad CLBP feature, for istace, reported classificatio error is 88.74% i RGB ad 92.65% i HSV. Oe importat aspect whe dealig with apple fruit disease classificatio is the accuracy per class. This iformatio poits out the classes that eed more attetio whe solvig the cofusios. Fig. 9 depicts the accuracy for each oe of 4 classes usig LBP ad CLBP features i RGB ad HSV color spaces. Clearly, is oe class that eeds attetio i both color spaces. It yields the lowest accuracy whe compared to other classes. Fig. 9 also shows that, the behavior of is early same i each sceario. Normal Apples are very easily distiguishable with diseased apples ad a very good classificatio result is achieved for the Normal Apples i both color spaces. For CLBP feature ad HSV color space, for istace, reported classificatio accuracy are 89.88%,.71%, 96.66%, ad 99.33% for the,,, ad Normal Apple respectively, resultig average accuracy 93.14% whe traiig is doe with images per class (a) LBP i RGB color space. (b) LBP i HSV color space (c) CLBP i RGB color space. (d) CLBP i HSV color space. Figure 9. Accuracy per class for the LBP ad CLBP features i RGB ad HSV color spaces usig MSVM as a classifier. 3

6 IV. CONCLUSIONS A image processig based solutio is proposed ad evaluated i this paper for the detectio ad classificatio of apple fruit diseases. The proposed approach is composed of maily three steps. I the first step image segmetatio is performed usig K-Meas clusterig techique. I the secod step features are extracted. I the third step traiig ad classificatio are performed o a Multiclass SVM. We have used three types of apple diseases amely:,, ad as a case study ad evaluated our program. Our experimetal results idicate that the proposed solutio ca sigificatly support automatic detectio ad classificatio of apple fruit diseases. Based o our experimets, we have foud that ormal apples are easily distiguishable with the diseased apples ad CLBP feature shows more accurate result for the classificatio of apple fruit diseases ad achieved more tha 93% classificatio accuracy. Further work icludes cosideratio of fusio of more tha oe feature to improve the output of the proposed method. REFERENCES [1] J. Hartma, Apple Fruit Diseases Appearig at Harvest, Plat Pathology Fact Sheet, College of Agriculture, Uiversity of Ketucky, FShtml/PPFS-FR-T-2.pdf, viewed o December [2] Q. Li, M. Wag, ad W. Gu, Computer visio based system for apple surface defect detectio, Computers ad Electroics i Agriculture, vol. 36, pp , Nov [3] P. M. Mehl, K. Chao, M. Kim, ad Y. R. Che, Detectio of defects o selected apple cultivars usig hyperspectral ad multispectral image aalysis, Applied Egieerig i Agriculture, vol. 18, pp , [4] M. S. Kim, A. M. Lefcourt, Y. R. Che, ad Y. Tao, Automated detectio of fecal cotamiatio of apples based o multispectral fluorescece image fusio, Joural of food egieerig, vol. 71, pp , [5] O. Kleye, V. Leemas, ad M. F. Destai, Developmet of a multi-spectral visio system for the detectio of defects o apples, Joural of Food Egieerig, vol. 69, pp , [6] V. Leemas, H. Magei, ad M. F. Destai, Defect segmetatio o joagold apples usig colour visio ad a bayesia classificatio method, Computers ad Electroics i Agriculture, vol. 23, pp , Jue [7] V. Leemas, H. Magei, ad M. F. Destai, Defect segmetatio o golde delicious apples by usig colour machie visio, Computers ad Electroics i Agriculture, vol. 20, pp , July [8] T. Ojala, M. Pietikäie, ad T. T. Mäepää, Multiresolutio gray-scale ad rotatio ivariat texture classificatio with Local Biary Patter, IEEE Tras. o Patter Aalysis ad Machie Itelligece, vol. 24, o. 7, pp , [9] T. Ojala, T. Mäepää, M. Pietikäie, J. Viertola, J. Kyllöe, ad S. Huovie, Outex ew framework for empirical evaluatio of texture aalysis algorithm, i Proc. Iteratioal Coferece o Patter Recogitio, 2002, pp [10] T. Ahoe, A. Hadid, ad M. Pietikäie, Face recogitio with Local Biary Patters: applicatio to face recogitio, IEEE Tras. o Patter Aalysis ad Machie Itelligece, vol. 28, o. 12, pp , [11] G. Zhao, ad M. Pietikäie, Dyamic texture recogitio usig Local Biary Patters with a applicatio to facial expressios, IEEE Tras. O Patter Aalysis ad Machie Itelligece, vol. 27, o. 6, pp , [12] X. Huag, S. Z. Li, ad Y. Wag, Shape localizatio based o statistical method usig exteded local biary patter, i Proc. Iteratioal Coferece o Image ad Graphics, 2004, pp [13] Z. Guo, L. Zhag, ad D. Zhag, A completed modelig of local biary patter operator for texture classificatio, IEEE Tras. O Image Processig, vol. 19, o. 6, pp , [14] J. MacQuee, Some methods for classificatio ad aalysis of multivariate observatios, I L. M. LeCam ad J. Neyma, editors, Proceedigs of the Fifth Berkeley Symposium o Mathematical Statistics ad Probability, 1967, vol. 1, pp , Uiversity of Califoria Press. [15] R. Gozalez, R. Woods, Digital Image Processig, 3rd ed., Pretice-Hall, [16] G. Pass, R. Zabih ad J. Miller, Comparig images usig color coherece vectors, I ACM Multimedia, 1997, pp [17] A. Rocha, C. Hauagge, J. Waier, ad D. Siome, Automatic fruit ad vegetable classificatio from images, Computers ad Electroics i Agriculture, Elsevier; vol., pp ,