Robust Classfcaton of ph Levels on a Camera Phone B.Y. Loh, N.K. Vuong, S. Chan and C.. Lau AbstractIn ths paper, we present a new algorthm that automatcally classfes the ph level on a test strp usng color mage processng technques. hs algorthm s mplemented on a camera phone that captures color mages of ph test strps for healthcare or medcal purpose. Expermental results show that ths new approach s more robust n handlng reflecton and skewed placements of the test strps. Index ermsph test, color mage processng, moble applcaton. I. INRODUCION est strp Image captured Moble ph Applcaton Image Process Unt Result ph tests are wdely used to measure the acdc or alkalne level of substances. ph levels typcally range from 1 to 14 wth 7 beng neutral; 1 s hghly acdc and 14 s hghly alkalne. In a healthy person, the body fluds should nether be too alkalne nor acdc [1]. hus montorng the ph level of body fluds s a smple and effectve way to check for early ndcaton of varous dseases. ph tests can be easly carred out by dppng a ph test strp n the flud. he ph level s ndcated by the color change on the test strp. By vsually comparng the test strp s color aganst a color reference, the ph level of the flud under test s known. For example, orange s level 3, green s level 7 and blue s level 1. For ndvduals who fnd t dffcult or mpossble to compare the colors vsually, we propose a convenent soluton usng a camera phone - a devce that many people own and use n ther daly lfe nowadays. We have earler developed and presented a novel assstve technology system for measurng and classfyng ph levels from a dgtal camera phone mage [2, 3]. Fg. 1 shows the overall structure of the system. A moble phone camera s used to capture an mage of the ph test strp and the color reference chart. he mage processng unt (IPU) runnng on the moble phone dentfes the ph level by matchng the color of the ph test strp wth one of the colors on the reference chart. he result of the automatcally dentfed ph level s dsplayed on the phone. At the same tme, the ph result can be stored n the phone or transmtted to the doctors for further analyss f necessary. Manuscrpt receved August 31, 21; revsed January 7, 211. he authors are wth the School of Computer Engneerng, Nanyang echnologcal Unversty, Nanyang Avenue, Sngapore (emal: van.loh@pmal.ntu.edu.sg, {vuon2, asschan, asctlau} @ntu.edu.sg). Fg. 1. Structure of the proposed moble ph applcaton. he moble ph applcaton s able to assst ndvduals who montor ther body fluds for medcal and healthcare reasons. hs tool s partcularly helpful for color-blnd patents and elderly people wth poor eye sght. It s also benefcal to those wth normal eyesght but have dffcultes n dstngushng colors that are smlar. hs automated applcaton may also be helpful to medcal professonals such as nurses or doctors who have to conduct and nterpret numerous tests daly. hs applcaton has several advantages. Frstly, t does not ncur much cost snce t requres no specal hardware other than the camera phone tself. Moreover, t s easy to use wth no techncal sklls requred. hrdly, ths software soluton can be readly ntegrated nto a comprehensve moble healthcare system to provde holstc servce whch can be customzed for ndvdual patents. Most mportantly, ths software tool enables patents to montor ther health regularly n the comfort of ther home thus savng tme and money that would otherwse be ncurred for medcal consultatons and laboratory vsts. II. PREVIOUS WORKS he orgnal approach we employed to process mages of the ph test strps n a prevous study [2] was based on edge detecton flters. Sobel edge detecton was used to locate those pxels on the edges of the test strp and the color reference strps. Subsequently, smple scan-lne and thresholdng technques were used to locate the boundares of the test strp and the color targets. Eventually, the classfcaton was acheved by measurng color dstances between the strp color and color targets. he color regon that best matched the color of the test strp (.e. they had mnmum color dstance from each other) revealed the ph level of the test strp. Expermental results produced by the edge detecton based (EDB)
approach were consstent wth the ground truth estmaton [2]. However, the EDB could not process mages whose test strps were skewed more than 7 angular degrees to the horzontal axs. We then developed an mproved approach known as CQB [3] based on Wu s color quantzaton [4]. It comprsed two steps. Frst we quantzed the colors n the orgnal mage to two clusters n order to remove the background as well as some nose. Next, the colors n the resdual mage were quantzed to 11 clusters. he cluster wth the largest number of pxels was thus dentfed as that of the ph test strp. he computaton speed of ths approach was about 2 tmes faster than EDB. It was also able to handle test strps that were placed n skewed orentatons. However, ths method dd not work well f parts of the mage suffered from reflecton. Also, CQB dd not explctly dentfy the numercal ph level but reled on the user to vew the result from the resdual mage dsplayed. o overcome the above lmtatons, we propose a new approach that can robustly handle mages n whch the test strps are placed n a skewed manner or mages that are partally affected by reflecton. Frst, we use the approach smlar to EDB to obtan the boundares of the test strp and the color reference strps. Next, we use K-means clusterng to separate the edge ponts nto three clusters. Usng pror knowledge of the lengths of the color reference strps, the edge ponts are automatcally dentfed as three separate sets: one belongng to the test strp, one belongng to the longer color reference strp (ph1-6) and the last one belongng to the shorter color reference strp (ph7-11). hs step contrbutes a major mprovement over the prevous algorthm because ts functon s robust even when the strps are placed n a skewed manner. Next, the color reference strps are further parttoned nto 11 segments (ph1-11) usng the edge ponts obtaned earler. Wth pror knowledge of the colors n the color reference strps, a color lookup table s then used to recognze the color reference segments. hs s an mportant enhancement as t helps to recognze the color reference strps even when they are placed n arbtrary orders or orentatons. Moreover, by matchng aganst the color lookup table, even f one or two segments of the color reference strps are affected by reflecton or other llumnaton effects, the reference strps can stll be correctly dentfed and the color dstortons of the affected segments can be rectfed. hs greatly mproves the robustness of our algorthm. In the fnal step, the average color of the test strp s matched to one of the 11 color segments n the reference strp to automatcally dentfy the test strp s ph level. III. IMAGE PROCESSING UNI he mage captured by the phone camera wll go through the sequence of processes as shown n Fg. 2. A. Edge Detecton Fg. 2. Sequence of operatons n the IPU. he IPU begns wth Sobel Edge Detecton (SED) [5] on the mage captured. SED s generally used for grayscale mages and modfcaton to the algorthm s requred for t to be used on color mages. he modfed process depcted n Algorthm 1 s obtaned from J2ME- EDB approach [2]. Input: Color mage, h and weght w where w 1 for each color band Output: Bnary mage contanng edges for the nput mage 1. G 1 1 Gy1 1 x1 G x G x2 2 2, G y Gy2 G 1 1 x3 Gy3 1 2 2 1 1 Z1 z11 z12 z13 2. for each 3x3 mage sub-area Z 2 z21 z22 z23 of the Z 3 z31 z32 z33 nput color mage: 3. for each color band : 4. compute: 5. and 6. compute: f G x f G y f x1( Z1 ) Gx2( Z2 ) Gx3( Z3 ) y1( Z1 ) Gy2( Z2 ) Gy3( Z3 ) Edge Detecton K-means Clusterng Post K-means Clusterng Pont-n-Polygon est Converson to CIELab Determne Color Chart ph value Fnal Classfcaton of est Strp ph value f w x 2 f w y 7. f f h then mark the pxel z 22 as an edge pxel 8. else z 22 s a non-edge pxel 9. return bnary mage contanng only edge or non-edge pxels for the nput mage Algorthm 1. Pseudo-code for SED of color mages. 2 he output of SED produces an mage that outlnes all the edges. A further processng returns a set of XY coordnates whch dentfes all the edge pxels n the mage.
B. K-Means Clusterng K-means Clusterng s a cluster analyss method that separates a set of ponts nto k clusters or regons where each regon has a centrod. An nteger k representng the numbers of centrods s frst chosen. Next, the dstance of a pont to a centrod s calculated. Each pont has k dstances to k centrods. A regon conssts of all the ponts that are nearest to the cluster s centrod. At the end of ths frst teraton, all the ponts n a regon are used to calculate the new poston of the centrod. he process s repeated by computng the dstance of each pont to the new k centrods. he teraton stops when there s no change to the ponts n each regon as llustrated n Fg 3. Next, the ph 1-6 reference strp s dvded nto 6 regons and the ph 7-11 reference strp s dvded nto 5 regons to segregate each ph level nto ts ndvdual regon. Each ph regon s bounded by ts 4 corners, and ther coordnates are stored for use n future steps. he 4 corner coordnates of the test strp are also stored. Start Choose number of cluster k Determne Centrods Dstance of each pont to centrods Put ponts nto dfferent clusters Yes Change n cluster s members No End Fg. 5. Bounded regons. D. Pont-n-Polygon est and Converson to CIELab Edge Detecton Images from most dgtal cameras use RGB encodng and a 3x3 lnear color transformaton s performed to map all the pxels n RGB color space P to CIELab reference values M usng equaton (1). Fg. 3. K-means clusterng algorthm. In our applcaton, k = 3 regons; namely est strp, Color reference ph 1-6, Color reference ph 7-11. Usng k-means clusterng, we are able to determne the boundng areas of each strp n 3 teratons as shown n Fg 4. Centrods - ndcated by the red dots Fg. 4. Complete clusterng n 3 teratons. C. Post K- Means Clusterng est strp Color ref ph 1-6 Color ref ph 7-11 After separaton of all ponts nto 3 regons, the 4 corners of each rectangular strp are determned by computng the dstance of each edge pont to the 4 corners of the mage. he edge pont nearest to the respectve mage corner s labeled as a corner and the bounded area of the strp s then marked out as shown n Fg. 5. he strp wth the longest length s the test strp, followed by the ph 1-6 reference strp and the shortest length s the ph 7-11 reference strp. M * A L 11 M * a A21 M * b A31 A12 A22 A32 A13 Pred A23 Pgreen A33 Pblue hs step s performed because the dfference between any two colors n Lab format can be approxmated by treatng each color as a pont n a three dmensonal space (wth three components: L, a, b) and takng the Eucldean dstance between them [6]. Each ph regon conssts of pxels whch are bounded by the 4 corners whose coordnates have earler been determned. A pont-n-polygon check s performed to determne f the pxel s wthn the ntended regon. he applcaton converts all pxels wthn each regon nto Lab color space. Pxels that fall outsde the regons are dscarded. he entre procedure s performed for all the 12 regons and n addton, a two-pass Lab converson procedure avalable n our prevous verson of the applcaton [2] s used to further elmnate nose and the text n the test strp and color reference strps. E. Determnaton of Color Chart ph Value It mght be noted that the poston of the 3 strps can be placed n any order. However, t s requred that the applcaton has knowledge of the regon type and the orentaton of the color reference chart. hs allows correct dentfcaton of the ph value of the respectve regon after the test strp has been matched to one of the ph regons. Each ph regon s matched to a pre-defned average Lab color as shown n able I. he average Lab values for (1)
the 11 regons are computed based on mages obtaned from 3 dfferent moble phones, each wth 7 samples taken. ABLE I AVERAGE LAB VALUES ph Regon L a b 1 63.625 46.125 15.25 2 75.125 42.875 41.5 3 91.25 25.25 78.375 4 99.625 1.75 74. 5 95.375-5.75 79.875 6 9.875-15. 75.375 7 83.125-2.375 58.875 8 72.375-15.375 25.25 9 61.5-13.375 5.625 1 57-3.875-3.125 11 54.75 2.25-13.5 llumnaton problem, ths process provdes a more accurate Lab values for the fnal ph matchng and classfcaton. F. Fnal ph Classfcaton Wth the enhanced Lab values of the 11 ph color chart regons and the knowledge of each regon ph value and ts coordnates, the fnal step s to compare the test strp wth the 11 ph regons. he RMS of the test strp Lab values are computed aganst the 11 ph regons Lab values. he regon wth the smallest RMS ndcates that t s the best match to the test strp and s consdered as the fnal ph classfcaton. Fg. 6 shows the result wth Lab values n yellow and the RMS value n green. he classfcaton ndcates that the test strp has ph level of 3. he procedure begns by computng the root mean square (RMS) of ph 1 regon n the frst color reference chart (ph1-6 strp) as compared to the pre-defned average color values of ph1-6. hs procedure s repeated for the other fve regons n the frst color reference chart. Snce ph1 and ph6 have dverse color values, we are able to determne whch extreme end of the frst color chart belongs to ph1 or ph6 based on the RMS result. Wth ths nformaton, the system wll also know whch regon belongs to ph2, ph3, ph4 and ph5. he same procedure s repeated for the second color reference chart (ph7-11) as llustrated n Algorthm 2. Input: ph1-6 color ref chart coordnates (R1 where =1 to 6), ph7-11 color ref chart coordnates (R2 where =7 to 11), Orgnal Lab Matrx Vector (L where =1 to 11), Color Lookup able Matrx Vector (C j where j=1 to 11) Output: Sorted Lab Matrx Vector 1. for each Lab Matrx Vector, L where =1 to 6 2. compute: rmscol1[][j] = RMS of L to C j where j=1 to 6 3. set mn1 = rmscol1[1][1] and mnindex1=1 4. f mn of rmscol[][1] < mn1 then mn1 = rmscol[][1], mnindex1=; =1 to 6 5. set mn2 = rmscol1[1][1] and mnindex2=1 6. f mn of rmscol[][1] < mn2 then mn2 = rmscol[][1], mnindex2=; =1 to 6 7. f mnindex2=6 or (mnindex1 >= 4 AND mnindex2 <= 3) then swop the Lab Matrx for L and color reference chart coordnates R1 where =1 to 6. 8. end for 9. repeat 1 to 6 where =7 to 11 1. f mnindex2=11 or (mnindex1 >= 9 AND mnindex2 <= 9) then swop the Lab Matrx for L and color reference chart coordnates R2 where =7 to 11. 11. end for 12. return Sorted Lab Matrx Vector L Algorthm 2. Pseudo-code for Computaton of ph Regon Value wth Color Lookup able. In addton to determnng the ph value of each regon n the color reference chart, the 11 average Lab values n the color table are used to resolve wrong ph value classfcaton when a part of the captured mage has suffered reflecton. If the ndvdual ph regon Lab value dffers by more than 2% from the correspondng Lab value n the average color table, the ndvdual ph regon Lab value s replaced wth that n the average color table. In cases when the mages are affected by reflecton or Fg. 6. Fnal classfcaton of ph=3. IV. RESULS AND DISCUSSION A seres of experments were conducted to evaluate the effectveness of the applcaton. Eght test solutons were ndvdually tested wth 5 dfferent ph test strps [7] and 5 respondents were tasked to vsually match the test strp to the ph value on the color reference chart. Each respondent vewed a dfferent test strp tested on each of the 8 solutons and ther responses are collated n able II. he same ph strps vewed by the respondents were captured by the phone camera and classfed usng the applcaton. Each soluton has 5 sample mages for evaluaton. For evaluaton of our ph classfcaton applcaton, we compare the results aganst those obtaned by the respondents n able II. he ph classfcaton results acheved by our applcaton are consstent wth those ndcated by the respondents.
ABLE II EXPERIMENAL RESULS Soluton Index 1 2 3 4 5 6 7 8 Respondent 1 7 3 3 4 7 9 1 11 Respondent 2 7 3 3 4 7 9 11 11 Respondent 3 7 3 4 4 7 9 1 11 Respondent 4 7 3 4 3 6 9 11 11 Respondent 5 7 4 4 4 6 9 11 11 Applcaton Classfcaton 7 3 3 4 7 9 11 11 Images wth reflecton and slanted strp placements as shown n Fg. 7 have been tested and the results ndcate that the ph values of the test strps are correctly classfed by our applcaton. [4] X. Wu, Color Quantzaton by Dynamc Programmng and Prncpal Analyss, ACM ransactons on Graphcs, vol. 11(4), pp. 348-372, 1992 [5] I.E. Sobel, Camera models and machne percepton, Ph.D. dssertaton, Stanford Unversty, Stanford, Calf, USA, 197 [6] M. kalcc, and J.F. asc, Color spaces perceptual, hstorcal and applcatonal background, EUROCON 23 [7] Johnson ph est Paper, http://www.kaagat.com/ Fg. 7. Images wth reflecton (left) and slanted strp placements. V. CONCLUSION AND FUURE WORKS hs paper has presented a robust soluton to the problem of moble ph classfcaton. We have overcome the major lmtatons n two earler approaches. hs new algorthm s able to handle test strps or color reference strps that are placed n a skewed manner or arbtrary order. It can also automatcally dentfy the correct ph level even when a part of the mage s affected by reflecton. hs s one major step closer to accomplshng an effcent, robust, low cost, accurate, and ntellgent moble ph reader that s of great use to the elderly or color-blnd people. In our future work, we ntend to focus on the ncluson of dfferent types of ph test strps whch can present new challenges due to ther dfferent shapes, colors and precsons. REFERENCES [1] J.A. Smervlle, W.C. Maxted, and J.J. Pahra, Urnalyss: a Comprehensve Revew, Amercan Famly Physcan, vol. 71(6), pp. 1153-1162, 25 [2] N.K. Vuong, S. Chan, C.. Lau, Classfcaton of ph Levels Usng a Camera Phone, he 13 th IEEE Internatonal Symposum on Consumer Electroncs, 29 [3] N.K. Vuong, S. Chan, C.. Lau, ph Levels Classfcaton by Color Quantzaton on a Camera Phone, Internatonal Conference on Communcatons and Moble Computng, 21