A New Automatic Method to Adjust Parameters for Object Recognition

Transcription

1 A New Automatc Method to Adjust Parameters for Object Recognton Issam Qaffou Département Informatque, FSSM Unversté Cad Ayyad, Mohamed Sadgal Département Informatque, FSSM Unversté Cad Ayyad Azz Elfazzk Département Informatque, FSSM Unversté Cad Ayyad Abstract To recognze an object n an mage, the user must apply a combnaton of operators, where each operator has a set of parameters. These parameters must be well adjusted n order to reach good results. Usually, ths adjustment s made manually by the user. In ths paper we propose a new method to automate the process of parameter adjustment for an object recognton task. Our method s based on renforcement learnng, we use two types of agents: User Agent that gves the necessary nformaton and Parameter Agent that adjusts the parameters of each operator. Due to the nature of renforcement learnng the results do not depend only on the system characterstcs but also the user s favorte choces. Keywords- component; Parameters adjustment; mage segmentaton; Q-learnng; renforcement learnng. I. INTRODUCTION New tools and new algorthms for vson applcatons cause new system parameters that must be properly adjusted. Ths adjustment requres a specfc knowledge, takes a long tme, and sometmes even has to be done n an expermental process. To accomplsh a segmentaton task, the user must apply some operators, where each one has a set of parameter to adjust. The lack of a general rule that gudes the user n hs choces, the fxaton of parameter values s usually made ntutvely. The user proceeds by tryng manually all possble cases untl fndng the desred result. Usually, n the majorty of vson tasks we need to apply a combnaton of several operators where each one has a multtude of parameters to adjust. So, the manual adjustment becomes very tedous and not trustworthy. Therefore, an automatc method to adjust the values of each parameter s needed. The qualty of results depends essentally on the operator chosen and the values assgned to ts parameters. Some GUI, for example Arane [1], help users to accomplsh a vson task by proposng them an nteractve nterface, but the values assgned to the parameters are selected manually by the user. Very few systems had succeeded to automate the process of parameter adjustment. In [2], B.NICKOLAY et al. proposed a method to automatcally optmze the parameters of a machne vson system for surface nspecton by usng specfc Evolutonary Algorthms (EA). A few years later, Taylor proposed a renforcement learnng framework whch uses connectonst systems as functon approxmators to handle the problem of determnng the optmal parameters for a computer vson applcaton even n the case of a hghly dmensonal, contnuous parameter space [3]. More recently, Farhang et al. [9] ntroduced a new method for the segmentaton of the prostate n transrectal ultrasound mages, usng a renforcement learnng (RL) scheme. He dvded the ntal mage nto sub-mages and works on each one n order to reach a good result. In ths paper we propose a new method to adjust automatcally the parameters of vson operators. Our method s based on renforcement learnng. We use two agents: User Agent (UA) and Parameter Agent (PA). The UA gves the necessary nformaton to the system. It gves the combnaton of applcable operators, the set of adjustable parameters for each operator, values ranges for each parameter. The PA uses renforcement learnng to assgn the optmal values for each parameter n order to extract the object of nterest from an mage. Due to the nature of RL, n terms of the nteracton between state, acton and reward, our approach takes n account not only the system opportuntes but also the user preferences, and through the learnng mechansm t wll suggest trustworthy solutons. An overvew of renforcement learnng s gven n secton 2. Secton 3 outlnes the proposed approach and ntroduces a general framework for parameter adjustment. Secton 4 presents the expermental results, and secton 5 concludes the paper. II. REINFORCEMENT LEARNING Renforcement learnng (RL) s learnng what to do, how to map stuatons to actons, so as to maxmze a numercal reward sgnal. The learner s not told whch actons to take, as n most forms of machne learnng, but nstead must dscover whch actons yeld the best reward by tryng them. One of the challenges that arse n renforcement learnng and not n other knds of learnng s the tradeoff between exploraton and explotaton. To obtan a lot of reward, a renforcement learnng agent must prefer actons that t has tred n the past and found to be effectve n producng reward. But to dscover such actons t has to try those that t has not selected before. Renforcement learnng uses a formal framework defnng the nteracton between agent and ts envronment n terms of states, actons, and rewards, Fg 1. Reward or punshment s determned from the envronment, dependng on the acton taken. The agent must 213 P a g e

2 fnd a trade-off between mmedate and long-term returns. It must explore the unseen states, as well as the states whch maxmze the return by choosng what the agent already knows. Therefore, a balance between the exploraton of unseen states and the explotaton of famlar (rewardng) states s crucal. Watkns has developed Q-learnng, a wellestablshed on-lne learnng algorthm, as a practcal RL method [6]. In ths algorthm, the agent mantans a numercal value for each state-acton, representng a predcton of the worthness of takng an acton n a state. III. THE PROPOSED APPROACH Generally, to accomplsh an object recognton task, the user must apply sequentally some operators, and for each operator there s some parameter to adjust. Because there s no general rule that gudes the user n hs choces, he s based usually on hs ntuton to select values for each parameter. In the majorty of vson tasks, we have to apply a multtude of operators that have several parameters to adjust. So adjustng manually these parameters basng only on the experence and on the ntuton s not evdent. It s a tedous work wth a huge wasted tme. In ths paper we propose a new automatc method to fnd the best values for each parameter n a recognton task. In our method we use two types of agents: User Agent (UA) and Parameter Agent (PA). Fg 2 shows the general framework of our method. Fgure 1: A general model for Renforcement learnng agent Table 1 represents an teratve polcy evaluaton for updatng the state-acton values where r s the reward value receved for takng acton n states, s' s the next state, α s the learnng rate, and γ s the dscount factor [7]. There are some polces for takng acton a gven states. One of them s the Boltzman polcy whch estmates the probablty of takng each acton n each state. There are other polces for Q- learnng such as ε- greedy and greedy. In the greedy polcy, all actons may not be explored, whereas the ε-greedy selects the acton wth the hghest Q-value n the gven state wth a probablty of 1 ε, and other ones wth a probablty of ε [7,8]. In ths work an ε-greedy polcy s used to make a balance between exploraton and explotaton. The reward r(s, a) s defned accordng to each state-acton par (s, a). The goal s to fnd a polcy to maxmze the dscounted sum of rewards receved over tme. The prncpal concerns n RL are the cases where the optmal solutons cannot be found, but can be approxmated. The onlne nature of RL dstngushes t from other technques that approxmately solve Markov decson processes (MDP) [5,7]. TABLE 1. Q-LEARNING ALGORITHM Intalze arbtrary Repeat (for each epsode) Intalze state s Repeat (for each step of epsode) Choose acton from state s usng polcy derved from (e.g., Take acton, observe reward r, next state s ; Untl s s termnal In ths paper, we attempt to ntroduce the RL concept for parameters adjustment. Fgure 2: General framework for the proposed approach. The UA gves to the PA the needed nformaton: the combnaton of operators to apply, the set of parameters for each operator and the values ranges for each parameter. The PA receves ths nformaton and proceeds automatcally to fnd the best values for each parameter. Fg. 3 shows the general functonng of the PA. The agent PA nteracts wth ts envronment by actons, states. A set of mages contanng the object of nterest s gven to PA. Each mage has ts ground-truth, the object extracted by an expert. An mage wth ts ground truth s ntroduced to the system. A combnaton of operators to extract an object of nterest s proposed. Each operator has some parameters that have to be well adjusted. Each value gven to a parameter gves a dfferent result. The agent PA must fnd the optmal values that gve the best result. It proceeds then by tral and error untl fndng the best parameter values. For that t uses renforcement learnng. Actons, states and a reward functon must then be defned. 214 P a g e

3 Fgure 3: the general process of PA usng renforcement learnng. A. A. Defnng actons Generally, all possble combnaton of parameters values s defned as an acton for the RL agent. The set of the actons s then the set of all possble values combnaton, see fg. 3. Each operator OP k has a seres of parameters: Each parameter k j ( P, P,..., P ) k k k 1 2 n P has a range of values: V { V, V,..., V } k k k k j j1 j2 jm An elementary acton of the operator OP k s: k k k k ak ( u j1,..., u jr ) where uj1 Vj An acton of the agent PA s defned by the combnatons of the elementary actons of operators as t s defned above: a a1 a2 a n (,,..., ) Actons for object recognton task are gven n the experence. B. Defnng states A state s defned by a set of features extracted from the resultng mage: s 1, 2,..., n s a feature reflectng the state of the mage after the processng. The type of the extracted features depends on the task at hand. Here we gve a general defnton, and n the experence we defne them explctly for a recognton task. C. Defnng the reward The return s a reward f the agent chooses the rght acton, else t s a punshment. The reward s defned accordng to the qualty of the processng result. Ths qualty s assessed by usng ground-truth models. To defne the return we calculate the smlarty between the resultng mage and ts ground truth. The smlarty s calculated accordng to some features extracted from the two mages. The type of these features depends on the task at hand. For example, f we want to detect an object n an mage we extract the number of the objects, ther areas, ther szes, etc. We express the dfference between these scalars by: D wd The weghts w are chosen accordng to the mportance of each feature. A general form of the reward defnton n the proposed approach s presented by: Reward: r= -10, 0 or 10; f (D < ) r = +10; f=true; elsef ( (D > ) && (D < + ) ) r = 0; else r = -10; end end The values 10 and -10 represent respectvely the reward and the punshment dependng to a predefned threshold. IV. EXPERIENCE We use a dataset of 30 textured mages contanng the same object to extract. The object s a textured dsc njected n all the 30 mages. The used mages are textured so the UA proposes a combnaton of two operators: GLCM (Gray Level Cooccurrence Matrces) to segment textures and k-means to classfy them. Each one of these operators has some parameters to be adjusted n order to be executable. In GLCM, texture s always defned n relaton to some local wndow. The sze n x n of ths wndow affects the result of the segmentaton, so we propose the sze of the wndow as the parameter to adjust for GLCM. UA proposes a range of values for n, t may have seven values, the odd values between 9 and 21 {9, 11,, 21}. GLCM extracts fourteen texture features [8]. In ths paper we lmt our self to four of the most popular features: Angular Second Moment (energy), Contrast, Correlaton and entropy. After extractng these textures, we classfy them usng the algorthm k-means. The parameter to adjust for ths operator s k, the number of the possble clusters. It can take fve values {1,,5}. How are actons, states and reward are defned accordng to our experence s gven below. A. Actons Defnton The UA proposes two operators: GLCM and k-means. GLCM has n the sze of the sldng wndow as the parameter to adjust. n can take seven odd values: 9, 11, 13,, 21, so an elementary acton for GLCM s one of these values. k-means has k the number of possble cluster, ts possble values are {1,2,3,4,5}. An elementary acton for k-means s one of these values. Then an acton for the agent PA s consttuted by a couple of a value of n and a value of k. All actons are all possble combnatons of the values of n and k. B. States Defnton States are defned, accordng to the features whch represent the status of the resultng mage. For object recognton we extract four features to defne the state space: 215 P a g e

4 Where s the selected feature. s the Number of Objects n the resultng mage after segmentaton. s the rato between the area of the extracted object and the area of the whole mage. s the rato between the area of the resultng object and the object reference. s the mean of the used textural features: Angular Second Moment (energy), Contrast, Correlaton and entropy. C. Reward Defnton The rewards and punshments are defned accordng to the qualty crteron that represents how well the mage s segmented. A straghtforward method s comparng the resultng mage wth ts ground truth. Ths comparson s made between the scalar features of the obtaned regons and those of the desred one. In ths paper we defne the reward accordng to a dfference between the components of the mage. We defne ths dfference as: D = weghted sum of the four followng dfferences n the two mages (the resultng mage and ts ground truth): D1= dfference of the number of the objects; D2= dfference of the szes of the objects; D3= dfference of the surfaces of the objects; D4= dfference of the feature textures. Fg. 4 shows the three mages taken randomly from the process of recognton. The three mages contan the same object of nterest, the textured dsc. Fgure 4: the three mages contanng the dsc to extract. The agent PA proceeds by renforcement learnng and fnds that the optmal acton that gves the best result s (. So the best value for the sze of the sldng wndow s 13 wth the best number of possble clusters s 3. Fg. 5 shows the reference dsc and the resultng one by our approach. The ground truth Our approach result Fgure 5: the resultng mage and ts reference. Fg. 6 shows the curve of learnng of the agent PA. At frst t has not much knowledge and experence to behave, so t uses several steps per epsode. Over tme the curve learnng becomes almost constant, whch proves that really there s a learnng whle the processng s done, the number of steps decreases wth epsodes. It means that our agent RL accumulates an experence that wll help hm to take decson n the future. Fgure 6: Learnng that makes our RL agent durng ts processng V. CONCLUSION Determnng the values of parameters of the vson operators s a challengng task. In ths paper, we have proposed a renforcement learnng approach to handle ths problem even n the case of a vson task needng many operators to sequence. A texture segmentaton applcaton s presented to test our approach. Our goal sn t comparng our method to others, but our goal s to present another manner of thnkng that uses learnng concepts and show that really t gves good results. Our method can be appled to any decson process usng parametrc methods. Due to the nature of renforcement learnng, the proposed approach takes n account not only the system opportuntes but also the user preferences, and through the learnng mechansm t suggests trustworthy solutons. As perspectves, our approach wll be used on a large set of dfferent mages and ts results wll be compared to other methods. REFERENCES [1] R. Clouard, A. Elmoataz & F.Angot, PANDORE : une bblothèque et un envronnement de programmaton d opérateurs de tratement d mages, Rapport nterne du GREYC, Caen, France, Mars [2] [3] B. Nckolay, B. Schneder, S.Jacob, Parameter Optmzaton of an Image Processng System usng Evolutonary Algorthms CAIP 1997: [4] G. W.Taylor, A Renforcement Learnng Framework for Parameter Control n Computer Vson Applcatons Proceedngs of the Frst Canadan Conference on Computer and Robot Vson (CRV 04), IEEE [5] Sutton RS, Barto AG: Renforcement Learnng Cambrdge, MA: MIT Press; [6] Watkns CJCH, Dayan P: Q-Learnng. Machne Learnng 1992, 8: P a g e

5 [7] Russell SJ, Norvg P: Artfcal ntellgence: a modern approach Englewood Clffs, N J: Prentce Hall; [8] Sngh S, Norvng P, Cohn D: Introducton to Renforcement Learnng Harlequn Inc; [9] R. M. Haralck, K. Shanmugan, I. Dnsten. Textural Features for Image Classfcaton. IEEE Transactons on Systems, Man and Cybernetcs, Vol. 3, 1973, No 6, [10] F. Sahba, H. R. Tzhoosh, M. Salama. Applcaton of renforcement learnng for segmentaton of transrectal ultrasound mages BMC Medcal Imagng P a g e