Study of System Model of Image Inpainting Combining Subjective Estimation and Impersonal Estimation

Transcription

1 Study of System Model of Image Inpantng Combnng Subjectve Estmaton and Impersonal Estmaton Welan Wang, Huamng Lu Abstract Amng at the present condton of the fled of dgtal mage npantng lack a all-purpose or a general algorthm to modfy varous damaged mage, n order to npant more types of damaged mages n dfferent domans, a new system model of mage npantng s proposed n ths paper Then performance npantng, the system model wll gve some nformaton, whch conssts of the satsfacton of subjectve and mpersonal estmaton of the npanted results, the used algorthm and so on Thus, user can select the better-suted algorthm to be npanted mage by referrng ths nformaton At the same tme, the system model has the functon of self-learnng Index Terms Image npantng, subjectve estmaton and mpersonalty estmaton, system model of mage npantng, self-learnng I INTRODUCTION Dgtal mage npantng s an mportant ssue n the doman of mage restoraton and an nternatonal nterestng research topc n recent years A lot of papers on mage npantng were publshed Although the presented algorthms to modfy damaged regons of mage had ther merts, they only had some specfc ams and at present there sn t a unversal algorthm model of guarantee to cure all dseases and t s not an easy matter The chef reason for the problem s that the segmentaton of mage s stll a dffcult ssue due to the segmentaton algorthms are some partcular queston- orented at present, and the damaged regons of mage must be segmented before npanted Segmentaton wll frst affect the results of npantng Therefore, t s more dffcult to get a unversal algorthm to solve all of the problems We attempt to desgn a Thangka mage npantng system whch can npant dfferent knds of damaged Thangka mage, whch s a more powerful mage npantng system than the sngle algorthm s system on the bass of prevous work [6][7][8] Ths system, n whch ntegrated varous npantng algorthms, can npant varous knds of Manuscrpt receved December 6, 008 Ths work was supported n part by The Fund of State Ethnc Affars Commsson under Grant, The Scentfc Research Foundaton for the Returned Overseas Chnese Scholars, Mnstry of Personnel of People s Republc of Chna, The Nature Scence Foundaton of Chna under Grant No Welan Wang was wth School of Computer Scence and Informaton Engneerng, Northwest Unversty for Natonaltes, Lanzhou, Gansu Prvnce, Chna; e-mal:wangwelan@xbmueducn) Huamng Lu was wth School of Computer and Informaton, Fuyang Normal College, Anhu Provnce, 3609 Chna; e-mal: luhuamng888@ 6com damaged mage, after npanted, the results of evaluaton are stored n database for dfferent algorthms, the results nclude subjectve estmaton and mpersonal estmaton At the some tme the classfed results are also recoded to database about subjectve classfcaton and mpersonal classfcaton to type of damaged regon of mage Thus the most sutable algorthm to each category of damaged regon can be found The system model also has the functon of self-learnng n the practcal npantng process, t s convenent to provde decson support for users whle system modfyng an mage II QUALITY EVALUATION OF IMAGE INPAINTING Generally speakng, the npanted mage s not every bt as good as the orgnal, thus the dfferences n the sze wll be evdence to apprase the qualty of mage npantng There are two methods of evaluatng the qualty of mage npanted: subjectve estmaton and mpersonal estmaton Where mpersonal estmaton conssts of comparatve analyss method of Peak Sgnal to Nose Rato (PSNR)and color dfference A Subjectve estmaton We can say that the effectveness of an npantng algorthm s satsfactory depend on whether the npanted mage s very natural and wthout the tral of npanted n subjectvty estmaton of human eyes B Impersonal estmaton () PSNR method The value of PSNR ( Peak Sgnal to Nose Rato) s defned as follows: 55 PSNR=0log MSE (-) M N ^ A(, A(, = 0 j= 0 MSE = (-) M N Where MSE denote the Mean Square Error of pxel value of two mages correspondng the pxel ponts; M N ndcates the sze of mage, namely numbers of pxel pont; A (, j ) and ^ A (, j ) are pxel value of pont (, of the orgnal regon and the npanted regon If the value of PSNR s more, the loss of mage s less, and namely the mage wll be better npanted by the formula (-) and (-) ISBN: IMECS 009

2 () The method of chromatc aberraton The dsplay use RGB color space n dsplay of dgtal mage, and color mode s CIELan space of color dfference, so color space need to converson n order to calculaton the chromatc aberraton The formula as follows (-3) and (-4): X = 0564 R G B (-3) Y = 0963R + 069G B Z = R + 046G B L = 6( Y Y0 ) 6 (-4) a = 500[( X / X0) ( Y Y0 ) ] Y / Y0 > 00 b = 00[( Y Y0 ) ( Z / Z0) ] Where X,Y and Z s trstmulus values; X 0 Y0 Z0 s trstmulus values of CIE, here, use lght sources of D65 ( X 0, Y0, Z0 equal 9505, 00, 089 respectvely) L denotes psychologcal lghtness, a and b denotes psychologcal chromnance respectvely The computaton formula of chromatc aberraton for color space of unformty of CIE Lab s (-5): ΔE = ( ΔL) + ( Δα ) + ( Δb) (-5) where Δ L denote brghtness dfference can be compute by the formula (-6) Δ L = L L (-6) Δ a and Δb denote chromatc aberraton whch can be computed by the (-7) Δa = a a (-7) Δa = b b L > L llumnate that color s shallow than color, brghtness hgher; L < L represent that color s fuscous than color, brghtness lower a > a show that color s slantng red than color ; a < a show that color s slantng green than color b > b denote color slantng yellow than color ; b < b ndcates color s slantng blue than color When the npanted mage are apprased n our research, Δ E ndcate the mean value chromatc aberraton of full pxels whch are the damaged regon and after npanted regon, the sze of Δ E wll show the effect of npantng The calculaton of ΔE s (-9) The sze of ΔE wll be employed to estmate the effect of npantng, Δ E (, and Δ E (, s all of pxels chromatc aberraton n front and behnd of npantng for the damaged regon of the mage Smaller the value Δ E s, more effectve the npantng s, as shown n formula (-9) M N ( ΔE(, ΔE(, ) = 0 j= 0 (-9) Δ E = M N damaged mage, and take damaged Thangka mage as examples The crteron of classfcaton vares wth the ndvdual, and classfcaton of damaged mage n other felds could dffer from each other, such as remote sensng mage, medcal mage, etc these do not mpact the proposed system model of npantng A The category of damaged Thangka mage Damaged Thangka mage can be classfed nto 4 classfcatons () Crease-Thangka Fg Crease-Thangka Fg Rp-Thangka The crease, such as the whte regon n Fg, usually t was worked due to nadequate safekeepng () Rp-Thangka Damaged regon s caused by natural envronment, that color break off from the damaged part, and the damage regon s a very thn gap, the color of mage s degraded as shown n Fg (3) The types of the damaged patch The damaged mage as Fg 3 show, there are 3 damaged patches, each of whch has characterstc tself The damaged mages can be classfed nto 3 types: smooth patch, edge patch and texture patch (3 whte parts from the above, below to rght sde are smooth patch, texture patch and edge patch respectvely n Fg 3), by the nformaton surround those patches Smooth patch, the characterstcs of such patch s: sngle color, gray vares slow, varety of gray range s small; Edge patch, the characterstcs are, the gray vares fast, gray range s large, and devaton value s large, etc ; Texture patch, gray dstrbuton of texture regon commonly have certan perodcty, and there are some approxmately same structure n texture area any place Some tme, damaged regons are comparatve complex snce they could belong to sngle-type or mult-type Therefore the classfcaton of these types s also comparatve dffculty, but we can adopt npantng algorthm n sngle damaged regon accordng to ts type III SYSTEM MODEL OF INPAINTING In order to understand the system model of npantng proposed n ths paper, we frst llumnate the sort of ISBN: IMECS 009

3 Fg 3 Smooth patch, texture patch and edge patch (4)Spot-Thangka There are large numbers of damaged regons, dfferent shape and sze, and area s not especally bg as shown n Fg 4 Fg 4 Spot-Thangka (5)The key regon be lose The damaged part s a key regon that s exclusve n the mage, ths brng huge dffculty for mage npantng, the full regon of mouth mssng as shown n Fg 5 Fg 5 The key regon be lose We only gve 5 categores of damaged Thangka mage as examples, researcher can process classfcaton by practcal cases There s stll some damaged mage that damaged cases are comparatve complexty, ts npant to need semantc knowledge; they are dffcult problem n the doman of mage npantng B Inpantng algorthms At present manly two classfyng-technques can be found n the lterature related to dgtal mage npantng The frst one deals wth the small-scale defcent of dgtal mage npantng technques The technque was ntroduced nto mage processng by Marcelo Bertalmo, Gullermo Sapro,Vcent Caselles and Coloma Bellester [], usng margnal nformaton of damaged regon, and essentally ths s an npantng algorthm base on partal dfferental equaton (PDE) Another s mage npantng based on varatonal method of the geometrc mage models The second one s mage completon based on texture synthess fll-n the large damaged regon Ths technque ncludes two methods: one s to decompose mage nto structure part and texture part, where use npantng algorthm to npant structure part, and use the method of texture synthess fll-n texture part [][3]; another s texture synthess technque basng on patch, whch selects a pxel n the border of the to be npanted regon, take the pont as center, select a proper texture patch n sze by texture characterstc of the mage, then look for the most smlar texture matchng patches around to be panted regon to replace the patch[4][5] C System model of mage npantng In vew of present stuaton of mage npantng, we attempt to solve the drawback of all knds of algorthms, to npant more classes of damaged mage and generate hgh qualty results, thus novel system model of mage npantng are proposed () The structure system model of mage npantng Take Thangka mage of damaged as an llustraton, damaged mage are classfed 5 classfcatons by user s knowledge as 3 secton descrbed Frst, for the nput mage, we need to confrm what category of object ths damaged mage s, parameter v denote category of damaged mage n the system of subjectve and mpersonal estmaton ntegrated The detectng algorthm of damaged regon wll be utlzed, the statstcal nformaton of proporton of damaged regons for texture patches, edge patches and smooth patches are ndcated by v, v 3 and v 4 4 respectvely, and v =, v [0,], =,3,4 = The system wll modfy the damaged regons after the damaged regon of Thangka mage are segmented The methods of npantng can be one or more algorthms, for the proposed model, ntegrated algorthms the more the better, ultmately the power of npantng dfferent damaged mage for each algorthm wll be statstcal calculated The system structure model of mage npantng shown n Fg 6 After npantng the damaged regons, the system adopt the method of subjectve estmaton and mpersonal estmaton, fnally receve two parameters v 7 and v 8, whch present the npanted effect, where v [0,], = 7, 8 and 0 denote worst, denote best v 6 s used storage the seral number of one or more algorthms by order of the used ISBN: IMECS 009

4 algorthms The varety of v 7 and v 8 wll dependent the varety of v 6 () The tranng of system model Frst,the system need to tran use damaged mage, fnally the statstc results of dfferent algorthms to be used to dfferent type damaged regons and varety npanted mages are stored database, when user use the system to npant the damaged mages agan, these statstc nformaton are fed back to user or researcher, so that supply decson-makng For example, suppose that a damaged mage wll be npanted If a seres of algorthms s used, a good effect s ganed, we suppose that the seres of algorthms s -3-5; f another seres of algorthms that s s used, then a results seres wll generate More combnaton of seres of algorthms may develop So t s a questons that s worth consderng that how to select a seres of algorthms Where, a seres of algorthms may be regarded as a route of algorthm (3) Select the seres of algorthms Sometmes the result of npantng s satsfactory when usng only one algorthm In these cases we do not need to select the seres of algorthms We can compare the result of npantng and take subjectve estmaton and mpersonal estmaton for each algorthm, the results of the estmaton wll be recorded n the character database of results of npantng n detal If the result of npantng s not satsfactory when usng sngle algorthm, here the seres of algorthms should be a better choce The choce of the seres of algorthms s very mportant, because t wll mmedately mpact the result of npantng Therefore t s necessary to have a great detaled analyss for the seres of algorthms n order to use less seres of algorthms (the number of the seres of algorthms s few) and less number of the seres of algorthms (the number of algorthm route) The seres of algorthms wll be adopted when modfy a damaged mage by usng dfferent algorthms, and at the same tme subjectve estmaton and mpersonal estmaton are obtaned, and f two ntegrated estmaton reach the requested degree; Otherwse, the algorthms wll be abandoned, the system wll return to the prevous npantng results, and select another algorthm The detaled content of the algorthm s as follows: Step Open an mage to be npanted, subjectvely judge ts label of classfcaton, smultanety calculate the values of v, v 3 and v 4 of damaged regons of texture patch, edge patch and smooth patch, and the acreage v 5 of damaged regons Step Select one algorthm and analyze ts degree of excellence by the hnt of system Here, suppose that algorthm s selected, and then employ the algorthm to npant the damaged mage Step 3 Do subjectve estmaton and mpersonal estmaton to the result of npantng n Step, and make the decson that whether or not use the prevous algorthm n Step : f the result of npantng s not satsfactory, then gve away the algorthm and return to prevous state of npanted, perform Step agan and select another algorthm to contnue executon; f the result of npantng s acceptable, then reserve the algorthm and select another algorthm to contnue executon Step 4 Repeat the Step and the Step 3, untl get a satsfactory result of npantng, and then a seres of algorthms are created, store the seres to v 6, ultmately put v, =,,, 8 n character database Step 5 Repeat the Step, Step, Step 3 and Step 4, ultmately create another seres of algorthms Repeat the step, several seres of algorthm wll be created Step 6 Compare the several seres of algorthms, fnd the optmum route of algorthm (4) Self-learnng of system model The performance of system model s ndependent tran samples of damaged mage Wth the mage to be npanted abundance, system s the hnt nformaton of results of npantng for dfferent damage mage wll be more abundant, more perfect and valuable nformaton wll be provded by the system Therefore, the system has self-learnng functon and wll have ntellgent nformaton to hnt for users Such systems are few for the moment, because ths s a practcal requrement of complex damaged mage npantng, however In addton, user can update or translates the characterstc database of results of npantng of present system to a new characterstc of tself Classfcaton of damaged mage standard must s consstent n ths case IV CONCLUSION We notce that the categores of damaged regon to be npanted are less when use sngle algorthm to npant the damaged Thangka mage n our research of damaged Thangka mage We want to fnd an all-purpose or wth generalty algorthm to modfy varous damaged mage, but the study found that t has many dffcultes Then amng at the present condton of the fled of dgtal mage npantng lackng general algorthm to modfy varous damaged mage, n order to make more types and more damaged mage n dfferent domans can be npanted better, a new system model of mage npantng s proposed The system model not only npant damaged Thangka mage n dfferent categores, but also can npant damaged mage of dfferent classfcaton n other domans Varety npantng algorthms are ntegrated n the system that ft for dfferent the damaged mage The system mplementng needs to research further, wth the further study, more parameters probably need to be added so that supply the better decson-makng for npantng of damaged mage REFERENCES [] Bertalmo M, Sapro G, Caselles V, et al Image npantng In Proceedngs of Internatonal Conference on Computer Graphcs and Interactve Technques,, New Orleans, Lousana, USA, 000pp47-44 ISBN: IMECS 009

5 [] Bertalmo M, Vese L, Sapro G, et al Smultaneous structure and texture mage npantng IEEE Transactons on Image Processng, 003, (8)pp [3] Efros A A, Leung T K Texture synthess by non-parametrc samplng In: Proceedngs of the IEEE Computer Socety Internatonal Conference on Computer Vson, Washngton DC, USA, 999, pp [4] Crmns A, Perez P, Toyama K Regon fllng and object removal by exemp larbased mage npantng IEEE Transactons on Image Processng, 004, 3 (9)pp 00- [5] Crmns A, Perez P, Toyama K Object removal by exemplar-based npantng In: Proceedngs of IEEE Computer Socety Conference on ComputerVson and Pattern Recognton,, Monona Terrace Conventon Center Madson, Wsconsn, USA, 003, pp8-0 [6] Huamng Lu, Welan Wang and Hu Xe Thangka Image Inpantng Usng Adjacent Informaton of Broken Area Internatonal MultConference of Engneers and Computer Scentsts 008 Hong Kong, March, 008 pp [7] Welan Wang, Huamng Lu Study on Thangka Image Inpantng Based on Damaged Area Classfcaton Internatonal Conference on Informaton Technology and Envronmental System Scences (ITESS 008) Jao Zuo, Henan, Chna May 008 pp [8] Wang WeLan, Tang ShX Tbetan TangKa Image Inpantng n Intrcate Dsrepared Regon Computer Engneerng & Desgn (Chanse) 0073 pp Damage mage User classfy v Informaton of proporton of texture patches, edge patches and smooth patches: V,V3,V4; areage of damage regon V5 Inpantng Algorthm Inpantng Algorthm Inpantng Algorthm V6 Results of npantng Subjectve and mpersonal Estmaton V7,V8 feature database of npantng results:v,v,v3, v4, v5,v6,v7 feedback Inpantng Algorthm n Fg 6 System model of mage npantng wth subjectve estmaton and mpersonal estmaton ntegrated ISBN: IMECS 009