Reduced complexity Retinex algorithm via the variational approach q

Transcription

1 J. Vs. Commun. Image R. 14 (2003) Reduced complexty Retnex algortm va te varatonal approac q M. Elad, a, * R. Kmmel, b D. Saked, c and R. Keset c a Computer Scence Department, Stanford Unversty, Stanford, CA 94305, USA b Computer Scence Department, Tecnon Israel Insttute of Tecnology, Hafa 32000, Israel c Hewlett-Packard Laboratores, Israel Receved 10 December 2002; accepted 6 June 2003 Abstract Retnex teory addresses te problem of separatng te llumnaton from te reflectance n a gven mage, and tereby compensatng for non-unform lgtng. In a prevous paper (Kmmel et al., 2003), a varatonal model for te Retnex problem was ntroduced. Ts model was sown to unfy prevous metods, leadng to a new llumnaton estmaton algortm. Te man drawback wt te above approac s ts numercal mplementaton. Te computatonal complexty of te llumnaton reconstructon algortm s relatvely g, snce n te obtaned Quadratc Programmng (QP) problem, te wole mage s te unknown. In addton, te process requrements for obtanng te optmal soluton are not cosen a pror based on ardware/ software constrants. In ts paper we propose a way to compromse between te full fledged soluton of te teoretcal model, and a varety of effcent yet lmted computatonal metods for wc we develop optmal solutons. For computatonal metods parameterzed lnearly by a small set of free parameters, t s sown tat a reduced sze QP problem s obtaned wt a unque soluton. Several specal cases of ts general soluton are presented and analyzed a Look-Up-Table (LUT), lnear or nonlnear Volterra flters, and expanson usng a truncated set of bass functons. Te proposed solutons are sub-optmal compared to te orgnal Retnex algortm, yet ter numercal mplementatons are muc more effcent. Results ndcate tat te proposed metodology can enance mages for a reduced computatonal effort. Ó 2003 Elsever Inc. All rgts reserved. Keywords Retnex; Illumnaton; Quadratc programmng; Look-Up-Table; Volterra flters; Gamma correcton q Ts researc was carred out n Hewlett-Packard laboratores, Israel. * Correspondng autor. E-mal address elad@sccm.stanford.edu (M. Elad) /$ - see front matter Ó 2003 Elsever Inc. All rgts reserved. do /s (03)

2 370 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Introducton Retnex teory deals wt compensaton for llumnaton effects n mages. Te prmary goal s to decompose a gven mage S nto two dfferent mages, te reflectance mage R, and te llumnaton mage L, suc tat at eac pont ðx; yþ n te mage doman Sðx; yþ ¼Rðx; yþlðx; yþ. Te benefts of suc a decomposton nclude te ablty to remove llumnaton effects of back/front lgtng, enance potos tat nclude spatally varyng llumnaton suc as mages tat contan ndoor and outdoor zones, and correct te colors n mages by removng llumnaton nduced color sfts. Recoverng te llumnaton from a gven mage s known to be a matematcally ll-posed problem. In order to allevate ts problem, addtonal assumptons on te unknowns are requred. Te most commonly used assumpton s tat te spatally smoot parts of S orgnate from te llumnaton mage, wereas edges n S are due to te reflectance n te mage (Blake, 1985; Branard and Wandell, 1986; Faugferas, 1979; Horn, 1974; Jobson et al., 1997a,b; Land, 1977, 1983, 1986; Land and McCann, 1971; PersComm, 1998; Stockam, 1972; Terzopoulos, 1986). In a prevous paper (Kmmel et al., 2003), a new varatonal based Retnex formulaton to te Retnex problem was ntroduced and compared to oter state-of-te-art metods. Ts formulaton takes nto account te llumnaton smootness assumpton. In addton, t explots te known lmted range of te reflectance mage, and te fact tat ts mage, beng te process output, sould be vsually pleasng. Te new formulaton s sown to ave a Quadratc Programmng structure, wc guarantees an exstng unque soluton. It s also sown tat dfferent prevous Retnex algortms are essentally solutons to smlar varatonal problems. One mportant drawback wt te new varatonal approac s ts numercal mplementaton. Te unknown to be recovered n te obtaned QP optmzaton problem s te reflectance or te llumnaton mage. Tus, te number of unknowns s te number of pxels n te treated mage, wc s typcally a very large number. Solvng suc a problem requres an teratve algortm, were eac teraton ncludes bot radometrc and spatal operatons. Suc a process s known to be computatonally demandng, even f effcent QP solvers, as te one proposed n Kmmel et al. (2003), are used. Anoter problem wt te above formulaton s tat te nduced numercal process for obtanng te optmal soluton s not constraned by software/ardware consderatons. For example, n case were te llumnaton reconstructon system s restrcted to a lnear flter of pre-specfed sze, followed by a general LUT (Look- Up-Table) operaton, te algortm cannot take ts constrant nto account n te reconstructon process. In ts paper we propose to explot te same varatonal formulaton n order to defne an optmal system wt a pre-specfed structure. A general framework for suc a soluton s constructed for general processes controlled by a lmted set of parameters. Te Retnex problem n ts case translates nto a searc for optmal values for tese small number of parameters. For structures controlled lnearly by a set of free parameters, we sow tat a reduced sze QP problem s obtaned, wc guarantees a unque soluton. Several specal cases of ts general soluton are presented and analyzed

3 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Expressng te unknown llumnaton by a truncated set of bass functons. 2. A general lnear flter wt a pre-specfed kernel sze. 3. A general nonlnear Volterra flter wt pre-specfed kernel sze and order. 4. A full or polynomal Look-Up-Table (LUT). 5. A Gamma-correcton process. For all tese cases, te resultng soluton s sub-optmal compared to te orgnal Retnex metod, yet te numercal mplementatons are more effcent. Neverteless, we sow tat tese solutons succeed n enancng te nput mage for te above coces. Ts paper proposes a novel approac to tunng an mage processng algortm. We solve a reduced sze QP problem wc optmzes for te parameters of a ardware mplementaton or alternatvely an effcent computatonal algortm tat wll actually perform te mage processng effcently. Oter effcent algortms for adaptve mage processng exst, owever tey are usually not a result of a rgorous problem formulaton and optmzaton. One example of an effcent adaptve mage enancement s tone-mappng algortm by Holm (1996) wc proposes to talor a tone-map to an nput mage accordng to parameters extracted from a stogram of a tumbnal of an mage (rater tan te stogram of te full mage). Anoter example s an effcent algortm, wc muc lke te Retnex reduces te dynamc rage of mages (Durand and Dorsey, 2002). Te algortm s formulated as a nonlnear flter on te full mage. However, t s mplemented as an nterpolaton between a set of convolutons of down-sampled mages. Ts s, ndeed, a sgnfcant algortmc mprovement amed specfcally at effcent convoluton ardware/software. It s, owever, dfferent from te metod proposed n ts paper n tat t s specfc to a unque computatonal tool (lnear convolutons) wereas we propose a generc metod to determne parameters n a large class of effcent mplementatons. Furtermore, we determne tose parameters va optmzaton wt respect to a goal formulated, n our case, as a dynamc range compresson problem. Ts paper s organzed as follows In te next secton, we brefly present te varatonal Retnex formulaton, as presented n Kmmel et al. (2003). Secton 3 sows te proposed metod for extractng sub-optmal solutons to te exact Retnex problem, wle preservng te QP structure. In Secton 4, we dscuss te propertes of te new reduced sze QP problem. Results are gven n Secton 5, wt concludng remarks n Secton Te varatonal Retnex formulaton Our startng pont s te Retnex formulaton as presented n Kmmel et al. (2003). Te varatonal formulaton for Retnex reles on te followng assumptons (Kmmel et al., 2003) 1. Te llumnaton s spatally smoot. 2. Te reflectance mage, R, s restrcted to te unt nterval (0 6 R 6 1), and terefore, L P S ¼ L R.

4 372 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Te llumnaton mage s close to te nput mage, S, enancng te local contrast of te reflectance mage R. 4. Te reflectance mage R s lkely to ave a g pror probablty (Blake and Zsserman, 1987; Geman and Geman, 1984; Lagendjk and Bemond, 1991; Marroqun et al., 1987). One of te smplest pror functons used for natural mages assgns g probablty to spatally smoot mages (Lagendjk and Bemond, 1991). Defne s ¼ logðsþ, r ¼ logðrþ, andl ¼ logðlþ. If we ntegrate all te above assumptons nto one expresson we get te followng penalty functonal Mnmze Subject to l P s; Z F ½lŠ ¼ ðjrlj 2 þ aðl sþ 2 þ bjrðl sþj 2 Þ dx dy X were X s te support of te mage. a and b are free non-negatve real parameters. In te functonal F ½lŠ, te frst penalty term (jrlj 2 ) forces spatal smootness on te llumnaton mage. Te second penalty term ðl sþ 2 forces a proxmty between l and s. Te trd term s te penalty expresson for te pror. Ts term forces r to be spatally smoot. Note tat more complcated pror penalty expressons may be used allowng for sarp edges, textures, 1=f beavor, etc. (Blake and Zsserman, 1987; Geman and Geman, 1984; Lagendjk and Bemond, 1991; Marroqun et al., 1987). As long as ts expresson s purely quadratc, te above mnmzaton problem remans farly smple. Snce te numercal mplementaton s appled on sampled mages, we can rewrte te above problem usng dscrete notatons. As we sall see n te next secton, a dscrete representaton lends tself to te defnton of optmal pre-specfed system structure. Let us defne te vectors l, r, and s as te column-stack lexcograpc orderng of te llumnaton, reflectance, and orgnal mages, respectvely. Te matrces D x and D y stand for a orzontal and vertcal dscrete frst dervatve operatons. Tus, te varatonal problem transforms nto ð1þ Mnmze F ½lŠ ¼kD x lk 2 þkd y lk 2 þ akl sk 2 Subject to l P s þ bðkd x ðl sþk 2 þkd y ðl sþk 2 Þ ð2þ Te above problem (n bot representatons) as a Quadratc Programmng (QP) form (Bertsekas, 1995; Luenberger, 1987). In Kmmel et al. (2003) t was sown tat te Hessan of te functon F ½lŠ s postve defnte f a > 0. As suc, ts problem s strctly convex and as a unque soluton. An nterestng nterpretaton of te above functonal s obtaned for te case were b a. As t turns out, suc a coce for te parameters leads to a Gamma-correcton soluton. More detals about ts anomaly can be found n Appendx A.

5 3. General smplfcaton for te Retnex problem Solvng for te optmal llumnaton mage l, as defned by Eq. (2), requres an teratve algortm. Assume tat due to ardware or computatonal lmtatons, tere s a pre-defned procedure P tat we are wllng to apply on s n order to get ^l. We furter assume tat ts operaton s governed by a relatvely small number, N, of parameters denoted by. Tus, ^l ¼ Pf; sg. In order to get a good qualty estmate ^l, wc wll mtate te soluton of te exact varatonal Retnex, we defne te optmal parameter set as te soluton of te problem Mnmze F ½Š ¼F ½Pf; sgš ¼ kd x Pf; sgk 2 þkd y Pf; sgk 2 þ akpf; sg sk 2 þ bðkd x ðpf; sg sþk 2 þkd y ðpf; sg sþk 2 Þ Subject to Pf; sg P s If te operaton P s lnear wt respect to te parameter set, t can be rewrtten as Pf; sg ¼Mfsg, were Mfsg s a matrx of sze ½L x L y N Š, wt L x L y te sze of te mage. Mfsg s a possbly nonlnear functon matrx of te mage s. Ts specal case s mportant snce ten Eq. (3) becomes Mnmze F ½Š ¼kD x Mfsgk 2 þkd y Mfsgk 2 þ akmfsg sk 2 Subject to Mfsg P s þ bðkd x ðmfsg sþk 2 þkd y ðmfsg sþk 2 Þ and ts problem as agan a Quadratc Programmng form. In order to assure tat te functon F ½Š s strctly convex, we ave to verfy tat te Hessan of F ½Š s postve defnte (Bertsekas, 1995). Te Hessan s gven by o 2 F ½Š o 2 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) ¼ M T fsg ai þð1þbþ D T x D x þ D T y D y Mfsg Te term D T x D x þ D T y D y s te Laplacan operator (Kmmel et al., 2003). Tus, f a > 0 and Mfsg s full rank (meanng tat ts columns are lnearly ndependent), ten te Hessan s postve defnte, te functonal s strctly convex, and tere s a unque soluton. Let us explore several possbltes for te constructon of te matrx M 1. Bass functons. Snce te llumnaton s known to be spatally smoot, t can be spanned by relatvely small number of smoot bass functons. One suc possblty s to use a truncated Fourer bass. Eac suc bass functon s a complete mage of sze L x L y, ordered lexcograpcally nto a sngle column n te matrx M. Note tat n ts case, M s not a functon of s. 2. Lnear flter. In some stuatons we mgt be forced to use a lnear space nvarant flter, usng a ð2k þ 1Þð2K þ 1Þ kernel, for te constructon of ^l. Let us defne a global dsplacement operaton Dsp ½;jŠ fsg, wc dsplaces mage s by ½; jš (n te ð3þ ð4þ ð5þ

6 374 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) two axes). Ts defnton must assume a specfc boundary condton (e.g. for te condton mentoned earler, one sft left causes te rgtmost column of te mage to be replcated and represent te new enterng column from te rgt). Tus, M s bult by Mfsg ¼ Dsp ½ K; KŠ fsg;...; Dsp ½0;0Š fsg;...; Dsp ½K;KŠ fsg ;.e., by dsplacng s to all possble postons n a block of ð2k þ 1Þð2K þ 1Þ. A lnear combnaton of tese columns s a smple lnear space nvarant convoluton, as requred. A bas can be added to te lnear flter by addng one more column to te matrx Mfsg, contanng ones. Ts way, eac pxel n te estmated llumnaton mage s created by a wegted average of te local negborood, added to a prespecfed optmal bas value. 3. Full Look-Up-Table. One of te smplest, and terefore, computatonally appealng, possble operatons on an mage s a Look-Up-Table (LUT). In te general case, LUT s a map tat assgns an output value to eac nput gray-value. Assumng an 8 bt nput, te 256 output values are te parameters of ts operaton. It s not trval to see ow a LUT operaton falls nto te lnear structure Mfsg. To see tat, let us defne an ndcator operaton Ind v fsg as Ind v fsg ¼ 1; s½; jš ¼v; 0; s½; jš 6¼ v;.e., all pxels n te mage wc are equal to v are set to 1, and te remanng pxels are zeroed. Usng ts operaton, te LUT operaton can be modeled as te multplcaton of te followng sparse matrx Mfsg ¼½Ind 0 fsg; Ind 1 fsg;...; Ind 255 fsgš by a vector representng te 256 output gray-values. In ts case, M uses s n a nonlnear manner. 4. Polynomal Look-Up-Table. If 256 unknowns are ard to get, or f te desred LUT sould be smoot, a polynomal approxmaton of t can be used nstead. Te operaton on te nput mage s wll be ^l½; jš ¼ XN 1 k s k ½; jš k¼0 In ts case, te unknown vector wll be te coeffcents of te above sum, and te matrx M s represented by Mfsg ¼ s 0 ; s 1 ;...; s N 1 ; were te operaton s k s appled per entry. 5. Volterra flterng. A possble nonlnear extenson to te lnear flter s te Volterra fler. Instead of lnearly wegtng gray values of a negborood, Volterra flter proposes lnear wegt of polynomals of te gray values. Te resultng matrx M turns out to combne te matematcal macnery of bot te lnear flter and te ð6þ

7 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) polynomal LUT approxmaton. For example, for a smple 2-pxels spatal Volterra operaton of up to second -order polynomals we get Mfsg ¼ Dsp ½1;0Š fsg; Dsp ½0;0Š fsg; Dsp ½0;0Š fsg 0 ;...; Dsp ½0;0Š fsgdsp ½1;0Š fsg; Dsp ½0;0Š fsg 2 ; Dsp ½1;0Š fsg 2 ; were Dsp ½0;0Š fsg 0 s a vector of ones. Oter possbltes can be formulated usng ts approac, and n partcular, a combnaton of te above optons s also possble. It s mportant to note tat wle coosng a lnear structure of te form Pf; sg ¼Mfsg s lmtng, we see tat t leads to a dverse set of optons, commonly used as processng-blocks n mage processng systems (especally ardware ones). Tus, not only we ave ganed some smplfcaton wt respect to te computatonal complexty due to ts coce, but we also gan te ablty to perform te Retnex-correcton process n ardware wt optmzed parameters of lnear flterng, LUT, and more. We also note tat te approac taken above (bot te general and te subsequent lnear) can be posed as te orgnal varatonal Retnex metod as posed n (2) wt an addtonal constrant of te form ^l ¼ Pf; sg. Tus ts addtonal constrant lmts te soluton space and terefore yelds suboptmal result. As we sall see next, ts loss n output qualty comes wt a gan n stablty speed of mplementaton. Te lmted soluton space could be nterpreted as a varant of regularzaton tat stablzes te problem and ts numercal soluton. 4. Propertes of te reduced Retnex problem In all te above optons for te coce of Mfsg, te QP problem becomes Mnmze F ½Š ¼ 1 2 T Hfsg þ T QfsgþConst ð7þ Subject to Mfsg P s; were Hfsg ¼M T fsg ai þð1þbþ D T x D x þ D T y D y Mfsg; Qfsg ¼ 2M T fsg b D T x D x þ D T y D y þ ai s; ð8þ Const ¼ s T b D T x D x þ D T y D y þ ai s and bot H and Q are relatvely small. For a typcal problem treatng mages of sze pxels, te orgnal Retnex procedure requres te recovery of 1e6 unknown pxels and te Hessan of te QP s of sze 1e6 1e6 entres. Gong to te new approac wt 1000 parameters (a reasonable number for te optons dscussed n te prevous secton), te number of unknowns s 1000 and te Hessan s of sze

8 376 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) entres. Moreover, for te MðsÞ proposed ere, te condton-number of te new Hessan s far better tan te orgnal one, leadng to a better numercal stablty. In order to solve te above QP problem, we need to compute Hfsg, Qfsg, and te set of constrants. Te constant value appearng n te penalty functon as no mpact on te soluton and tus can be omtted. Here are some comments about ts computatonal process. For te computaton of H, we ave to apply a lnear operator ai þð1þbþ D T x D x þ D T y D y ð9þ on eac of te mages representng te columns of M T fsg. Te fnal stage s an applcaton of an nner product between tese mages and te mages n Mfsg. If te columns of Mfsg are obtaned by pure global dsplacement (as n te lnear flter case), ten, nstead of computng te flterng results per row, we can smply generate tem by dsplacng te fltered result for te center column. For te case were Mfsg s bult by bass functons, ts matrx does not depend on s. Terefore, te matrx Hfsg can be computed off-lne once. Computng Q s done by applyng te flter ½aI þ bðd T x D x þ D T y D yþš on te nput mage s, and agan, performng an nner product wt te columns of M T fsg. A second opton s to use te matrx ai þð1þbþ D T x D x þ D T y D y Mfsg as obtaned from te computaton of H, and perform an nner product wt te nput mage s. In general, te constrant set MðsÞ P s as L x L y nequalty constrants. Ts may become probtve, especally because we would lke to avod te actual storage of te matrx M. As t turns out, a very effectve sortcut can be used n order to prune te number of constrants for te LUT desgn n bot te full and te polynomal approxmaton. In tese two cases, te number of dfferent constrants s smaller or equal to 256, snce for all te pxels gettng gray value v, all te correspondng constrants are dentcal. Prunng te constrants-set s done by frst fndng all te exstng gray values n te mage s, and ten creatng for eac one of tem a scalar nequalty constrant. For example, f all (8 bts) gray values are occuped, ten for te full LUT desgn we ave te constrant P Ts constrant requres te resultng LUT to be bounded from below by te unty LUT operaton.

9 For te same case, te trd-order polynomal LUT wll ave te constrant P For te oter cases were suc smple prunng s not possble, an nterestng problem s ow to effcently prune redundant constrants from suc a set, eter as an accurate or as an approxmated process. Ts problem s left for future researc. In our smulatons we exploted te fact tat constrants for negborng pxels are expected to be smlar. Terefore, we smply decmated te constrants lst. Anoter nterestng opton wt respect to te LUT-based approaces s to enforce monotoncty on te results. If desred, ts property can be forced as a set of addtonal nequalty constrants. Monotoncty s guaranteed f te frst dervatve of te obtaned LUT s non-negatve. For te complete LUT desgn, ts requrement s formulated by M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) P In te polynomal approac we requre tat te dervatve of (6) s non-negatve for all. d^l ds ¼ XN 1 k k s k 1 P 0; k¼1 wc mples ðn 1Þ0 N ðn 1Þ1 N ðn 1Þ2 N P ðn 1Þ255 N 2 N 1 0 Assumng tat te mage s s gven n te Log doman, Gamma correcton s merely te multplcaton of s by a constant value, 1=c. Tus, suc a case turns out to be a specal case of te polynomal approxmated LUT, usng a sngle column n te matrx M,.e., Mfsg ¼s. Note tat n ts case, te constrant s s 0 P s, wc s equvalent to 0 P 1.

10 378 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) All te above refers to treatng te mage n te 8 bt gray-value doman. If te algortms are to be appled n te Log doman, eac nput gray value x s replaced by logð1 þ xþ=8. Note tat for te LUT operatons, ts cange mples a cange of entres n te constrant matrx. As to te overall complexty of te resultng algortm, t generally depends on te specfc metod used (essentally te coce of MðsÞ and te number of te parameters n te assumed model). Generally speakng, te complexty obtaned s of te order of one teraton of te orgnal Retnex algortm or below, and tus expected to be solved muc faster. 5. Results In ts secton we present several examples to demonstrate te qualty of te proposed reduced complexty approac. Trougout ts secton we apply an RGB Retnex algortm,.e., eac color layer s treated separately (Kmmel et al., 2003). For all te sown results we ave used te parameters a ¼ 001, b ¼ 1e 5. We start by sowng te test mages, and te full fledged varatonal-based Retnex algortm results (Fg. 1) tat serve as reference. Next, Fg. 2 sows te results of a full LUT for te two nput mages. Fg. 3 sows te obtaned LUT for te tree color components. As can be seen, te look-up-tables are above te dentty functon, wc means tat te reflectance mage turns out to be a brgter verson of te nput mage, as expected. Fg. 1. Full fledged varatonal-based Retnex results (left) source mage; (mddle) estmated reflectance mage; (rgt) estmated llumnaton mage.

11 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 2. Optmal complete Look-Up-Table results (left) source mage; (mddle) estmated reflectance mage; (rgt) estmated llumnaton mage. Fg. 3. Optmal complete Look-Up-Table results (left) te LUT for te frst mage ( cld ); (rgt) te LUT for te second mage ( Houses ). Note tat wen comparng te results of te full fledged Retnex to te full LUT n te frst mage, we get te mpresson tat te LUT result s better. Ts s because Retnex algortms n general attempt to recover te reflectance mage, tereby enancng contrast n te dark tones. In order to get a vsually pleasng output, some of te llumnaton sould be returned to te mage (Kmmel et al., 2003). In tese smulatons te orgnal Retnex succeeded better n recoverng detals, wc would ave resulted n a better vsual qualty f te obtaned reflectance results were to be used n te approprate enancement algortm (Kmmel et al., 2003).

12 380 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 4 sows te results of a polynomal approxmated LUT of sxt order. Fg. 5 sows te resultng LUT for te tree color components. A close resemblance can be seen between tese results and te ones obtaned by te full LUT. Note tat te LUT are not monotone for te frst mage, but te descendng part of te LUT corresponds to gray values tat do not exst n te mage, and tus, te fact tat te LUT s not monotone sould not be dsturbng. Our last example n te famly of LUT operatons s Gamma correcton, n wc a sngle parameter s determned. Fgs. 6 and 7 sow te results and te resultng Fg. 4. Optmal polynomal Look-Up-Table results (left) source mage; (mddle) estmated reflectance mage; (rgt) estmated llumnaton mage. Fg. 5. Optmal polynomal Look-Up-Table results (left) te LUT for te frst mage ( cld ); (rgt) te LUT for te second mage ( Houses ).

13 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 6. Optmal Gamma Look-Up-Table results (left) source mage; (mddle) estmated reflectance mage; (rgt) estmated llumnaton mage. Fg. 7. Optmal Gamma Look-Up-Table results (left) te LUT for te frst mage ( cld ); (rgt) te LUT for te second mage ( Houses ). LUT. Agan, we see tat te results resemble te ones obtaned te prevous two metods. Fg. 8 sows te results obtaned for a 5 5 lnear kernel, combned wt a bas value. Fg. 9 sows te actual kernels for te tree colors. Notce tat te llumnaton mage s obtaned from te nput mage by blurrng, and terefore, te reflectance mage as a sarpenng effect. In ts smulaton we cose to decmate te set of constrants by a factor 101,.e., for every 10 constrants, only te frst was

14 382 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 8. Optmal lnear flter + bas results (left) source mage; (mddle) estmated reflectance mage; (rgt) estmated llumnaton mage. used. Ts way we smplfed te QP soluton, relayng on te expected spatal smootness of tese nequaltes. As a fnal example we sow te results of te bass functons approac. We cose to use a two-dmensonal DCT bass functons, usng te L 2 functons taken from te square of L L startng from te orgn. In te followng smulaton L ¼ 5. Fg. 10 sows te results obtaned for ts metod. All tese smulatons were done usng Matlab v6.0, run on a Pentum-III Wndows macne wt 500 MHz processor and 200 MB RAM. Te mages descrbed ere are of sze Te orgnal Retnex algortm bult on mult-resoluton solver approac (see (Kmmel et al., 2003) for more detals) takes 16 s 1 ts s a gly effcent numercal sceme, mplyng tat a regular teratve solver s expected to take muc longer. Te smplfed parametrc algortm wt a drect or a polynomal LUT requres 2 s, te Gamma LUT takes on te average 0.7 s, te lnear + bas parameterzaton requres 1.5 s, and te DCT metod requres 2.1 s. We sould stress, owever, tat te proposed parametrc metods are expected to be far more effcent wen ardware, DSP, or even plan C-code mplementaton s consdered. Also, wen dscussng runnng te same algortm for a set of mages we expect to see a furter speedup snce some preparaton of te matrces nvolved could be done only once. As a last pont n ts secton we return to te clam made n Secton 3 about te coce b a causng te soluton to become a pure Gamma correcton. Appendx A gves an explanaton for ts property and Appendx B sows tat ndeed, for suc a 1 All run-tmes reported ere are average ones.

15 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 9. Optmal lnear flter + bas results (Up) te kernels for te frst mage ( cld ); (down) te kernels for te second mage ( Houses ). coce of parameters, te above metods tend to gve close to Gamma-correcton results. 6. Concludng remarks In ts paper we presented several metods for reducng te complexty of te varatonal Retnex metod, as descrbed n Kmmel et al. (2003), wle restrctng te soluton to ave a pre-specfed structure. Systems based on a Look-Up-Table, lnear or nonlnear flterng, and expanson by arbtrary bass functons are sown to be

16 384 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 10. Optmal combnaton of 2D DCT bass functons (left) source mage; (mddle) estmated reflectance mage; (rgt) estmated llumnaton mage. specal cases of te proposed metodology. Ts way, nstead of searcng for an unknown llumnaton mage, te newly defned problem focuses ts searc on a very small number of free parameters and tereby controllng te cosen system. It was sown tat te proposed approac yelds reasonable qualty output wt very effcent numercal mplementatons. An alternatve lesson from te obtaned results s a caracterzaton of te eurstcs we want to employ n order to get a Retnex mtaton. For example, usng a LUT t was found (see Fg. 3) tat smple dynamc range stretcng, followed by Gamma correcton s te effectve soluton. As anoter example, te lnear flter approac (see Fg. 9) ndcated tat Retnex effect s obtaned by sarpenng, followed, agan, by Gamma correcton. Appendx A. Te Retnex algortm for b a teory Let us look at te varatonal expresson (Eq. (2)), and assume tat b a. Ifwe gnore te nequalty constrants, te optmal mage l sould satsfy ð1 þ bþ D T x D x þ D T y D y þ ai ^l ¼ b D T þ D T y D y þ ai s ð10þ Usng te assumpton b a we get tat ^l ð1 þ bþ D T x D x þ D T y D y 1 b D T x D x x D x þ D T y D y s ¼ b 1 þ b s;

17 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Fg. 11. Optmal LUT desgned for b a (left) te results as a LUT mappng for te mage Cld ; (rgt) te results as a LUT mappng for te mage Houses. were we ave assumed tat te Laplacan operator s nvertble. Snce ts s not true, te term ai stands as an algebrac regularzaton to te nverted matrx. An nterestng property of ts soluton s tat t also satsfes te constrants, and terefore, ts s te soluton for te orgnal optmzaton problem. We sould remember tat te mages are n te Log doman, and ter values are negatve. 2 Tus, multplyng by a postve fracton smaller tan 1, we get tat ð1 þ bþ 1 bs s ndeed ger tan s, as requred. Ts result mples tat ^r ¼ s ^l 1 1 þ b s ) ^R ¼ expf^rg ¼ S^L S1=ð1þbÞ ; wc s exactly te Gamma-correcton operaton S 1 c. Terefore, f te requred Gamma for a gven mage s known, t may gve an ndcaton as to te requred value of te parameter b. Note tat n Eq. (10), addng a constant to te soluton does not mpact te correctness of ts equaton, snce D T x D x þ D T y D y þ b 1 þ b s ¼ D T x D x þ D T y D b y 1 þ b s Te constant must be suc tat t does not contradct te constrant þ b 1 þ b s P s! P 1 1 þ b MaxðsÞ P 1 1 þ b s Among all te possble values of, preferred values are tose wc cause te llumnaton mage to be as close as possble to te nput mage. Ts s true f a s not 2 We assume tat te nput mage s normalzed to te range ½0; 1Š.

18 386 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) zero (even f t s very small). Ts mples tat sould be te smallest possble value wc stll satsfes te constrant. For te case were MaxðsÞ ¼0 (n te Log doman), we get tat ¼ 0. For cases were s does not fll te entre dynamc range, can be cosen as a negatve value tat wll mprove te proxmty between s and ^l. Terefore 1 ¼ mn 1 þ b MaxðsÞ; 0 Fg. 12. Optmal lnear flter + bas for b a (Up) te obtaned kernels for te mage Cld ; (down) te obtaned kernels for te mage Houses.

19 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) Effectvely, te value of plays a role of stretcng before te Gamma correcton, snce ^r ¼ s ^l ¼ þ 1 1 þ b s ¼ 1 1 þ b s mn 1 1 þ b MaxðsÞ; 0 1 ¼ mn 1 þ b ½s b MaxðsÞŠ; 1 þ b s ; ( ) 1=ð1þbÞ S ^R ¼ mn ; ½SŠ 1=ð1þbÞ MaxðSÞ Appendx B. Te Retnex algortm for b a results In ts appendx we sow troug several examples tat ndeed we get a Gamma correcton for te coce b a. More specfcally, n te followng smulatons, we ave cosen a ¼ 1e 6, b ¼ 1. Obvously, followng te results of te prevous appendx, te results are supposed to be very close to exact Gamma correcton wt effectve Gamma value of c ¼ 1 þ b ¼ 2. Fg. 11 sows te obtaned look-up-tables for te full LUT case. For bot mages, we can see tat te optmal LUT as a sape of a stretcng, followed by Gamma correcton. Fg. 12 sows te obtaned kernels n te applcaton of lnear flter + bas approac. In ts case, nstead of performng a spatal sarpennng operaton, te kernels are cosen to be a smple unt operaton multpled by some constant. Fg. 13 sows te results of te lnear + bas approac by plottng te output versus te nput as a Look-Up-Table. Agan, we see tat te results are an effectve LUT avng te sape of a Gamma correcton, as expected. Fg. 13. Optmal lnear flter + bas for b a (left) te results as a LUT mappng for te mage Cld ; (rgt) te results as a LUT mappng for te mage Houses.

20 388 M. Elad et al. / J. Vs. Commun. Image R. 14 (2003) In all tese examples, te measured Gamma n tese graps was found to be 2, as expected. References Bertsekas, D.P., Non-Lnear Programmng. Atena Scentfc, Belmont, MA. Blake, A., Boundary condtons of lgtness computaton n Mondran world. Comput. Vson Grapcs Image Process. 32, Blake, A., Zsserman, A., Vsual Reconstructon. MIT Press, Cambrdge, MA. Branard, D.H., Wandell, B., Analyss of te Retnex teory of color vson. J. Opt. Soc. Am. A 3, Durand, F., Dorsey, J., Fast blateral flterng for te dsplay of g dynamc range mages. ACM Trans. Grapcs 21 (3), Faugferas, O.D., Dgtal mage color processng wtn te framework of a uman vsual system. IEEE Trans. ASSP 27, Geman, S., Geman, D., Stocastc relaxaton, Gbbs dstrbuton, and te Bayesan restoraton of mages. IEEE Trans. Pattern Anal. Macne Intell. 6, Holm, J., Strategy for Pctoral Dgtal Image Processng (PDIP). In Proceedngs of te Color Imagng Conference, November, pp Horn, B.K.P., Determnng lgtness from an mage. Comput. Grapcs Image Process. 3, Jobson, D.J., Raman, Z., Woodell, G.A., 1997a. Propertes and performance of te center/surround retnex. IEEE Trans. Image Proc. 6, Jobson, D.J., Raman, Z., Woodell, G.A., 1997b. A multscale Retnex for brdgng te gap between color mages and te uman observaton of scenes. IEEE Trans. Image Proc. 6. Kmmel, R., Elad, M., Saked, D., Keset (Kresc), R., Sobel, I., A varatonal framework for Retnex. Int. J. Comput. Vson 52 (1), Lagendjk, R.L., Bemond, J., Iteratve Identfcaton and Restoraton of Images. Kluwer Academc Publsng, Boston, MA. Land, E.H., Te Retnex teory of color vson. Sc. Am. 237, Land, E.H., Recent advances n te retnex teory and some mplcatons for cortcal computatons color vson and te natural mage. Proc. Natl. Acad. Sc. USA 80, Land, E.H., An alternatve tecnque for te computaton of te desgnator n te Retnex teory of color vson. Proc. Natl. Acad. Sc. USA 83, Land, E.H., McCann, J.J., Lgtness and te Retnex teory. J. Opt. Soc. Am. 61, Luenberger, D.G., Lnear and Non-Lnear Programmng, second ed. Addson-Wesley, Menlo-Park, CA. Marroqun, J., Mtter, J., Poggo, T., Probablstc soluton for ll-posed problems n computatonal vson. J. Am. Statst. Assoc. 82, McCann, J.J., Sobel, I.,1998. Experments wt Retnex, HPL Color Summt. Stockam Jr., T.G., Image processng n te context of a vsual model. Proc. IEEE 60, Terzopoulos, D., Image analyss usng multgrd relaxaton metods. IEEE Trans. PAMI 8,