Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images

Transcription

1 1 Modelng Radometr Unertanty for Vson wth Tone-mapped Color Images Ayan Chakrabart, Yng Xong, Baohen Sun, Trevor Darrell, Danel Sharsten, Todd Zkler, and Kate Saenko arxv: v [s.cv] 9 Apr 14 Abstrat To produe mages that are sutable for dsplay, tone-mappng s wdely used n dgtal ameras to map lnear olor measurements nto narrow gamuts wth lmted dynam range. Ths ntrodues non-lnear dstorton that must be undone, through a radometr albraton proess, before omputer vson systems an analyze suh photographs radometrally. Ths paper onsders the nherent unertanty of undong the effets of tone-mappng. We observe that ths unertanty vares substantally aross olor spae, makng some pxels more relable than others. We ntrodue a model for ths unertanty and a method for fttng t to a gven amera or magng ppelne. One ft, the model provdes for eah pxel n a tone-mapped dgtal photograph a probablty dstrbuton over lnear sene olors that ould have ndued t. We demonstrate how these dstrbutons an be useful for vsual nferene by norporatng them nto estmaton algorthms for a representatve set of vson tasks. Index Terms Radometr Calbraton, Camera Response Funtons, Tone-mappng, Statstal Models, Sgnal-dependent Nose, HDR Imagng, Image Fuson, Depth Estmaton, Photometr Stereo, Image Restoraton, Deblurrng. 1 INTRODUCTION The prolferaton of dgtal ameras has reated an exploson of photographs beng shared onlne. Most of these photographs exst n narrow-gamut, low-dynam range formats typally those defned n the srgb or Adobe RGB standards beause they are ntended prmarly for dsplay through deves wth lmted gamut and dynam range. Whle ths workflow s effent for storage, transmsson, and dsplay-proessng, t s unfortunate for omputer vson systems that seek to explot onlne photo olletons to learn objet appearane models for reognton; reonstrut three-dmensonal (3D) sene models for vrtual toursm; enhane mages through proesses lke denosng and deblurrng; and so on. Indeed, many of the omputer vson algorthms requred for these tasks use radometr reasonng and therefore assume that mage olor values are dretly proportonal to spetral sene radane (alled RAW olor hereafter). But when a onsumer amera renders or globally tone-maps ts dgtal lnear olor measurements to an output-referred, narrow-gamut olor enodng (alled JPEG olor hereafter), ths proportonalty s RAW Blue Raw.6 Green.4 (,44,48) (161,6,46) (37,53,53) (55,55,55) (199,49,1) (5,5,3) (55,55,145) (55,4,5) (189,1,83) (55,187,159) RAW Red Fg. 1. Clusters of RAW measurements that eah map to a sngle JPEG olor value (ndated n parentheses) n a dgtal SLR amera (Canon EOS 4D). Close-ups of the lusters emphasze the varatons n luster sze and orentaton. When nvertng the tone-mappng proess, ths strutured unertanty annot be avoded. almost always destroyed. 1 In omputer vson, we try to undo the non-lnear effets of tone-mappng so that radometr reasonng about onsumer photographs an be more effetve. To ths end, there are many methods for fttng parametr forms to the global tone-mappng operators appled by olor ameras so-alled radometr albraton" meth- AC, YX, and TZ are wth the Harvard Shool of Engneerng and Appled Senes, Cambrdge, MA 138. BS and KS are wth the Unversty of Massahusetts Lowell, Lowell, MA TD s wth the Unversty of Calforna Berkeley, Berkeley, CA 947. DS s wth Mddlebury College, Mddlebury, VT E-mals: {ayan@ees.harvard.edu, yxong@seas.harvard.edu, bsun@s.uml.edu, trevor@ees.berkeley.edu, shar@mddlebury.edu, zkler@seas.harvard.edu, saenko@s.uml.edu}. 1. Some omments on termnology. We use olloqual phrases RAW olor and JPEG olor respetvely for lnear, sene-referred olor and non-lnear, output-referred olor. The latter does not nlude lossy ompresson, and should not be onfused wth JPEG ompresson. Also, we use (global) tone-map for any spatally-unform, non-lnear map of eah pxel s olor, ndependent of the values of ts surroundng pxels. It s nearly synonymous wth the ommon phrase radometr response funton [1], but generalzed to nlude ross-hannel maps.

2 Fg.. Derenderng Unertanty (for the Canon PowerShot S9). (Left) Total varane of the dstrbutons of RAW values that are tone-mapped to a set of rendered olor values that le along a lne n JPEG spae; along wth ellpses deptng orrespondng dstrbutons along two dmensons n the RAW sensor spae. (Rght) 3D volumes deptng the hange n derenderng varane aross JPEG spae. ods [1] [7] and t s now possble to ft many global tone-mappng operators wth hgh preson and auray [6]. However, one these maps are estmated, standard prate for undong olor dstorton n observed non-lnear JPEG olors s to apply a smple nverse mappng n a one-to-one manner [1] [7]. Ths gnores the rtal fat that forward tone-mappng proesses lead to loss of nformaton that s hghly strutured. Tone-mappng s effetve when t leads to narrowgamut mages that are nonetheless vsually-pleasng, and ths neessarly nvolves non-lnear ompresson. One the ompressed olors are quantzed, the reverse mappng beomes one-to-many as shown n Fg. 1, wth eah nonlnear JPEG olor beng assoated wth a dstrbuton of lnear RAW olors that an ndue t. The amount of olor ompresson n the forward tone-map, as well as the (hue/lghtness) dretons n whh t ours, hange onsderably aross olor spae. As a result, the varanes of reverse-mapped RAW olor dstrbutons unavodably span a substantal range, wth some predted lnear RAW olors beng muh more relable than others. How an we know whh predted RAW olors are unrelable? Intutvely, the forward ompresson (and thus the reverse unertanty) should be greatest near the boundary of the output gamut, and prattoners often leverage ths ntuton by heurstally gnorng all JPEG pxels that have values above or below ertan thresholds n one or more of ther hannels. However, as shown n Fg., the varanes of nverse RAW dstrbutons tend to hange ontnuously aross olor spae, and ths makes the hoe of suh thresholds arbtrary. Moreover, ths heurst approah reles on dsardng nformaton that would otherwse be useful, beause even n hghvarane regons, the RAW dstrbutons tell us somethng about the true sene olor. Ths s espeally true where the RAW dstrbutons are strongly orented (Fg. 1 and bottom-left of Fg. ): even though they have hgh total varane, most of ther unertanty s ontaned n one or two dretons wthn RAW olor spae. In ths paper, we argue that vson systems an beneft substantally by norporatng a model of radometr unertanty when analyzng tone-mapped, JPEG-olor mages. We ntrodue a probablst approah for vsual nferene, where (a) the albrated estmate of a amera s forward tone-map s used to derve a probablty dstrbuton, for eah tone-mapped JPEG olor, over the RAW lnear sene olors that ould have ndued t; and (b) the unertanty embedded n these dstrbutons s propagated to subsequent vsual analyses. Usng a varety of ameras and new formulatons of a representatve set of lass nferene problems (mult-mage fuson, photometr stereo, and deblurrng), we demonstrate that modelng radometr unertanty s mportant for ahevng optmal performane n omputer vson. The paper s organzed as follows. After related work n Se., Se. 3 revews parametr forms for modelng

3 3 the global tone-maps of onsumer dgtal ameras and desrbes an algorthm for fttng model parameters to offlne tranng data. In Se. 4, we demonstrate how any forward tone-map model an be used to derve per-pxel nverse olor dstrbutons, that s, dstrbutons for lnear RAW olors ondtoned on the JPEG olor reported at eah pxel. Seton 5 shows how the unertanty n these nverse dstrbutons an be propagated to subsequent vsual proesses, by ntrodung new formulatons of a representatve set of lassal nferene tasks: mage fuson (eg., [3]); three-dmensonal shape va Lambertan photometr stereo (eg., [8]); and removng amera shake va mage deblurrng (eg., [9]). RELATED WORK The problem of radometr albraton, where the goal s nvertng non-lnear dstortons of sene radane that our durng mage apture and renderng, has reeved onsderable attenton n omputer vson. Untl reently, ths albraton has been formulated only for graysale mages, or for olor mages on a per-hannel-bass by assumng that the radometr response funton n eah hannel ats ndependently [1] [4]. Whle early varants of ths approah parametrzed these response funtons smply as an exponentaton (or gamma orreton ) wth the exponent as a sngle model parameter, later work sought to mprove modelng auray by onsderng more general polynomal forms [4]. Sne these models have a relatvely small number of parameters, they have featured n several algorthms for selfalbraton parameter estmaton from mages aptured n the wld, wthout albraton targets through analyss of edge profles [1], [11], mage statsts [1], [13], or exposure staks of mages [1] [3], [14] [16]. However, per-hannel models annot aurately model the olor proessng ppelnes of most onsumer ameras, where the lnear sensor measurements span a muh wder gamut than the target output format. To be able to generate mages that look good on lmted-gamut dsplays, these ameras ompress out-ofgamut and hgh-lumnane olors n ways that are as pleasng as possble, for example by preservng hue. Ths means that two sene olors wth the same raw sensor value n ther red hannels an have very dfferent red values n ther mapped JPEG output f one RAW olor s sgnfantly more saturated than the other. Chakrabart et al. [5] nvestgated the auray of more general, ross-hannel parametr forms for global tonemappng n a number of onsumer ameras, nludng mult-varate polynomals and ombnatons of rosshannel lnear transforms wth per-hannel polynomals. Whle they found reasonable fts for most ameras, the resdual errors remaned relatvely hgh even though the albraton and evaluaton were both lmted to mages of a sngle relatvely narrow-gamut hart. Km et al. [6] mproved on ths by expltly reasonng about the mappng of out-of-gamut olors. Ther model onssts of a asade of: a lnear transform, a per-hannel polynomal, and a ross-hannel orreton for out-ofgamut olors usng radal bass funtons. The forward tone-map model we use n ths paper (Se. 3) s strongly motvated by ths work, although we fnd a need to augment the albraton tranng data so that t better overs the full spae of measurable RAW values. All of these approahes are foussed on modelng the dstorton ntrodued by global tone-mappng. They do not, however, onsder the assoated loss of nformaton, nor the strutured unertanty that exsts when the dstorton s undone as a pre-proess for radometr reasonng by vson systems. Indeed, whle the beneft of undong radometr dstorton has been dsussed n the ontext of varous vson applatons (eg., deblurrng [11], [17], hgh-dynam range magng [18], vdeo segmentaton [19]), prevous methods have reled exlusvely on determnst nverse tone-maps that gnore the strutured unertanty evdent n Fgures 1 and. The man goal of ths of ths paper s to demonstrate that the benefts of undong radometr dstorton an be made sgnfantly greater by expltly modelng the unertanty nherent to nverse tone-mappng, and by propagatng ths unertanty to subsequent vsual nferene algorthms. A earler verson of ths work [] presented a dret method to estmate nverse RAW dstrbutons from albraton data. In ontrast, we ntrodue a two-step approah, where (a) albratons mages are used to ft the forward determnst tone-map for a gven amera, and (b) the model s nverted probablstally. We fnd that ths leads to better albraton and better nverse dstrbutons wth less albraton data. Fnally, we note that our proposed framework apples to statonary, global tone-mappng proesses, meanng those that operate on eah pxel ndependent of ts neghborng pxels, and are unhangng from sene to sene. Ths s applable to many exstng onsumer ameras loked nto fxed magng modes ( portrat, landsape et. ), but not to loal tone-mappng operators that are ommonly used for HDR tone-mappng. 3 CAMERA RENDERING MODEL Before ntrodung our radometr unertanty model n Se. 4-5, we revew and refne here a model for the forward tone-maps of onsumer ameras, along wth offlne albraton proedures. We use a smlar approah to Km et al. [6], and employ a two-step model to aount for a amera s proessng ppelne a lnear transform and per-hannel polynomal, followed by a orretve mappng step for out-of-gamut and saturated olors. The end result s a determnst forward map J : x y from RAW trolor sensor measurements at a pxel x [, 1] 3 to orrespondng rendered JPEG olor values y {[, 55] Z} 3. Readers famlar wth [6] may prefer to skp dretly to Se. 4, where we present how to nvert the modeled tone-maps probablstally.

4 4 Fg. 3. Renderng Model. We model a amera s proessng ppelne usng a two step-approah: (1) a 3 3 lnear transform and ndependent per-hannel polynomal; followed by, () a orreton to aount for devatons n the renderng of saturated and out-of-gamut olors. 3.1 Model As shown n Fg. 3, we model the mappng J : x y as: ỹ 1 f(v1 T x) ỹ = ỹ = f(v T x), (1) f(v3 T x) ỹ 3 y = Q B(ỹ) + g 1 (ỹ) g (ỹ) g 3 (ỹ), () where v 1, v, v 3 R 3 defne a lnear olor spae transform, B( ) bounds ts argument to the range [, 55], and Q( ) quantzes ts arguments to 8-bt ntegers. Equaton (1) above orresponds to the ommonly used per-hannel polynomal model (eg., [4], [5]). Spefally, f( ) s assumed to be a polynomal of degree d: f(x) = d α x, (3) = where α are model parameters. We use seventh-order polynomals (.e., d = 7) n our mplementaton. Motvated by the observatons n [6], ths polynomal model s augmented wth an addtve orreton funton g( ) n () to aount for devatons that result from amera proessng to mprove the vsual appearane of rendered olors. We use support-vetor regresson (SVR) wth a Gaussan radal bass funton (RBF) kernel to model these devatons,.e., eah g ( ), {1,, 3} s of the form: g (ỹ) = λ : exp ( γ ỹ y : ), (4) where λ :, y : and γ are also model parameters. 3. Parameter Estmaton Next, we desrbe an algorthm to estmate the varous parameters of ths model from a set of albraton mages. Usng pars of orrespondng RAW-JPEG pxel values {(x t, y t )} T t=1 from the albraton set, we begn by estmatng the parameters of the standard map n (1) as: {ˆv }, {ˆα } = arg mn {v },{α } w t f(v T x t ) y t:, (5) t where {w t } are salar weghts. Lke [5], we also restrt {α } suh that f( ) s monotonally nreasng. The weghts w t are hosen wth two objetves: (a) to promote a better ft for non-saturated olors, sne we expet the orretve step n () to retfy renderng errors for saturated olors, and (b) to ompensate for non-unformty n the tranng set,.e., more tranng samples n some regons over others. Aordngly, we set these weghts as: [ T w t = S(y t ) exp ( x t x t ) ] 1, (6) t =1 σ t where S(y) s a salar funton that vares from 1 to.1 wth nreasng saturaton n y, and the seond term effetvely re-samples the tranng set unformly over the RAW spae. We set σ t = T 1/3 to orrespond to the expeted separaton between T unformly sampled ponts n the [, 1] 3 ube. One we have set the weghts, we use an approah smlar to the one n [5] to mnmze the ost n (5). We alternately optmze over only the lnear or polynomal parameters, {v } and {α } respetvely, whle keepng the other onstant. For fxed {v }, the optmal {α } an be found by usng a standard quadrat program solver, sne the ost n (5) s quadrat n {α }, and the monotonty restrton translates to lnear nequalty onstrants. For fxed {α }, we use gradent desent to fnd the optmal lnear parameters {v }. We begn the above alternatng teratons by assumng f(x) = x and settng {v } dretly usng least-squares on tranng samples for whh y t s small ths s based on the assumpton that f(x) s nearly lnear for small values of x. We then run the teratons tll onvergene, but sne the ost n (5) s not onvex, there s no guarantee that the teratons above wll yeld the global mnmum. Therefore, we restart the optmzaton multple tmes wth estmates of {v } orrespondng to random devatons around the urrent optmum. Fnally, we ompute the parameters of the gamut mappng funtons {g ( )} by usng support-vetor regresson [1] to ft ỹ [y C(ỹ)], where the tranng samples {ỹ t } are omputed from {x t } usng (1) wth the parameter values estmated above, and the kernel bandwdth parameters {γ } set usng ross-valdaton. 3.3 Datasets Our database onssts of mages aptured usng a number of popular onsumer ameras (see Tables 1 and ),

5 5 TABLE 1 RMSE for Estmated Renderng Funtons n Gray Levels for RAW-apable Cameras Camera Name Unform 1 Exp. 1 Exp. 4 Exp. 8 Exp. 4 Exp. 8 Exp. 8k Samples 1 Illum. Illum. 4 Illum. 4 Illum. 6 Illum. 6 Illum. Panason DMC LX Canon EOS 4D Canon PowerShot G Canon PowerShot S Nkon D usng an X-Rte 14-path olor heker hart as the albraton target as n [5] and [6]. However, although the hart ontans a reasonably wde gamut of olors, these olors only span a part of the spae of possble RAW values that an be measured by a amera sensor. To be able to relably ft the behavor of eah amera s tone-mappng funton n the full spae of measurable sene olors, and to aurately evaluate the qualty of these fts, we aptured mages of the hart under sxteen dfferent llumnants (we used a standard Tungsten bulb pared wth dfferent ommerally avalable gelbased olor flters) to obtan a sgnfantly wder gamut of olors. Moreover, for eah llumnant, we aptured mages wth dfferent exposure values that range from one where almost all pathes are under-exposed to one where all are over-exposed. We expet ths olleton of mages to represent an exhaustve set that nludes the full gamut of rradanes lkely to be present n a sene. Most of the ameras n our dataset allow aess to the RAW sensor measurements, and therefore dretly gve us a set of RAW-JPEG pars for tranng and evaluaton. For JPEG-only ameras, we aptured a orrespondng set of mages usng a RAW-apable amera. To use the RAW values from the seond amera as a vald proxy, we had to aount for the fat that the exposure steps n the two ameras were dfferently saled (but avalable from the mage metadata), and for the possblty that the RAW proxy values n some ases may be lpped whle those reorded by the JPEG amera s sensors were not. Therefore, the exposure stak for eah path under eah llumnant from the RAW amera was used to estmate the underlyng sene olor at a anonal exposure value, and these were then mapped to the exposure values from the JPEG amera wthout lppng. For a varety of reasons, we expet the qualty of ft to be substantally lower when usng a RAW proxy. Our model does not aount for the fat that there may be dfferent degrees of vgnettng n the two ameras, and t mpltly assumes that spetral senstvty funtons n the RAW and JPEG ameras are lnearly related (.e., that there s a bjetve mappng between lnear olor sensor measurements n one amera and those n the other), whh may not be these ase [], [3]. Moreover, despte the fat that the whte balane settng n eah amera s kept onstant we usually use daylght or tungsten we observe that some ameras exhbt varaton n the whte balane multplers they apply for dfferent senes (dfferent llumnants and exposures). For RAW-apable ameras, these multplers are n the metadata and an be aounted for when onstrutng the albraton set. However, these values are not usually avalable for JPEG-only ameras, and thus ntrodue more nose n the albraton set. 3.4 Evaluaton For eah amera, we estmated the parameters of our renderng model usng dfferent subsets of the olleted RAW-JPEG pars, and measured the qualty of ths albraton n terms of root mean-squared error (RMSE) values (between the predted and true JPEG values, n terms of gray levels for an 8-bt mage) on the entre dataset. These RMSE values for the RAW-apable amera are reported n Table. 1. The frst of these subsets s smply onstruted wth 8 random RAW-JPEG pars sampled unformly aross all pars, and as expeted, ths yelds the best results. Sne apturng suh a large dataset to albrate any gven amera may be pratally burdensome, we also onsder subsets derved from a lmted number of llumnants, and wth a lmted number of exposures per-llumnant. The exposures are equally spaed from the lowest to the hghest, and the subset of llumnants are hosen so as to maxmze the dversty of nluded hromattes spefally, we order the llumnants suh that for eah n, the onvex hull of the RAW R-G hromattes of pathes from the frst n llumnants has the largest possble area. Ths order s determned usng one of the ameras (the Panason DMC LX3), and used to onstrut subsets for all ameras. We fnd that dfferent ameras show dfferent degrees of senstvty to dversty n exposures and llumnants, but usng four llumnants wth eght exposures represents a reasonable aquston burden whle also provdng enough dversty for relable albraton n all ameras. On the other hand, mages of the hart under only a sngle llumnant, even wth a large number of exposures, do not provde a dverse enough samplng of the RAW sensor spae to yeld good estmates of the renderng funton aross the entre dataset. Table shows RMSE values obtaned from albratng JPEG-only ameras, and as expeted, these values are substantally (approx. 3 to 4 tmes) hgher than those

6 G RAW Sensor Spae R.8 1 Gamut Correton Fg. 4. Estmated Gamut Correton Funton for Canon PowerShot S9. For dfferent sles of the RAW ube, we show the magntude of the shft n eah rendered JPEG hannel (saled by a fator of 8) due to gamut orreton. TABLE RMSE n Gray Levels for JPEG-only ameras Camera Name Raw Proxy RMSE (w/ 8k Samples) Fujflm J1 Panason DMC LX3 1.4 Panason DMC LZ8 Canon PowerShot G9 9.8 Samsung Galaxy S3 Nkon D for RAW-apable ameras. Note that for ths ase, we only show results for the unformly sampled tranng set, sne we fnd parameter estmaton to be unstable when usng more lmted subsets. Ths mples that albratng JPEG-only ameras wth a RAW proxy s lkely to requre the aquston of larger sets of mages, and perhaps more sophstated fttng algorthms that expltly nfer and aount for vgnettng effets, senedependent varatons n whte balane, et.. Fg. 4 llustrates the devatons due to the gamut orreton step n our model, usng the estmated renderng funton for one of the albrated ameras. We see that whle ths funton s relatvely smooth, ts varatons learly an not be deomposed as per-hannel funtons. Ths onfrms the observatons n [6] on the neessty of nludng a ross-hannel orreton funton. 4 PROBABILISTIC INVERSE The prevous seton dealt wth omputng an aurate estmate of tone-mappng funton appled by a amera. However, the man motvaton for albratng a amera s to be able to nvert ths tone-map and use avalable JPEG values bak to derve radometrally meanngful RAW measurements that are useful for omputer vson applatons. But t s easy to see that ths nverse s not unquely defned, sne multple sensor measurements an be mapped to the same JPEG output as a result of the quantzaton that follows the ompressve map n (), wth hgher ntenstes and saturated olors experenng greater ompresson, and therefore more unertanty n ther reovery from reported JPEG values. Therefore, nstead of usng a determnst nverse funton, we defne the nverse probablstally as a dstrbuton p(x y) of possble RAW measurements x that ould have been tone-mapped to a gven JPEG output y. Whle formulatng ths dstrbuton, we also aount for errors n the estmate J of the renderng funton, treatng them as Gaussan nose wth varane σf, where σ f s set to twe the n-tranng RMSE. Spefally, we defne p(x y) as: ( ) p(x y) = 1 Z p(x) exp y J(x) σf, (7) where Z s the normalzaton fator ( ) Z = p(x ) exp y J(x ) σf dx, (8) and p(x) s a pror on sensor-measurements. Ths pror an range from per-pxel dstrbutons that assert, for example, that broadband refletanes are more lkely than saturated olors; to hgher-order sene-level models that reason about the number of dstnt hromattes and materals n a sene we expet that the hoe of p(x) wll be dfferent for dfferent applatons and envronments. In ths paper, we smply hoose a unform pror over all possble sensor measurements whose hromattes le n the onvex hull of the tranng data. Note that these dstrbutons are omputed assumng that the whte balane multplers are known (and norporated n J). For some ameras, even wth a fxed whte-balane settng, the atual whte-balane multplers mght vary from sene to sene. In these ases, the varable x n the dstrbuton above wll be a lnear transform whh s fxed for all pxels n a gven mage away from a sene-ndependent RAW measurement. Ths may be suffent for applatons that only reason about olors n a sngle mage, or n multple mages of the same sene where the whte balane multplers an reasonably be expeted to reman fxed, but other applatons wll need to address ths ambguty when usng these nverse dstrbutons. Whle the expresson n (7) s the exat form of the nverse dstrbuton orrespondng to a unform dstrbuton over all RAW values x predted by the amera model to map to a gven JPEG value y, wth added slak for albraton error t has no onvenent losed form. Prattoners wll therefore need to ompute them. Note that whte-balane orreton s typally a lnear dagonal transform n the amera s sensor spae. For ameras that are not RAWapable and have been albrated wth a RAW proxy, ths wll be a general lnear transform n the proxy s sensor spae.

7 7 Camera Name TABLE 3 Mean Empral log-lkelhoods under Inverse Models for RAW-apable Cameras Determnst Inverse Prob. Inverse Prob. Inverse Prob. Inverse Prob. Inverse Unform 8k samples Unform 8k samples 1 Exp., Illum. 4 Exp., 4 Illum. 8 Exp., 4 Illum. Panason DMC LX Canon EOS 4D Canon PowerShot G Canon PowerShot S Nkon D expltly over a grd of possble values of x for eah JPEG value y, or approxmate them wth a onvenent parametr form for use n vson applatons. We employ mult-varate Gaussan dstrbutons to approxmate the exat form n (7), as an example to demonstrate the benefts of usng a probablst nverse n the remander of ths paper, but ths s only one possble hoe and the optmal representaton for these dstrbutons wll lkely depend on the target applaton and platform. Formally, we approxmate p(x y) as p(x y) = N (x µ(y), Σ(y)), µ(y) = xp(x y) dx, Σ(y) = (x µ(y)) (x µ(y)) T p(x y) dx. (9) Note that here µ, n addton to beng the mean of the approxmate Gaussan dstrbuton, s also the sngle best estmate of x gven y (n the mnmum least-squares error sense) from the exat dstrbuton n (7). And sne (7) s derved usng a amera model smlar to that of [6], µ an be nterpreted as the determnst RAW estmate that would be yelded by the algorthm n [6]. The ntegratons n (9) are performed numerally, and by storng pre-omputed values of J on a denselysampled grd to speed up dstane omputatons. A MATLAB mplementaton s avalable on our projet page [4], whh takes roughly 15ms to ompute the mean and o-varane above for a sngle JPEG observaton on a modern mahne. Tables 3 and 4 report the mean empral loglkelhoods,.e., the mean value of log p(x y) aross all RAW-JPEG pars (x, y) n the valdaton set, for our set of albrated ameras. For the RAW-apable ameras, we report these numbers for nverse dstrbutons omputed usng estmates of the renderng funton J from dfferent albraton sets as n Table 1. As expeted, better estmates of J usually lead to better estmates of p wth hgher log-lkelhoods, and we fnd that our hoe of albratng usng 8 exposures and 4 llumnants for RAW ameras yelds sores that are lose to those aheved by random samples aross the entre valdaton set. Moreover, to demonstrate the benefts of usng a probablst nverse, we also report log-lkelhood sores from a determnst nverse that outputs sngle predton (µ from (9)) for the RAW value for a gven JPEG. TABLE 4 Mean Empral log-lkelhoods for JPEG-only Cameras Camera Name Determnst Inverse Prob. Inverse Fujflm J Panason DMC LZ Samsung Galaxy S Note that strtly speakng, the log-lkelhood n ths ase would be unless µ s exatly equal to x. The sores reported n Tables 3 and 4 are therefore omputed by usng a Gaussan dstrbuton wth varane equal to the mean predton error (whh s the hoe that yelds the maxmum mean log-lkelhood). We fnd that these sores are muh lower than those from the full model, demonstratng the benefts of a probablst approah. Fnally, we show vsualzatons of the nverse dstrbutons for four of the remanng RAW-apable ameras n our database. These plots represent the dstrbutons p(x y) usng ellpsods to represent mean and ovarane, and an be nterpreted as RAW values that are lkely to be mapped to the same JPEG olor by the amera. We see that these dstrbutons are qualtatvely dfferent for dfferent ameras, sne dfferent manufaturers typally employ ther own strateges for ompressng wde gamut sensor measurements to narrow gamut mages that are vsually pleasng. Moreover, the szes and orentatons of the ovarane matres an also vary sgnfantly for dfferent JPEG values y obtaned from the same amera. 5 VISUAL INFERENCE WITH UNCERTAINTY The probablst derenderng model (9) provdes an opportunty for vson systems to explot the strutured unertanty that s unavodable when nvertng global tone-mappng proesses. To demonstrate how vson systems an beneft from modelng ths unertanty, we ntrodue nferene algorthms that norporate t for a broad, representatve set of vsual tasks: mage fuson, photometr stereo, and deblurrng. 5.1 Image Fuson We begn wth the task of ombnng multple olor observatons of the same sene to nfer aurate estmates of sene olor. Ths task s essental to hgh dynamrange (HDR) magng from exposure-staks of JPEG

8 Canon EOS 4D Nkon D7 Canon PowerShot G9 Canon PowerShot S9 8 Fg. 5. Probablst Inverse. For dfferent ameras, we show ellpsods n RAW spae that denote the mean and ovarane of p(x y) for dfferent JPEG values y ndated by the olor of the ellpsod. These values y are unformly sampled n JPEG spae, and we note that the orrespondng dstrbutons an vary sgnfantly aross ameras. mages n the sprt of Debeve and Malk []; and varatons of t appear when stthng mages together for harmonzed, wde-vew panoramas or other ompostes, and when nferrng objet olor (ntrns mages and olor onstany) or surfae BRDFs from Internet mages. Formally, we onsder the problem of estmatng the lnear olor x of a sene pont from multple JPEG observatons {y } aptured at known exposures {α }. Eah observaton y s assumed to be the rendered verson of sensor value α x, and we assume the amera has been pre-albrated as desrbed prevously. The nave extenson of RAW HDR reonstruton s to use a determnst approah to derender eah JPEG value y, and then ompute sene olor x usng least-squares. Ths strategy onsders every derendered JPEG value to be equally relable and s mplt, for example, n tradtonal HDR algorthms based on self-albraton from non-lnear mages [1] [3], [14] [16]. When the magng system s prealbrated, the determnst approah orresponds to gnorng varane nformaton and omputng a smple, exposure-orreted lnear ombnaton of the derendered means: x = arg mn x X P α µ kµ α xk = P, α (1) where µ = µ(y ). In ontrast to the determnst approah, we propose usng the probablst nverse from Se. 4 to wegh the ontrbuton of eah JPEG observaton based on ts relablty, thereby mprovng estmaton. Estmaton s also mproved by the fat that nverse dstrbutons from dfferent exposures of the same sene olor often arry omplementary nformaton, n the form of dfferently-orented ovarane matres. Spefally, eah observaton provdes us wth a Gaussan dstrbuton p(x y, α ) = N (x µ, Σ ), Σ = Σ(y ) + σz I3 3, α (11) where σz orresponds to the expeted varane of photosensor nose, whh s assumed to be onstant and small relatve to most Σ. The most-lkely estmate of x from

9 9 3 x 14 Determnst Approah Probablst Approah Number of Colors Exposure 1 Exposure Improvement n Error Exposure Exposure Error Fg. 6. HDR Results on the Panason DMC LX3. (Left) Hstogram of mprovement n errors over the determnst baselne for all sene olors usng every possble exposure par. (Rght) Mean errors aross all olors for eah approah when usng dfferent exposure pars. all observatons s then gven by x = arg max x ( = Σ 1 N (x µ, Σ ) ) 1 ( ) Σ 1 µ. (1) An mportant effet that we need to aount for n ths probablst approah s lppng n the photo-sensor. To handle ths, we nsert a hek on the derendered dstrbutons (µ(y ), Σ(y )), and when the estmated mean n any hannel s lose to 1, we update the orrespondng elements of Σ to reflet a very hgh varane for that hannel. The same strategy s also adopted for the baselne determnst approah (1). To expermentally ompare reonstruton qualty of the determnst and probablst approahes, we use all RAW-JPEG olor-pars from the database of olors aptured wth the Panason DMC LX-3, orrespondng to all olor-pars exept those from the four tranng llumnants. We onsder the olor heker under a partular llumnant to be the target HDR sene, and we onsder the dfferently-exposed JPEG mages under that llumnant to be the nput mages of ths sene. The task s to estmate for eah target sene (eah llumnant) the true lnear path olor from only two dfferentlyexposed JPEG mages. The true lnear path olor for eah llumnant s omputed usng RAW data from all exposures, and performane s measured usng relatve RMSE: Error(x, x true ) = x x true. (13) x true Fgure 6 shows a hstogram of the reduton n RMSE values when usng the probablst approah. Ths s the hstogram of dfferenes between evaluatng (13) wth probablst and determnst estmates x aross 168 dstnt lnear sene olors n the dataset and all possble un-ordered pars of exposures 3 as nput, exludng the trval pars for whh α 1 = α (a total of 3888 test ases). In a vast majorty of ases, norporatng derenderng unertanty leads to better performane. We also show n the rght of the fgure, for both the determnst and probablst approahes, two-dmensonal vsualzatons of the error for eah exposure-par. Eah pont n these vsualzatons orresponds to a par of nput exposure values (α 1, α ), and the pseudo-olor depts the mean RMSE aross all 168 lnear sene olors n the test dataset. (Dagonal entres orrespond to estmates from a sngle exposure, and are thus dental for the probablst and determnst approahes). We see that the probablst approah yelds aeptable estmates wth low errors for a larger set of exposure-pars. Moreover, n many ases t leads to lower error than those from ether exposure taken ndvdually, demonstratng that the probablst modelng s not smply seletng the better exposure, but n fat ombnng omplementary nformaton from both observatons. 5. Photometr Stereo Another mportant lass of vson algorthms nlude those that deal wth reoverng sene depth and geometry. These algorthms are espeally dependent on havng aess to radometrally aurate nformaton and have therefore been appled tradtonally to RAW data, but the ablty to relably use tone-mapped JPEG mages, say from the Internet, s useful for applatons lke weather reovery [5], geometr amera albraton [6], and 3D reonstruton va photometr stereo [7]. As an example, we onsder the problem of reoverng shape usng photometr stereo from JPEG mages. Photometr stereo s a tehnque for estmatng the surfae normals of a Lambertan objet by observng that objet under dfferent lghtng ondtons and a fxed 3. These orrespond to the dfferent exposure tme stops avalable on the amera: [5e 4, 6.5e 4, 1e 3, 1.5e 3, e 3,.5e 3, 3.13e 3, 5e 3, 6.5e 3, 1e, 1.6e, 1.67e, e,.5e, 3.33e, 4e, 5e, 6.67e, 1e 1, e 1, 4e 1, 1] n relatve tme unts.

10 1 vewpont [8]. Formally, gven mages under N dfferent dretonal lghtng ondtons, wth l R 3 beng the dreton and strength of the th soure, let I R denote the lnear ntensty reorded n a sngle hannel at a partular pxel under the th lght dreton. If ν S and ρ R + are the normal dreton and albedo of the surfae path at the bak-projeton of ths pxel, then the Lambertan refletane model provdes the relaton ρ l, ν = I. The goal of photometr stereo s to nfer the materal ρ and shape ν gven the set {l, I }. Defnng a pseudo-normal b ρν, the relaton between the observed ntensty and the sene parameters beomes l T b = I. (14) Gven three or more {l, I }-pars, the pseudo-normal b s estmated smply usng least-squares as: b = (L T L) 1 L T I, (15) where L R N 3 and I R N are formed by stakng the lght dretons l T and measurements I respetvely. The normal ν an then smply be reovered as ν = b/ b. When dealng wth a lnear olor mage, Barsky and Petrou [8] suggest onstrutng the observatons I as a lnear ombnaton I = T x of the dfferent hannels of the olor vetors x. The oeffents R 3 are hosen to maxmze the magntude of the ntensty vetor I, and therefore the stablty of the fnal normal estmate µ, as = arg max I = arg max T x, ) = arg max T ( x x T, s.t. = 1. (16) The optmal s then smply the egenvetor assoated wth the largest egenvalue of the matrx ( x x T ). Intutvely, ths orresponds to the normalzed olor of the materal at that pxel loaton. In order to use photometr stereo to reover sene depth from JPEG mages, we need to frst obtan estmates of the lnear sene olor measurements x from the avalable JPEG values y. Rather than apply the above algorthm as-s to determnst nverse-mapped estmates of x, we propose a new algorthm that uses the dstrbutons p(x y ) = N (x µ, Σ ) derved n Se. 4. Frst, we modfy the approah n [8] to estmate the oeffent vetor by maxmzng the sgnal-to-nose rato (SNR), rather than smply the magntude, of I: = arg max = arg max E[I ] Var(I ) T ( µ µ T = arg max + Σ ) T ( Σ ) E[(T x ) ] Var[T x ] s.t. = 1. (17) It s easy to show that the optmal value of for ths ase s gven by the egenvetor assoated wth the largest egenvalue of the matrx ( Σ ) 1 ( µ ) µ T. Ths hoe of essentally mnmzes the relatve unertanty n the set of observatons I = T x, whh are now desrbed by unvarate Gaussan dstrbutons: I N (m, σ ) = N ( T µ, T Σ ). (18) From ths t follows (eg., [9]) that the maxmum lkelhood estmate of the pseudo-normal b s obtaned through weghted least-squares, wth weghts gven by the reproal of the varane,.e., b = (L T W L) 1 L T W m, (19) where m R N s onstruted by stakng the means m, and W = dag{σ } N =1. We evaluate our algorthm on JPEG mages of a fgurne aptured usng the Canon EOS 4D from a fxed vewpont under dretonal lghtng from ten dfferent known dretons. At eah pxel, we dsard the brghtest and darkest measurements to avod possble speular hghlghts and shadows, and use the rest to estmate the surfae normal. The amera takes RAW mages smultaneously, whh are used to reover surfae normals that we treat as ground truth. Fgure 7 shows the angular error map for normal estmates usng the proposed method, as well as the determnst baselne. We also show the orrespondng depth maps obtaned from the normal estmates usng [3]. The proposed probablst approah produes smaller normal estmate errors and fewer reonstruton artfats than the determnst algorthm quanttatvely, the mean angular error s 4.34 for the probablst approah, and 6.46 for the determnst baselne. We also ran the reonstruton algorthm on nverse estmates omputed by smple gamma-orreton on the JPEG values (a gamma parameter of. s assumed). These estmates had a muh hgher mean error Deonvoluton Deblurrng s a ommon mage restoraton applaton and has been an atve area of researh n omputer vson [31] [35]. Tradtonal deblurrng algorthms are desgned to work on lnear RAW mages as nput, but n most pratal settngs, only amera rendered JPEG mages are avalable. The standard prate n suh ases has been to apply an nverse tone-map assumng a smple gamma orreton of., but as has been reently demonstrated [36], ths approah s nadequate and wll often yeld poor qualty mages wth vsble artfats due to the fat that deblurrng algorthms rely heavly on lnearty of the nput mage values. Whle Km et al. [36] show that more aurate nverse maps an mprove deblurrng performane, ther maps are stll determnst. In ths seton, we explore the benefts of usng a probablst nverse, and ntrodue a modfed deblurrng algorthm that aounts for varyng degrees of unertanty n estmates of RAW values from pxel to pxel. Formally, gven a blurry JPEG mage y(n), we assume that the orrespondng blurry RAW mage x(n) s related

11 x Sample JPEG Image Normal Error Determnst Normal Error Probablst Ground Truth Depth (from RAW) Estmated Depth Determnst Estmated Depth Probablst Improvement n normal error Fg. 7. Photometr Stereo Results usng the Canon EOS 4D. (Left) Hstogram of the mprovement n angular error of normal estmate. (Rght) One of the JPEG mages used durng estmaton, and angular error (n degrees) for the normals estmated usng the determnst and probablst approahes, along wth the orrespondng depth maps. to a latent sharp RAW mage z(n) of the sene as x(n) = (k z)(n) + (n), () where k s the blur kernel and (n) s addtve whte Gaussan nose. The operator denotes onvoluton of the 3-hannel mage z wth a sngle-hannel kernel k, mplemented as the onvoluton of the kernel wth eah mage hannel separately. Although () assumes onvoluton wth a spatally-unform blur kernel, the approah n ths seton an be easly generalzed to aount for non-unform blur (eg., as n [35]). Deblurrng an mage nvolves estmatng the blur kernel k(n) atng on the mage, and then nvertng ths blur to reover the sharp mage z(n). In ths seton, we wll onentrate on ths seond step,.e., deonvoluton, assumng that the kernel k has already been estmated say by applyng the determnst nverse and usng a standard kernel estmaton algorthm suh as [31]4. We begn wth a modern lnear-mage deonvoluton algorthm [9] and adapt t to explot the nverse probablty dstrbutons from Se. 4. Gven an observed lnear blurred mage x and known kernel k, Krshnan and Fergus [9] provde a fast algorthm to estmate the latent sharp mage z(n) by mnmzng the ost funton X λx C(z) = k(k z)(n) x(n)k + k( z)(n)kγ, (1) n n, where { } are gradent flters (horzontal and vertal fnte dfferene flters n both [9] and our mplementaton), and the exponent γ s 1. The frst term measures 4. Emprally, we fnd that usng a determnst nverse suffes for the kernel estmaton step, as t nvolves poolng nformaton from the entre mage to estmate a relatvely small number of parameters. the agreement of z wth the lnear observaton whle the seond term mposes a sparse pror on gradents n a sharp mage. The salar weght λ ontrols the relatve ontrbuton of the two. Gven the tone-mapped verson y(n) of the blurry lnear mage x(n), the determnst approah would be to smply replae x(n) wth ts expeted value µ(y(n)) n the ost funton above. However, to aount for the strutured unertanty n our estmate of x(n) and the fat that some values of y(n) are more relable than others, we modfy the ost funton to norporate both the derendered means µ and o-varanes Σ: X T T =λ C(z) [(k z)(n) µn ] Σ 1 n [(k z)(n) µn ] n X + k( z)(n)kγ, () n, where µn = µ(y(n)), Σn = Σ(y(n)) + σz I3 3, and σz s the expeted varane of photo-sensor nose. The algorthm n [9] mnmzes the orgnal ost funton (1) usng an optmzaton strategy known as halfquadrat splttng. It ntrodues a new set of auxlary varables w (n) orrespondng to the gradents of the latent mage z(n), and arres out the followng mnmzatons suessvely: λx z = arg mn k(k z)(n) x(n)k z n βx + k( z)(n) w (n)k, (3) n, βx w (n) = arg mn k( z)(n) wk + kwkγ, (4) w n,

12 1 where β s a salar parameter that s nreased aross teratons. To mnmze our modfed ost funton n (), we need only hange (3) to z = arg mn z λ n [(k z)(n) µ n ] T Σ 1 n [(k z)(n) µ n ] + β ( z)(n) w (n). (5) n, A omplaton arses from ths hange: whle the orgnal expresson (3) an be omputed n losed-form n the Fourer doman, the same s not true for the modfed verson (5) beause the frst term has spatallyvaryng weghts. Thus, to ompute (5) effently, we use a seond level of teratons based on varable-splttng to ompute (5) n every outer teraton of the orgnal algorthm. Spefally, we ntrodue a new ost-funton wth new auxlary varables u(n): z = arg mn mn z u λ n [u(n) µ n ] T Σ 1 n [u(n) µ n ] + α (k z)(n) u(n) + β ( z)(n) w (n), (6) n, where α s a salar varable whose value s nreased aross teratons. Note that for α, the expressons n (5) and (6) are equvalent. The mnmzaton algorthm proeeds as follows: we begn by settng u(n) = µ n and then onsder nreasng values of α equally spaed n the log doman (n our mplementaton, we onsder eght values that go from 4λ tmes the mnmum to the ). For eah value of α, we perform the followng updates to z and u(n) n sequene: maxmum of all dagonal entres of all Σ 1 n z = arg mn z u(n) = arg mn u α (k z)(n) u(n) + β ( z)(n) w (n), (7) λ n n, [u µ n ] T Σ 1 n [u µ n ] + α (k z)(n) u. (8) Note that (7) has the same form as the orgnal (3) and an be omputed n losed-form n the Fourer doman. The updates to u(n) n (8) an also be omputed n losed-form ndependently for eah pxel loaton n. We evaluate the proposed algorthm usng three RAW mages from a publ database [37], [38] and eght (spatally-unform) amera-shake blur kernels from the database n [34]. We generate a set of twenty-four blurred JPEG observatons by onvolvng eah RAW mage wth eah blur kernel, addng Gaussan nose, and then applyng the estmated forward map of the Canon EOS-4D amera. 5 We ompare deonvoluton performane of the proposed approah to a determnst baselne onsstng of the algorthm n [9] appled to the derendered means µ(n). The error s measured n terms of PSNR values between the true and estmated JPEG versons of the latent sharp mage. Sne these errors depend on the hoe of regularzaton parameter λ (whh n prate s often set by hand), we perform a grd searh to hoose λ separately for eah of the determnst and probablst approahes and for every observed mage, seletng the value eah tme that gves the lowest RMSE usng the known ground-truth sharp mage. Ths measures the best performane possble wth eah approah. We set the exponent value γ to /3 as suggested n [9]. Fgure 8 shows a hstogram of the mprovement n PSNR aross the dfferent mages when usng the probablst approah over the determnst one. The probablst approah leads to hgher PSNR values for all mages, wth a medan mprovement of 1.4 db. Fgure 8 also nludes an example of deonvoluton results from the two approahes, and we see that n omparson to the probablst approah, the determnst algorthm yelds over-smoothed results n some regons whle produng rngng artfats n others. Ths s beause the sngle salar weght λ s unable to adapt to the varyng levels of nose or radometr unertanty n the derendered estmates. The rngng artfats n the determnst algorthm output orrespond to regons of hgh unertanty, where the probablst approah ) for the frst term of () and smooths out the artfats by relyng more on the pror (.e., the seond term). At the same tme, t yelds sharper estmates n regons wth more relable observatons by usng a hgher weght for the fdelty term, thus redung the effet of the smoothness pror. orretly employs a lower weght (.e., Σ 1 n 6 CONCLUSION Tradtonally, omputer vson algorthms that requre aurate lnear measurements of spetral radane have been lmted to RAW nput, and therefore to tranng and testng on small, spealzed datasets. In ths work, we present a framework that enables these methods to be extended to operate, as effetvely as possble, on tone-mapped nput nstead. Our framework s based on norporatng a prese model of the unertanty assoated wth global tone-mappng proesses, and t makes t possble for omputer vson systems to take better advantage of the vast number of tone-mapped mages produed by onsumer ameras and shared onlne. To a vson engneer, our model of tone-mappng unertanty s smply a form of sgnal-dependent Gaus- 5. Note that for generatng the synthetally blurred and groundtruth sharp mages, we use the forward map as estmated usng the unformly sampled 8k tranng set, whh as seen n Table 1 nearly perfetly predts the amera map. Durng deonvoluton, we use nverse and forward maps estmated usng only the smaller 8 exposures, 4 llumnants set.

13 13 Fg. 8. Deonvoluton Results. (Left) Hstogram of the mprovement n PSNR aross the twenty-four synthetally blurred mages, when usng a probablst approah nstead of a determnst one. (Rght) Example deonvoluton results usng dfferent approahes. san nose, and ths makes t oneptually appealng for nluson n subsequent vsual proessng. To prove ths pont, we ntrodued new, probablst adaptatons of three lassal nferene tasks: mage fuson, photometr stereo and deblurrng. In all of these ases, an explt haraterzaton of the ambguty due to tone-mappng allows the omputer vson algorthm to surpass the performane possble wth a purely determnst approah. In future work, the use of nverse RAW dstrbutons should be norporated n other vson algorthms, suh as depth from stereo, struture from moton, and objet reognton. Ths may requre explorng approxmatons for the nverse dstrbuton dfferent from the Gaussan approxmaton n (9). Whle some applatons mght requre more omplex parametr forms, others may beneft from smpler weghtng shemes that are derved based on the analyss n Se. 4. Also, t wll be benefal to fnd ways of mprovng albraton for JPEG-only ameras. Our hope s that eventually our framework wll enable a ommon repostory of tone-map models (or probablst nverse models) for eah magng mode of eah amera, makng the totalty of Internet photos more usable by modern omputer vson algorthms. [3] [4] [5] [6] [7] [8] [9] [1] [11] [1] [13] [14] [15] [16] [17] ACKNOWLEDGMENTS [18] The authors would lke to thank the assoate edtor and revewers for ther omments. Ths materal s based on work supported by the Natonal Sene Foundaton under Grants no. IIS-9543, IIS-95647, IIS-11347, IIS-11798, IIS-1198, IIS , and IIS-13715; by DARPA under the Mnd s Eye and MSEE programs; and by Toyota. [19] [] [1] [] R EFERENCES [1] [] T. Mtsunaga and S. Nayar, Radometr self albraton, n Pro. CVPR, P. Debeve and J. Malk, Reoverng hgh dynam range radane maps from photographs, n SIGGRAPH, [3] [4] [5] S. Mann and R. Pard, Beng undgtal wth dgtal ameras: Extendng dynam range by ombnng dfferently exposed ptures, n Pro. IS&T Annual Conf., M. D. Grossberg and S. K. Nayar, Modelng the spae of amera response funtons, PAMI, 4. A. Chakrabart, D. Sharsten, and T. Zkler, An empral amera model for nternet olor vson, n Pro. BMVC, 9. S. Km, H. Ln, Z. Lu, S. Süsstrunk, S. Ln, and M. Brown, A new n-amera magng model for olor omputer vson and ts applaton, PAMI, 1. J.-Y. Lee, Y. Matsushta, B. Sh, I. S. Kweon, and K. Ikeuh, Radometr albraton by rank mnmzaton, PAMI, 13. R. Woodham, Photometr method for determnng surfae orentaton from multple mages, Optal engneerng, 198. D. Krshnan and R. Fergus, Fast mage deonvoluton usng Hyper-Laplaan prors, n NIPS, 9. S. Ln, J. Gu, S. Yamazak, and H.-Y. Shum, Radometr albraton from a sngle mage, n Pro. CVPR, 4. Y. Ta, X. Chen, S. Km, F. L, J. Yang, J. Yu, Y. Matsushta, and M. Brown, Nonlnear amera response funtons and mage deblurrng: Theoretal analyss and prate, PAMI, 13. H. Fard, Blnd nverse gamma orreton, IEEE Trans. Imag. Pro.,. S. Kuthrummal, A. Agarwala, D. Goldman, and S. Nayar, Prors for large photo olletons and what they reveal about ameras, n Pro. ECCV, 8. M. Grossberg and S. Nayar, Determnng the amera response from mages: what s knowable? PAMI, 3. E. Renhard, G. Ward, S. Pattanak, and P. Debeve, Hgh dynam range magng. Elsever, 6. B. Sh, Y. Matsushta, Y. We, C. Xu, and P. Tan, Self-albratng photometr stereo, n Pro. CVPR, 1. X. Chen, F. L, J. Yang, and J. Yu, A theoretal analyss of amera response funtons n mage deblurrng, Pro. ECCV, 1. C. Pal, R. Szelsk, M. Uyttendaele, and N. Joj, Probablty models for hgh dynam range magng, n Pro. CVPR, 4. M. Grundmann, C. MClanahan, S. B. Kang, and I. Essa, Postproessng approah for radometr self-albraton of vdeo, n Pro. ICCP, 13. Y. Xong, K. Saenko, T. Darrell, and T. Zkler, From pxels to physs: Probablst olor de-renderng, n Pro. CVPR, 1. C.-C. Chang and C.-J. Ln, LIBSVM: A lbrary for support vetor mahnes, ACM Trans. Intellgent Systems and Tehnology, 11, (software avalable at lbsvm). J. Holm, Capture olor analyss gamuts, n IS&T/SID Color and Imagng Conferene, 6. J. Jang, D. Lu, J. Gu, and S. Süsstrunk, What s the spae of spetral senstvty funtons for dgtal olor ameras? n IEEE Workshop on the Applatons of Computer Vson (WACV), 13. [Onlne]. Avalable: L. Shen and P. Tan, Photometr stereo and weather estmaton usng nternet mages, n Pro. CVPR, 9.