Parameter Estimation for MRF Stereo

Transcription

1 Parameter Estimation for MRF Stereo Li Zhan Steven M. Seitz University of Washinton Abstract This paper presents a novel approach for estimatin parameters for MRF-based stereo alorithms. This approach is based on a new formulation of stereo as a maximum a posterior (MAP) problem, in which both a disparity map and MRF parameters are estimated from the stereo pair itself. We present an iterative alorithm for the MAP estimation that alternates between estimatin the parameters while fixin the disparity map and estimatin the disparity map while fixin the parameters. The estimated parameters include robust truncation thresholds, for both data and neihborhood terms, as well as a reularization weiht. The reularization weiht can be either a constant for the whole imae, or spatially-varyin, dependin on local intensity radients. In the latter case, the weihts for intensity radients are also estimated. Experiments indicate that our approach, as a wrapper for existin stereo alorithms, moves a baseline belief propaation stereo alorithm up six slots in the Middlebury rankins.. Introduction Stereo matchin has been one of the core challenes in computer vision for decades. Many of the current bestperformin techniques are based on Markov Random Field (MRF) formulations [8] that balance a data matchin term with a reularization term and are solved usin Graph Cuts [2, ] or Belief Propaation [6, 4]. Virtually all of these techniques require users to properly set hard-coded parameters, e.., reularization weiht, by trial and error on a set of imaes. In this paper, we arue that different stereo pairs require different parameter settins for optimal performance, and we seek an automated method to estimate those parameters for each pair of imaes. To see the effect of parameter settin on stereo matchin, we estimated disparity maps, D = {d i }, for and Map imae pairs [5] by minimizin the followin enery U(d i ) + λ V (d i, d j ) () i I (i,j) G where I is the set of pixels, G is the set of raph edes connectin adjacent pixels, U measures similarity between matchin pixels, and V is a reularization term that encouraes neihborin pixels to have similar disparities. We minimize Eq. () usin an existin MRF solver [4] and plot the Disparity error percentae Reularization weiht 50 Disparity error percentae Map Reularization weiht Fiure. Some stereo pairs require more reularization than others, as shown in the above raphs that plot error as a function of reularization weiht λ. The parameters shown above (dotted vertical lines) were computed automatically usin our alorithm. error rate of the disparity estimation versus round truth as a function of λ, as shown in Fiure. The fiure shows that, for the same alorithm, the optimal reularization weiht λ varies across different stereo pairs. As shown later in the paper, λ and other MRF parameters, e.., robust truncation thresholds, are related to the statistics of imae noise and variation of scene structures, and can all be estimated from a sinle stereo pair. Furthermore, we also show that neihborin pixel intensity difference [2] can be conveniently incorporated into our formulation to encourae the disparity discontinuities to be alined with intensity edes, and the relevant parameters can be estimated automatically. To estimate the MRF parameters, we interpret them usin a probabilistic model that reformulates stereo matchin as a maximum a posterior (MAP) problem for both the disparity map and the MRF parameters. Under this formulation, we develop an alternatin optimization alorithm that computes both the disparity map and the parameters. Our approach serves as a wrapper for existin MRF stereo matchin alorithms that solves for the optimal parameters for each imae pair. Our routine uses the output of the stereo matcher to update the parameter values, which are in turn fed back into the stereo matchin procedure it can interface with many stereo implementations without modification. Therefore, we emphasize that the oal of this paper is

2 not a specific stereo alorithm that performs better than existin alorithms. Rather, we introduce a methodoloy that boosts the performance of MRF-based stereo alorithms. Althouh we use a baseline stereo alorithm based on belief propaation, any MRF-based stereo alorithm could be used instead (e.., raph cuts, dynamic prorammin). The rest of the paper is oranized as follows. After reviewin related work in Section 2, we first ive the intuition for our parameter estimation technique in Section 3. We then formulate the idea as an MAP problem in Section 4, propose an optimization alorithm in Section 5, and extend it to estimate the weihts that depend on intensity radients in Section 6. Finally, we show experimental results in Section 7, and discuss future research directions in Section Previous work An early stereo method that requires no parameter settin is the adaptive window method of Kanade and Okutomi [0], which requires proper initialization for ood performance. The only prior work that addressed the problem of computin MRF parameters (aka hyper-parameters) for stereo matchin is by Chen and Caelli [3]. While their approach is an important first step, they relied on a restricted MRF model from the imae restoration literature [9], and did not support key features of the leadin stereo alorithms, e.., occlusion modelin and radient-dependent reularization. (Other MRF models in the imae restoration literature, e.., [2, 9, 4], also have this limitation when applied to stereo matchin.) In contrast, we desined our approach to support these features in order to interface with many of the leadin stereo alorithms our approach operates as an auxiliary routine that does not require modifyin existin stereo code. Towards this end, we show that the truncated absolute distance commonly used in leadin stereo alorithms [2, 4] corresponds to a mixture of an exponential distribution and an outlier process. We use hidden variables to model occlusions and other outliers, and apply expectation maximization (EM) to infer the hidden variables and estimate the mixture models. Because we use EM instead of MCMC, our approach is also simpler and more efficient compared to [3]. Finally, we benchmark our approach on the Middlebury database, and show that it dramatically improves the performance of a leadin alorithm with the recommended hand-tuned parameters (as opposed to showin improvement over randomly-chosen parameters [3]). In this work, we use insihts from statistical learnin to improve vision alorithms. Our work is therefore related to Freeman et al. [6] who formulate super-resolution as MRF inference based on trainin imaes, and apply belief propaation to obtain ood results. Similarly, with trainin imaes, Freeman and Torralba [5] infer 3D scene structure from a sinle imae. Unlike Freeman et al. s approach, our method doesn t require trainin imaes MRF parameters are estimated from the stereo pair itself. 3 Intuition In this section, we describe our basic idea for parameter estimation for MRF-based stereo. In the enery function in Eq. (), U measures similarity between matchin pixels, and V encouraes neihborin pixels to have similar disparities. Many functional forms have been proposed for U and V, includin squared differences, absolute differences, and many other robust metrics [5]. In this paper, we focus on truncated absolute difference because it is a popular choice of top performin stereo alorithms [2, 4] and it has several ood properties. First, it is derived from total variation [3], thus preservin discontinuities. Second, it does not have frontal parallel bias and satisfies the metric property required by the α-expansion alorithm in raph cut [2]. Third, it can be efficiently computed via distance transform [4] in belief propaation. Specifically, U(d i ) = min( I(x i, y i ) J(x i d i, y i ), σ) V (d i, d j ) = min( d i d j, τ) (2) where I and J are the imae pairs, and σ and τ are truncation thresholds. To best set the parameters σ, τ, and λ for a stereo pair, we need to know how well the correspondin pixels in two imaes can be matched and how similar the neihborin disparities are, in a statistical sense. However, without knowin the disparity map, those two questions can not be answered. This dilemma explains why existin MRF-based stereo alorithms require users to set parameters manually. To resolve this dilemma, let s first consider the case in which we know the disparity maps. In Fiure 2(a,c), usin the round truth disparities from the Middlebury website [5], we plot the historams of pixel matchin errors and neihborin disparity differences for the stereo pair. In Fiure 2(b,d), we show the same historams in lo-scale. Since the lo-scale historams are not straiht lines or quadratic curves, it means that the probability of pixel matchin errors and that of neihborin disparity differences are not simple exponential or Gaussian distributions. The heavy tail in the matchin error historam is due to occlusion and violation of brihtness constancy; the heavy tail in the neihborin disparity difference historam is due to disparity discontinuities. Those historams can be approximated by two sements, a mixture of an exponential distribution and a uniform outlier process. Fiure 2(a,c) shows the probability distribution of fitted mixture models overlaid on the historams. The fit is quite accurate: the errors are around 0 3, only noticeable in the lo-scale raphs in Fiure 2(b,d). From the shapes of the fitted distributions, we can recover the optimal set of MRF parameters, as we describe later in the paper.

3 Normalized count Error historam Distribution of the mixture model Absolute pixel matchin error Normalized count Error historam Distribution of the mixture model Absolute pixel matchin error Normalized count Difference historam Distribution of the mixture model Absolute neihborin disparity difference Normalized count Difference historam Distribution of the mixture model Absolute neihborin disparity difference (a) (b) (c) (d) Fiure 2. Historams of errors between correspondin pixels in two imaes in linear (a) and lo (b) scale. Superimposed on the plots is the fitted mixture model. (c) and (d) show historams and models for neihborin disparity difference. In practice, however, round truth disparities are unknown and we propose an iterative alorithm that alternates between estimatin MRF parameters from the current historams and estimatin disparities usin the current MRF parameters. The alorithm iterates until the estimated disparity map yields historams that aree with the MRF parameters or a fixed number of iterations is reached. In the next section, we present the details of this method by castin the problem in a probabilistic framework. 4 A probabilistic mixture model for stereo In this section, we present the mixture models for the historams of pixel matchin errors and neihborin disparity differences, and formulate stereo matchin probabilistically, based on those mixture models. 4. Matchin likelihood Given an imae pair I and J and the disparity map D, we define the mixture model for pixel matchin error as follows. We assin each pixel i in I a hidden binary random variable γ i, indicatin whether the correspondin scene point is visible in J. Let e(d i ) = I(x i, y i ) J(x i d i, y i ). We define the mixture model for e(d i ) as { ζe µ e(d i), γ P (e(d i ) d i, γ i ) = i =. N, γ (3) i = 0. where µ is the decay rate for the exponential distribution, e(d i ) takes discrete values, {0,,, N }, and ζ = exp( µ) exp( µn) is a normalization factor. We define the mixture probability P (γ i = ) = α (4) where α is the percent of pixels in I that are also visible in J. Summin over γ i ives the marinal matchin likelihood P (e(d i ) d i ) = αζe µ e(di) + ( α) N For brevity, we refer to γ i as a visibility variable, but it can also account for differences in brihtness, e.., due to specularity. (5) 4.2 Disparity prior Define d = d i d j to be the disparity difference on the raph ede connectin adjacent pixels i and j. Similarly as for pixel matchin probability, we assin each ede a binary random variable θ, indicatin whether the ede is continuous. We define the mixture model for d as { ηe ν d, θ P ( d θ ) = =. L, θ (6) = 0. where ν is the decay rate, d {0,,, L }, and η = exp( ν) exp( νl). We define the mixture probability P (θ = ) = β (7) where β is the percent of continuous edes in I. marinal distribution is P ( d ) = βηe ν d + ( β) L 4.3 Stereo as a MAP problem The Now we formulate stereo matchin as a MAP problem based on the two defined mixture distributions. Given an imae pair, I and J, our probabilistic model consists of a disparity field D = {d i } over I and two sets of random variables Γ = {γ i } and Θ = {θ } for pixel visibility and ede connectivity, respectively. 2 We seek to estimate D, α, µ, β, and ν, iven I and J by maximizin P (I,J,D α,µ,β,ν)p (α,µ,β,ν) P (I,J) P (D, α, µ, β, ν I, J) = P (I, J, D α, µ, β, ν) (9) where the prior on (α, µ, β, ν) is assumed to be uniform. We can factor P (I, J, D α, µ, β, ν) as (8) P (I, J, D α, µ, β, ν) = P (I, J D, α, µ)p (D β, ν) (0) and compute P (I, J D, α, µ) and P (D β, ν) by marinalizin over visibility variables Γ and continuity variables Θ, respectively, as follows. P (I, J D, α, µ) = i P (e(d i ) d i, α, µ) () 2 This model is called three coupled MRF s in [6].

4 (a) Fiure 3. (a) A raph for Eq. (8) for L = 6. (b) An illustration of Eq. (24) (solid curve) as an upper bound for Eq. (23) (dotted curve) for (a, b, c) = (, 2, ). which assumes P (Γ D, α) = P (Γ α), inorin the dependence of visibility on eometry for computational convenience. Similarly, P (D β, ν) = (b) P ( d β, ν) (2) which assumes the independence between d also for computational convenience. Puttin Eqs. (,2) toether, we obtain i P (D, α, µ, β, ν I, J) P (e(d i ) d i, α, µ) P ( d β, ν) } {{ } P (I,J D,α,µ) } {{ } P (D β,ν) (3) Given disparity map D, we can estimate α and µ by maximizin the marinal data likelihood P (I, J D, α, µ), and we can estimate β and ν by maximizin the marinal prior distribution P (D β, ν). Also, we can estimate D by maximizin the likelihood and the prior jointly. Next, we propose an alternatin optimization alorithm for this maximization and relate the probabilistic model parameters (α, µ, β, ν) to (σ, τ, λ) in Eqs. (,2). 5 An alternatin optimization In this section, we present an alternatin alorithm to maximize Eq. (3). Given D, we apply the EM alorithm to estimate (α, µ) by maximizin P (I, J D, α, µ) and estimate (β, ν) by maximizin P (D β, ν). From (α, µ, β, ν), we then compute the optimal MRF parameters (σ, τ, λ). 5. Estimatin β and ν iven D The EM alorithm is well suited for estimatin parameters for mixture models. Given D, we first compute L = max { d } + (4) Then we compute the conditional probability of θ as ω def = P (θ = d, β, ν) = βηe ν d βηe ν d + β L (5) Finally, we estimate β and ν by maximizin the expected lo-probability E θ [lo P ( d, θ β, ν)], computed as ω lo P ( d, θ = β, ν) + ( ω ) lo P ( d, θ = 0 β, ν) = ω (lo(βη) ν d ) + ( ω ) lo β L (6) By maximizin Eq. (6), we obtain the followin relation β = ω (7) G where G is the number of edes in G, and ν is the solution of the equation ω d e ν L e Lν = (8) ω L e Lν Let f(ν; L) = e ν be the left hand side of Eq. (8). f monotonically decreases from L 2 to 0 over [0, ), as shown in Fiure 3(a). When L is lare, the second term in f(ν; L),, is neliible, and the equation L e Lν has a close-form solution ν 0 = lo( y + ), where y is the riht hand side of Eq. (8). When L is small, we start from ν = ν 0 and refine ν usin the Newton-Raphson method. 5.2 Estimatin α and µ iven D The EM alorithm can also be used to estimate α and µ. Given D, we first compute N = max{e(d i )} + (9) i Then we can estimate α and µ in the same way as we estimate β and ν usin Eqs.(5,7,8) with the followin variable replacement: (β, ν, η, L,, G, d ) (α, µ, ζ, N, i, I, e(d i ), ) (20) 5.3 Estimatin D iven α, µ, β, and ν In this section, we describe how the estimated values of α, µ, β, and ν are used for stereo matchin. Given α, µ, β, and ν, we wish to maximize Eq. (3) by minimizin where Ψ def = lo P (D, α, µ, β, ν I, J) = ρ d (d i ; α, µ) + ρ p ( d ; β, ν) (2) i ρ d (d i ; α, µ) = lo(αζe µ e(di) + α N ) ρ p (d i ; β, ν) = lo(βηe ν d + β L ) (22) Eq. (2) can be minimized directly usin existin techniques. For example, Sun et al. [6] use Belief Propaation to minimize a form of Eq. (2).

5 5.3. From mixture model to reularized enery Althouh Eq. (2) can be optimized directly, it is not in a form that efficient MRF solvers [2, 4] assume. Recall that our objective is to interface with and boost the performance of existin stereo alorithms, and we therefore want to convert Eq. (2) to the form of Eqs. (,2). We notice that a function of the form h(x; a, b, c) = lo(a exp( b x ) + c) (23) is tihtly upper-bounded by h(x; r, s, t) = min(s x, t) + r (24) where s = ab a+c a+c, t = lo( c ), and r = lo(a + c), as shown in Fiure 3(b). Therefore, minimizin Eq. (2) can also be approximately achieved by minimizin Ψ = i where s d = s p = min(s d e(d i ), t d ) + αζµ αζ+( α) N βην βη+( β) L C = I lo(αζ + α N min(s p d, t p ) C t d = lo( + αζn α ) (25) t p = lo( + βηl β ) (26) β ) + G lo(βη + L ) To further simplify the problem, we let σ = t d sd, τ = tp s p, and λ = sp s d, and define Ψ = i min( e(d i ), σ) + λ min( d, τ) (27) Ψ differs from Ψ by an affine transform, which does not affect the estimation of D. Eq. (27) is the objective function used in [2, 4] for stereo matchin. Now, we summarize our alorithm as follows. Initialize (α, µ, N, β, ν, L), and iterate Compute s d, t d, s p, and t p usin Eq. (26) Set σ = t d sd, τ = tp s p, and λ = sp s d Compute D by STEREO-MATCHING with Eq. (27) Update L,β, and ν by iteratin EM Eqs. (4,5,7,8) Update N,α, and µ by iteratin EM Eqs. (9,5,7,8) with the variable replacement defined in Eq. (20) Until converence or a fixed number of iterations. We typically start with α = β = 0.5, µ = ν =.0, N = 255, and set L be the maximum disparity plus, althouh robust converence is observed with various initial values, as shown in Section 7. STEREO-MATCHING could be any stereo alorithm that works with Eq. (27). 6 Intensity radient cues Recent stereo alorithms use static cues, such as color sements [7, 6] and color edes [7, ], to improve performance. Here, we show that neihborin pixel intensity difference [2] can be conveniently incorporated into our formulation to encourae the disparity discontinuities to be alined with intensity edes, and the relevant weihtin parameters can be estimated automatically. Define I to be the intensity difference between the two pixels connected by a raph ede. To relate I to the continuity of the disparity map, we treat I as a random variable and define a correspondin mixture distribution. We require the mixture distribution of I to share the same hidden variable θ of d. Specifically, { ξe κ I, θ P ( I θ ) = =. K, θ (28) = 0. where κ is the decay rate, I {0,,, K }, and ξ = exp( κ) exp( κk). This model has the followin property: if a raph ede is continuous, both the color and the disparity differences are encouraed to be small; if a raph ede is discontinuous, the color and disparity differences are unconstrained. The correspondin marinal distribution is P ( I, d ) = βξηe (κ I +ν d ) +( β) KL (29) Given I, I, and J, our oal is to recover D, α, µ, β, ν, and κ, by maximizin i P (D, α, µ, β, κ, ν I, I, J) P (e(d i ) d i, α, µ) P ( I, d β, κ, ν) } {{ } P (I,J D,α,µ) } {{ } P ( I,D β,κ,ν) (30) The alternatin alorithm in Section 5 can still be applied with minor a chane. The estimation of α and µ is the same as before. The estimation of β, κ, and ν can be done as follows. Initially, we set K = max { I } +. For each iteration, we first update L as in Eq. (4). Then we compute the condition probability of θ as ω def = βξηe (κ I +ν d ) βξηe (κ I +ν d ) + β KL (3) Finally, we update β and ν usin Eqs. (7) and (8), respectively, and update κ also usin Eq. (8) with the followin variable replacement (ν, L, d ) (κ, K, I ). After estimatin (α, µ, β, ν, κ), we estimate D by minimizin Eq. (2) with ρ p dependin on I. Specifically, ρ p (d i ; I, β, κ, ν) = lo(βξηe κ I e ν d + β KL ) (32) Accordinly, s p and t p also depend on I. s p = βξηνe κ I βξηe κ I + β KL t p = lo( + βξηkle κ I β ) (33) In Eq. (33), s p approaches 0 in proportion to e κ I when I is lare; s p approaches when I is near 0. ν + β KLβξη Therefore, the reularization weiht λ = sp s d varies over the imae: lare in uniform areas and small across color edes.

6 7. Results We implemented the EM alorithm to estimate the MRF parameters, and implemented belief propaation (BP) usin distance transform [4] as our baseline stereo matcher. In all experiments, we alternated between EM and BP six times. In each alternation, BP was executed for 60 iterations and each iteration takes about second. The cost for EM is neliible and the total run-time is about 6 minutes. 7. Converence In our first experiment, we tested our alorithm on the four Middlebury benchmarks. In Fiure 4, we show the disparity maps and correspondin (σ, τ, λ) at iteration, 2, 4, 6 for the four cases. Initially, the reularization is weak and the disparity map is noisy. As the alorithm proceeds, the reularization increases and the disparity map becomes cleaner. In our second experiment, we repeated experiment, but with different initial values. We show the initial and final (σ, τ, λ), includin those of experiment, in the top five rows in the tables in Table. Despite the variation of scales in initial parameters, the final parameters are consistent, showin robust converence of our alorithm. We also compared the final disparity maps usin the round truth, and showed the error rate in the last column. The error rates are also consistent. In addition to tryin different initial values, we also tried startin with the round truth disparity maps and estimated the parameters. Then we estimated the disparity map while keepin those parameters fixed. The error rates are shown in the bottom riht corner. Both the error rates and the parameters are close to the results obtained without knowin round truth. However, the parameters computed from round truth disparities don t result in disparity maps with lower error rates. This unintuitive fact is because real scenes are not perfectly described by our MRF model, as discussed in Section Optimality In our third experiment, for each case in experiment, we fix σ and τ but vary λ from to 50 and estimate disparity usin BP. We plot the error as a function of λ in Fiure 5. The red bar indicate our estimated values, whose correspondin error rates are quite close to the minimum error rates of the raphs in all the four benchmarks. 7.3 Improvement In our fourth experiment, we show how our automatic parameter settin method can improve over choosin fixed parameters manually. We first run BP with the fixed parameters suested in [4] and the result is shown in the fourth row ( Fixed ) in Table 2. Then we compare this result with the result in the first experiment where we solve for (σ, τ, λ), shown in the third row ( Adaptive ) in Table 2. Our adaptive method shows similar results to the fixed parameters for Sawtooth and Venus, but dramatic improvement on Map. As reported in [6], the Map pair requires different parameter settins than the other three datasets. Our alorithm automatically finds appropriate parameters without any user intervention. For the data set, the result is slihtly worse than the results with the fixed manually-chosen parameters. The reason is that a user exploits the round truth or his perception as a reference when settin the parameters, while our alorithm estimates the parameters based on model fittin. When the round truth is not the optimal solution of our model, the estimated disparity map could deviate from the round truth. Overall, our automatic parameter settin technique improves the rankin of the baseline alorithm by six slots on the Middlebury benchmarks. The experiments so far do not include the intensity radient cue proposed in Section 6. Now we consider this cue. First, we use the estimated values for (α, µ, β, ν), but set κ = and 0.0, respectively. The results for the two κ values are shown in the fifth row ( Fixed+rad ) and the second row ( Fixed+rad 2 ), respectively. κ = is apparently too lare and κ = 0.0 is better. If we estimate (α, µ, β, ν, κ) toether, as described in Section 6, we et the first row ( Adaptive+rad ) of Table 2. As expected, error rates in discontinuity reions, shown in the columns under disc., are consistently reduced. Overall, our parameter estimation technique raises the rank of the baseline alorithm (with intensity radient cue) by six slots, and the resultin adaptive alorithm is ranked the fifth amon all stereo alorithms in the Middlebury rankins, as of April 24, Discussion In this paper, we presented a parameter estimation method for MRF stereo. Our method converes consistently and sinificantly improves a baseline stereo alorithm. Our method works as a wrapper that interfaces with many stereo alorithms without requirin any chanes to those alorithms. Here we discuss some ideas for future work. First, our model ives hiher enery to round truth than to the estimated disparity maps [8]. One of the reasons is that we model visibility photometrically, but not eometrically. In other words, we assume visibility variables Γ are independent of D. One piece of future work would be to model the occlusion process more precisely. Second, we use historams of pixel matchin errors and neihborin disparity differences for a whole imae, assumin the mixture models don t vary across the imae. This assumption may be valid for the matchin errors, which are larely due to sensor noise, but it may not be accurate for the disparity maps, which may have spatially varyin smoothness. Acknowledments We would like to thank Aaron Hertzmann and Richard Szeliski for helpful discussions. This work was supported in part by National Science Foundation rant IIS ,

7 σ = 5.2, τ = 6.0, λ = σ = 8.94, τ = 7.02, λ = 0.83 σ = 7.50, τ = 2.3, λ = 6.78 σ = 8.39, τ =.64, λ = 9.49 σ = 5.2, τ = 6.92, λ = σ = 6., τ = 0.76, λ = 0.95 σ = 34.43, τ = 2.42, λ = 3.78 σ = 34.79, τ =.73, λ = σ = 5.2, τ = 6.92, λ = σ = 3.29, τ =.3, λ = 0.76 σ = 28.0, τ = 2.65, λ =.4 σ = 28.87, τ =.90, λ = 5.59 σ = 5.2, τ = 8.49, λ = σ = 6.99, τ = 2.5, λ = 0.89 σ = 60.25, τ = 4.43, λ = 3.42 σ = 85.50, τ =.87, λ = 4.7 Fiure 4. Converence on the four Middlebury benchmarks. The four columns correspond to iteration, 2, 4, and 6. Disparity error percentae Reularization weiht 50 Disparity error percentae0 Sawtooth Reularization weiht Disparity error percentae Venus Reularization weiht Disparity error percentae Map Reularization weiht Fiure 5. Graphs of error rate with respect to round truth as a function of reularization weiht λ while fixin (σ, τ). The vertical dotted lines are our estimation for λ. an Office of Naval Research YIP award, and Microsoft Corporation. References [] A. F. Bobick and S. S. Intille. Lare occlusion stereo. IJCV, 33(3):8 200, 999. [2] Y. Boykov, O. Veksler, and R. Zabih. Fast approximate enery minimization via raph cuts. IEEE Trans. on PAMI, 23(): , 200. [3] L. Chen and T. Caelli. Bayesian stereo matchin. In Proc. CVPR Workshop, paes 92 92, [4] P. F. Felzenszwalb and D. P. Huttenlocher. Efficient belief propaation for early vision. In Proc. CVPR, paes , [5] W. Freeman and A. Torralba. Shape recipes: Scene representations that refer to the imae. In Proc. NIPS., paes , [6] W. T. Freeman, E. C. Pasztor, and O. T. Carmichael. Learnin low-level vision. IJCV, 40():25 47, [7] P. Fua. A parallel stereo alorithm that produces dense depth maps and preserves imae features. Machine Vision and Applications, 6:35 49, 993. [8] S. Geman and G. Geman. Stochastic relaxation, ibbs dis-

8 Initial para Final para disparity σ τ λ σ τ λ error(%) Ground truth disparities Venus Initial para Final para disparity σ τ λ σ τ λ error(%) Ground truth disparities Sawtooth Initial para Final para disparity σ τ λ σ τ λ error(%) Ground truth disparities Map Initial para Final para disparity σ τ λ σ τ λ error(%) Ground truth disparities Table. Converence of our alorithm under five different initializations. The last rows use round truth disparity maps to estimate parameters and then compute new disparity maps with these estimated parameters. Parameter Overall Sawtooth Venus Map settin rankin all untex. disc. all untex. disc. all untex. disc. all disc. Adaptive+Grad Fixed+Grad Adaptive Fixed Fixed+Grad Table 2. Performance comparison of fixed and adaptive stereo solvers. Fixed: (σ, τ, λ) = (0, 2, 0). Adaptive: estimated (σ, τ, λ). Fixed+Grad : estimated (α, µ, β, ν), fixed κ =. Fixed+Grad 2: estimated (α, µ, β, ν), fixed κ = 0.0. Adaptive+Grad: estimated (α, µ, β, ν, κ). The rankins are as of 4/24/2005. tribuition and the bayesian restoration of imaes. IEEE Trans. on PAMI, 6:72 74, 984. [9] D. M. Hidon, J. E. Bowsher, V. E. Johnson, T. G. Turkinton, D. R. Gilland, and R. J. Jaszczak. Fully bayesian estimation of ibbs hyperparameters for emission computed tomoraphy data. IEEE Trans. on Medical Imain, 6(5):56 26, 997. [0] T. Kanade and M. Okutomi. A stereo matchin alorithm with an adaptive window: Theory and experiment. IEEE Trans. on PAMI, 6(9): , 994. [] V. Kolmoorov and R. Zabih. Computin visual correspondence with occlusions usin raph cuts. In Proc. ICCV, paes , 200. [2] A. Mohammad-Djafari. Joint estimation of parameters and hyperparameters in a bayesian approach of solvin inverse problems. In Proc. ICIP, paes , 996. [3] L. I. Rudin and S. Osher. Total variation based imae restoration with free local constraints. In Proc. ICIP, paes 3 35, 994. [4] S. S. Saquib, C. A. Bouman, and K. Sauer. ML paramter estimation for markov random fields with applications to bayesian tomoraphy. IEEE Trans. on Imae Processin, 7(7): , 998. [5] D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence alorithms. IJCV, 47():7 42, [6] J. Sun, H.-Y. Shum, and N. N. Zhen. Stereo matchin usin belief proation. In Proc. ECCV, paes , [7] H. Tao, H. S. Sawhney, and R. Kumar. Dynamic depth recovery from multiple synchronized video streams. In Proc. CVPR, paes 8 24, 200. [8] M. F. Tappen and W. T. Freeman. Comparison of raph cuts with belief propaation for stereo, usin identical MRF parameters. In Proc. ICCV, paes , [9] Z. Zhou, R. M. Leahy, and J. Qi. Approximate maximum likelihood hyperparameter estimation for ibbs priors. IEEE Trans. on Imae Processin, 6(6):844 86, 997.