Supplementary Material for A Locally Linear Regression Model for Boundary Preserving Regularization in Stereo Matching

Transcription

1 Supplementary Material for A Locally Linear Regression Model for Boundary Preserving Regularization in Stereo Matching Shengqi Zhu 1, Li Zhang 1, and Hailin Jin 2 1 University of Wisconsin - Madison 2 Adobe Systems Incorporated {sqzhu,lizhang}@cs.wisc.edu hljin@adobe.com In this supplementary material, we provide: (1) details in the upper bound calculation and (2) additional results on static images. A Details in Evaluating the Upper bound in Section 4.2 In our paper, we formulate our stereo matching as a global optimization as: Φ(D) = Φ 0 (D) + β 1 Φ 1 (D) + β 2 Φ 2 (D), (A.1) This section provides the details of finding upper bounds for the terms in Eq. (A.1). A.1 Φ 0 : Finding Upper bound for the Relaxed Data Term Recall that in our paper, Φ 0 is the data term. We calculate the upper bound of its relaxed version in two steps, as illustrated in Supplementary Figure 1. Step 1: Approximate the relaxed data term (red dashed line, which is piecewiselinear) by calculating quadratic B-splines (blue solid line); Step 2: Calculate the tightest upper bound Φ 0 (lowest green line) for the quadratic B-splines. Step 1: B-spline Approximation. To construct the quadratic B-spline approximation, we convolve the piecewise linear function with a box filter (of support [ 0.5, 0.5]). From the B-spline theory, the resulting B-spline approximation has the following property. Let L be the number of disparity levels. For each l {1, 2,, L}, if the three points on the relaxed data term at l 1, l, and l + 1 are on a straight line, the B-spline remains to be the the same line in the interval of [l 0.5, l + 0.5]. If the three points form a corner, the B-spline over [l 0.5, l+0.5] is a parabola that connects the two points on the original function at l 0.5 and l with first order continuity. In summary, within each interval [l 0.5, l + 0.5], the B-spline approximation of the relaxed data term is a parabola defined as β i (d) = p l d 2 + q l d + r l (A.2) where p l, q l, and r l are coefficients that vary with each pixel i; we drop the subscript i to simplify notation. This simple box-filtering treatment enables us to use unconstrained quadratic optimization to solve the stereo problem.

2 2 Shengqi Zhu, Li Zhang, and Hailin Jin 11 x Supplementary Fig. 1. (Same as Figure 1(d) in the paper.) Relationship among relaxed data term (dashed red), its B-spline approximation (solid blue), and the upper bound for the B-spline approximation (solid green) at the current disparity estimation (black circle). We search for the upper bound parabola with the lowest curvature to approximate the relaxed data term in our optimization. Step 2: Upper bound for the Quadratic B-spline. Precisely, let D be the current disparity map and d i be the current disparity value at pixel i (black circle in Supplementary Figure 1). We first evaluate the value β i (d i ) and the derivative β i (d i ) of the B-spline approximation at d i. The current quadratic upper bound β i ( ) (solid green line) is set to be a parabola that has the same value and derivative as β i ( ) at d i, using the following form β i (d) = τ i (d d i ) 2 + β i(d i ) (d d i ) + β i (d i ) (A.3) where τ i 0 is the only free parameter. When τ i, β i ( ) will have infinite curvature and surely be an exact upper bound for β i ( ). Therefore, our goal is to find the smallest τ i so that the upper bound curve β i ( ) has low curvature and matches as closely as possible to the β i ( ) curve. This is the same as finding, for each l, the smallest value for τ i so that β i ( ) β i ( ) over [l 0.5, l + 0.5] and then taking the maximum of these L smallest values. The procedure has a complexity of O(L) for each pixel. A.2 Φ 1 : Finding Upper bound for the First Order Term Φ 1 Here we show how to get Eq (14). From Eq (8), we have Φ 1 (D, A) = v ij (d 2 j + at i f ifi Ta i 2a T i f jd j ) + a T i Λa i i j = D T diag(v j )D + (a T i F ia i 2a T i G id), i (A.4) where diag(v j ) is a diagonal matrix with elements v j ; v j, F i, and G i are defined as: v j = v ij i F i = Λ + v ij f i fi T (A.5) j G i = [v i1 f 1, v i2 f 2,, v in f N ]. Fixing D, Eq. (A.4) is minimized when a i = F 1 i G i D. (A.6)

3 Supp: A Local Regression Model for Stereo Matching Regularization 3 where Substituting Eq. (A.6) into Eq. (A.4), we have Φ 1 (D) = D T H 1 D, H 1 = diag(v j ) i B Additional Results B.1 More Comparisons with [11] (A.7) G T i F 1 i G i. (A.8) We provide one more example to compare with [11], which uses pair-wise weighted L 1 penalty as regularization. As shown in Supplementary Figure 2, our method better preserves sharp boundaries and avoids producing isolated small regions. (a) Left Image (c) [11] with color bandwidth = 4 (e) [11] with color bandwidth = 2 (g) Our method (b) Ground Truth (d) Bad Pixel Rate 13.57% (non-occ) (f) Bad Pixel Rate 11.50% (non-occ) (h) Bad Pixel Rate 8.04% (non-occ) Supplementary Fig. 2. Comparison between our method and [11] on Art dataset. The weighted L 1 penalty used in [11] makes its results sensitively depend on the color bandwidth parameter. A large color bandwidth yields over-smoothing across discontinuity boundaries ((c) and (d)) and a small color bandwidth results in many isolated areas in the background due to texture ((e) and (f)). Our method ((g) and (h)) better preserves boundaries and avoids producing isolated small regions. B.2 Middlebury Results In this section, we report the results of our algorithm on all 31 Middlebury datasets, including the 4 benchmarks, the 2005 and the 2006 datasets. We set the same parameter as in Section 6.3 (β 1 = β 2 = , λ = 10 3 ) for all the experiments. The error rates are for the non-occluded bad pixels.

4 4 Shengqi Zhu, Li Zhang, and Hailin Jin Tsukuba, 2001 groundtruth our result bad pixel [1.05%] Venus, 2001 groundtruth our result bad pixel [0.29%] Teddy, 2003 groundtruth our result bad pixel [4.56%] Cones, 2003 groundtruth our result bad pixel [2.17%] Art, 2005 groundtruth our result bad pixel [8.04%]

5 Supp: A Local Regression Model for Stereo Matching Regularization 5 Books, 2005 groundtruth our result bad pixel [9.08%] Dolls, 2005 groundtruth our result bad pixel [3.48%] Laundry, 2005 groundtruth our result bad pixel [9.76%] Moebius, 2005 groundtruth our result bad pixel [9.60%] Reindeer, 2005 groundtruth our result bad pixel [4.01%]

6 6 Shengqi Zhu, Li Zhang, and Hailin Jin Aloe, 2006 groundtruth our result bad pixel [4.89%] Baby1, 2006 groundtruth our result bad pixel [2.85%] Baby2, 2006 groundtruth our result bad pixel [3.82%] Baby3, 2006 groundtruth our result bad pixel [2.70%] Bowling1, 2006 groundtruth our result bad pixel [14.27%]

7 Supp: A Local Regression Model for Stereo Matching Regularization 7 Bowling2, 2006 groundtruth our result bad pixel [4.59%] Cloth1, 2006 groundtruth our result bad pixel [0.91%] Cloth2, 2006 groundtruth our result bad pixel [2.13%] Cloth3, 2006 groundtruth our result bad pixel [1.29%] Cloth4, 2006 groundtruth our result bad pixel [1.19%]

8 8 Shengqi Zhu, Li Zhang, and Hailin Jin Flowerpots, 2006 groundtruth our result bad pixel [11.78%] Lampshade1, 2006 groundtruth our result bad pixel [10.01%] Lampshade2, 2006 groundtruth our result bad pixel [3.56%] Midd1, 2006 groundtruth our result bad pixel [46.52%] Midd2, 2006 groundtruth our result bad pixel [43.85%]

9 Supp: A Local Regression Model for Stereo Matching Regularization 9 Monopoly, 2006 groundtruth our result bad pixel [36.80%] Plastic, 2006 groundtruth our result bad pixel [18.28%] Rocks1, 2006 groundtruth our result bad pixel [1.99%] Rocks2, 2006 groundtruth our result bad pixel [1.10%] Wood1, 2006 groundtruth our result bad pixel [3.34%]

10 10 Shengqi Zhu, Li Zhang, and Hailin Jin Wood2, 2006 groundtruth our result bad pixel [0.39%]