Computer Vision. Exercise Session 6 Stereo matching. Institute of Visual Computing

Transcription

1 Computer Vision Exercise Session 6 Stereo matching

2 Assignment 6 3 Tasks: Disparity computation Winner-takes-all Graph-cut Textured 3D model

3 Stereo Setup Bring two views to standard stereo setup Epipoles are at infinity Epipolar lines are parallel

4 Planar Rectification Code provided

5 Disparity SSD, Winner-Takes-All Match windows along scanline Remember offset of best matching window as disparity value

6 Disparity SSD Slow Version Find the offset d(x, y) of matching pixels Search algorithm (convert to gray scale rgb2gray) For each pixel (x, y), for each disparity d SSD = 0 For each pixel (i, j) in window SSD = SSD + (I1(x+i, y+j) - I2(x+i+d, y+j+d)).^2 Remember disparity with smallest SSD SLOW!

7 Disparity faster version for Matlab For each disparity d Shift entire image by d (code provided (shiftimage)) Compute image difference (SSD, SAD) Convolve with box filter Use conv2(..., same ) and fspecial( average, ) Remember best disparity for each pixel mask = Idiff < bestdiff Resize images if your stereo is taking too long

8 Disparity result Depth discontinuities Depth should be continuous

9 Disparity Graph-Cut Stereo is a labeling problem Assign each pixel the corresponding disparity (label) Matching pixels should have similar intensities Most nearby pixels should have similar disparities

10 Disparity Graph-Cut Familiarize yourself with the sample code See the gc_example() file on color segmentation compile_gc Adapt the code to compute the disparity Change the data cost (Dc) Compute for each pixel the SSD at each disparity Store SSD values in a m x n x r matrix, where m x n is the image size and r is the number of disparities (labels) The rest remains unchanged You may need to change the weighting of the terms

11 Graph-Cut - Results Result with simple cost function winner-takes-a ll graphcut

12 Textured 3D model For each pixel find the corresponding 3D point using Disparity maps Camera parameters, rectifying homographies Generate textured 3D model (code provided).obj-file.mtl-file Image file Put everything in the same folder, load.obj-file with Meshlab.

13 3D Point Cloud One 3D point per pixel Two possible approaches Standard stereo geometry Triangulation Do not forget that homographies were applied to the images in order to rectify them and maybe also scale them!

14 3D Point Cloud Standard stereo geometry formula given on the slides requires Equal intrinsics for both cameras Zero skew, principal point (0,0) Do not forget that applying homographies to the images changes the camera matrices for the respective images

15 3D Point Cloud Triangulation For each pixel in image 1 get the corresponding pixel in image 2 using the disparity map Triangulate using the camera matrices Do not forget that applying homographies to the images changes the camera matrices for the respective images

16 Textured 3D model from an image pair

17 Framework Functions that need to be completed/implemented (you can add functions, of course): stereodisparity.m diffsgc.m gcdisparity.m generatepointcloudfromdisps.m exercise6.m

18 Framework Several functions provided for convenience lintriang: linear triangulation leftrightcheck: see if disparity calulated for left image agrees with disparity calculated for right image Decompose: decompose camera matrix P into K, R and COP shiftimage: shift entire image along x-axis