Bilateral and Trilateral Adaptive Support Weights in Stereo Vision

Transcription

1 Cost -based In GPU and Support Weights in Vision Student, Colorado School of Mines April 7, / 36

2 Overview Cost -based In GPU 1 Cost 2 3 -based 4 In GPU 2 / 36

3 Cost -based In GPU 3 / 36

4 [3] Cost Reference Image (left) Target Image (right) Disparity Image (left) -based In GPU 4 / 36

5 Cost -based In GPU How Disparity Image is Generated 5 / 36

6 Cost -based In GPU Pixel Ambiguity issues 6 / 36

7 Cost -based In GPU Blocks of Pixels Reduce Ambiguity 7 / 36

8 Cost -based In GPU Left Image Disparity Image 8 / 36

9 Cost -based In GPU Cost, Cost Aggregation SAD = 2 SAD = 32 SAD = 17 9 / 36

10 Cost -based In GPU SAD Limitation What is wrong with using SAD on this reference window? 10 / 36

11 Cost -based In GPU 11 / 36

12 Cost -based In GPU Edge-Preserving Version on Gaussian Gaussian kernel (regardless of pixel values) kernel at a noisy step [4] 12 / 36

13 Cost -based In GPU Original Image Example ly Smoothed Image 13 / 36

14 Cost -based In GPU Introduction Reference Window Target Window 1 Target Window 2 14 / 36

15 Cost -based In GPU Individual s Reference Window Target Window 1 Target Window 2 Individual Window s 15 / 36

16 Cost -based In GPU Reference Window Target Window 1 Combined s Reference Window Target Window 1 Combined Support Weight 16 / 36

17 Cost -based In GPU Reference Window Target Window 2 Combined s, cont. Reference Window Target Window 2 Combined Support Weight 17 / 36

18 Cost -based In GPU Raw BM Output Improved Output Output[5] 18 / 36

19 Cost -based In GPU Other Methods Many alternative methods for adaptive support weight have been proposed based on the original bilateral-based implementation. Modifications include: Removing spacial component of bilateral filter Replacing bilateral with guided filter Replacing bilateral filter with some other filter Approximating the bilateral filter Calculating support weight for only the reference image For more discussion, see [2]. 19 / 36

20 Limitations Cost -based In GPU Image from [5]. 20 / 36

21 Cost -based In GPU 21 / 36

22 Cost -based In GPU filter, in terms of 22 / 36

23 Cost -based In GPU filter, in terms of, cont. 23 / 36

24 Cost -based In GPU Cut-off Term 24 / 36

25 Cost -based In GPU Original Window Additional support weight term -based support weight Combined support weight 25 / 36

26 Cost -based In GPU -based Notice the trilateral-based (left) outpreforms bilateral-based (right) near edges: Images adapted from [1] 26 / 36

27 Cost -based In GPU Performance Image Algorithm nonocc all disc Semi-Global BM Tsukuba Semi-Global BM Venus Semi-Global BM Teddy Semi-Global BM Cones / 36

28 Cost -based In GPU in 28 / 36

29 GPU Cost -based In GPU Preliminary GPU Runs as fast as 94 ms (Quadro K6000) (from paper)[5] Takes almost 60 seconds (CPU-only). 29 / 36

30 Cost -based In GPU Semi-Global Block OpenCV Options Disparity Filter Belief Propagation and Constant-Space Belief Propagation not shown 30 / 36

31 Cost -based In GPU Nvidia VisionWorks Semi-Global 31 / 36

32 Cost -based In GPU Tests ran on 450x375 cones images Algorithm Jetson TK1 Jetson TX1 i7 + Quadro K6000 CPU BM 29ms 29ms 12ms GPU BM 18ms 9.5ms 2.2ms GPU DBF 64ms 21ms 10ms CV SGBM 870ms 990ms 99ms VX BM 13ms 5.7ms 2.3ms VX SGBM 65ms 42ms 5.1ms GPU 8,900ms 6,800ms 94ms CPU 190,000ms 200,000ms 73,000ms Source code for test programs can be found at: 32 / 36

33 Cost -based In GPU Pros: Excellent edge accuracy High disparity detail Less noise Parallelizes well Cons: Much higher computational complexity No completed open-source GPU version My GPU-accelerated implementation can be found at: 33 / 36

34 Cost -based In GPU References I Dongming Chen, Mohsen Ardabilian, and Liming Chen. A novel trilateral filter based adaptive support weight method for stereo matching. IEEE Transactions on Circuits and Systems for Video Technology, 25: , Asmaa Hosni, Michael Bleyer, and Margrit Gelautz. Secrets of adaptive support-weight for local stereo matching. Computer Vision and Image Understanding, 117: , Daniel Scharstein and Richard Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, / 36

35 Cost -based In GPU References II C. Tomasi and R. Manduchi. filtering for gray and color images. IEEE International Conference on Computer Vision, Kuk-Jin Yoon and In So Kweon. support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28: , / 36

36 Cost -based In GPU Questions? 36 / 36