EECS 442 Computer vision. Announcements

Transcription

1 EECS 442 Computer vision Announcements Midterm released after class (at 5pm) You ll have 46 hours to solve it. it s take home; you can use your notes and the books no internet must work on it individually no collaborative solutions can ask general questions to GSIs Due by 3pm on Thursday in class Conversation hour tomorrow will used as an extra office hour.

2 EECS 442 Computer vision Volumetric stereo Definition Shape from Contours Shape from Shadows Voxel coloring

3 Traditional Stereo P p p O 1 O 2 Goal: estimate the position of P given the observation of P from two view points Assumptions: known camera parameters and position (K, R, T)

4 Traditional Stereo P p p O 1 O 2 Subgoals: 1. Solve the correspondence problem 2. Use corresponding observations to triangulate

5 Volumetric stereo Scene volume O O Hypothesis: pick up a point within the volume 2. Project this point into 2 (or more) images 3. Validation: are the observations consistent? Assumptions: known camera parameters and position (K, R, T)

6 Consistency based on cues such as: - Contours/silhouettes - Shadows - Colors

7 Apparent contour DEFINITION: projection of the locus of points on the surface which separate the visible and occluded parts on the surface [sato & cipolla] apparent contour Camera Image

8 Silhouettes silhouette Image Camera

9 Why contours are interesting visual cues? Provide information in absence of other visual cues No texture No shading

10 Relatively easy to detect Why contours are interesting visual cues? Object Camera Image Easy contour segmentation

11 How can we use contours? Object Visual cone Image Object apparent contour Camera

12 How can we use contours? Object Camera Image Image object s contour Image object s contour Visual cone in 2D: Camera Image Object

13 View point 1 Object Object The views are calibrated Object estimate (visual hull) View point 2

14 how to perform visual cones intersection? decompose visual cone in polygonal surfaces (among others: Reed and Allen 99)

15 Using contours/silhouettes in volumetric stereo also called Space carving [ Martin and Aggarwal (1983) ] voxel

16 Computing visual hull in 2D

17 Computing visual hull in 2D

18 Computing visual hull in 2D

19 Computing visual hull in 2D Visual hull: an upper bound estimate Consistency: A voxel must be projected into a silhouette in each image

20 Space Carving has complexity O(N 3 ) Octrees (Szeliski 93) N N N

21 Complexity reduction: octrees 1 1 1

22 Complexity reduction: octrees Subdiving volume in voxels of progressive smaller size 2 2 2

23 Complexity reduction: octrees 4 4 4

24 Complexity reduction: 2D example 4 voxels analyzed

25 Complexity reduction: 2D example 16 voxels analyzed

26 Complexity reduction: 2D example 52 voxels analyzed

27 Complexity reduction: 2D example 16x34 voxels analyzed x16 = 617 voxels have been analyzed in total (rather than 32x32 = 1024)

28 Advantages of space carving Robust and simple No need to solve for correspondences

29 Limitations of space carving Accuracy function of number of views Not a good estimate What else?

30 Limitations of space carving Concavities are not modeled Concavity For 3D objects: Are all types of concavities problematic?

31 Limitations of space carving Concavities are not modeled Concavity Visual hull Laurentini (1995) Closest approximation (hyperbolic regions are ok) Conservative

32 Space carving: a classic setup Camera Object Courtesy of seitz & dyer Turntable

33 Space carving: a classic setup

34 Space carving: Experiments 24 poses (15 O ) voxel size = 2mm 6cm 10cm

35 Space carving: Experiments 30 cm 24 poses (15 O ) voxel size = 1mm

36 Space carving: Conclusions Robust Produce conservative estimates Concavities can be a problem Low-end commercial 3D scanners blue backdrop turntable

37 Contours in the computer vision literature Analyzing changes in apparent contours Giblin and Weiss (1987) Cipolla and Blake (1992) Vaillant and Faugeras (1992) Ponce ( 92), Zheng( 94) Furukawa et al. ( 05 ) Picture from of Sato & Cipolla

38 Volumetric stereo Definition Shape from Contours Shape from Shadows Voxel coloring

39 Self-shadows are visual cues for shape recovery Self-shadows indicate concavities (no modeled by contours)

40 Shape from shadow in the literature Shafer & Kanade ( 83) Hatzitheodorou & Kender ( 89) et al Hatzitheodorou & Kender ( 89) shape as a 2.5D terrains -- i.e. surface modeled as f(x) accurate shadow detection i.e. x b and x e

41 Shape from shadow in the literature Shafer & Kanade ( 83) Hatzitheodorou & Kender ( 89) et al Hatzitheodorou & Kender ( 89) shape as a 2.5D terrains -- i.e. surface modeled as f(x) Using shadows in volumetric stereo accurate shadow detection i.e. x b and x e

42 Shadow Carving Savarese et al 01 Self-shadows + Robust with respect to shadow estimates Object with arbitrary topology (no 2.5D terrains) Object s upper bound

43 Shadow carving: setup Object Light Camera Image Let s define a 2D model

44 2D slice model Object Light Camera Image

45 2D slice model Object Light From 3D to 2D Camera Image Object Image line Camera Light

46 The idea Object s upper bound Object Shadow Image line Camera Image shadow Light

47 The idea Object s upper bound Object Shadow Image line Camera Image shadow Light

48 The idea Object s upper bound Image line Object Shadow Camera Image shadow Light

49 The idea Object s upper bound Image line Object Shadow Camera Image shadow Light

50 The idea Object s upper bound Image line Object Shadow Camera Image shadow Light

51 The idea Object s upper bound Image line Object Shadow Camera Image shadow Light

52 The idea Reconstruction after 3 iterations

53 Algorithm: step k Camera Image Image line Light source Image shadow Upper-bound from step k-1 Object

54 Algorithm: step k Camera Image Image line dual image shadow Light source Image shadow? Upper-bound from step k-1 Object

55 Algorithm: step k Camera Image Image line dual image shadow Light source Image shadow? Object Upper-bound from step k-1 Consistency: A voxel must be projected into both image shadow and dual image shadow

56 What can we carve? Camera Image Image line light image shadow Light source Image shadow No further volume can be removed Upper-bound from step k-1 Carving process always conservative Proof of correctness Object

57 Algorithm: step k Camera Image Image line light image shadow Light source Image shadow Upper-bound from step k-1 Complexity? O(2N 3 ) Object

58 Shadow carving: the setup Camera Object Array of lights Turntable

59 Shadow carving: the setup Camera Object Array of lights Turntable

60 Experimental results

61 Simulating the system with 3D studio Max - 24 positions - 4 lights - 72 positions - 8 lights

62 - 24 positions - 4 lights - 72 positions - 8 lights

63 - 16 positions - 4 lights

64

65 Shadow carving: summary Produces a conservative volume estimate Accuracy depending on view point and light source number Limitations with specular & low albedo regions

66 Volumetric stereo Definition Shape from Contours Shape from Shadows Voxel coloring

67 Voxel Coloring Seitz & Dyer ( 97) R. Collins (Space Sweep, 96) color/photo-consistency Jointly model structure and appearance

68 Basic idea Is this voxel in or out? View1 View2 View3

69 Basic idea View1 View2 View3

70 Uniqueness Multiple consistent scenes Ijcv seitz & dyer

71 Uniqueness Multiple consistent scenes Ijcv seitz & dyer

72 Tractability N M colors N N Combinatorial number possible assignments! Exhaustive search not feasible Model occlusions! Use visibility constraint

73 The algorithm C

74 Algorithm complexity voxel coloring visits each N 3 voxels only once project each voxel into L images O(L N 3 ) NOTE: not function of the number of colors

75 Photoconsistency test = corr (, ) Image 1 Image 2 If < Thresh voxel consistent

76 A critical assumption: Lambertian surfaces I 1 (p) I 1 (p) = I 2 (p) I 2 (p)

77 Non Lambertian surfaces

78 Experimental results Dinosaur 72 k voxels colored 7.6 M voxels tested 7 min to compute on a 250MHz

79 Experimental results Flower 70 k voxels colored 7.6 M voxels tested 7 min to compute on a 250MHz

80 Room + weird people Experimental results

81 Voxel coloring: conclusions Good things: model intrinsic scene colors and texture no assumptions on scene topology

82 Voxel coloring: conclusions Good things: model intrinsic scene colors and texture no assumptions on scene topology Limitations: Constrained camera positions Lambertian assumption

83 space carving Space carving is a binary voxel coloring out in

84 Shadow carving Camera Light source ^ S no visibility constraint needed

85 Further contributions A Theory of Space Carving (Kutulakos & Seitz 99) Voxel coloring in more general framework No restrictions on camera position Probabilistic Space Carving (Broadhurst & Cipolla, ICCV 2001) (Bhotika, Kutulakos et. al, ECCV 2002)

86 Next lecture Affine Structure from Motion