3D Scanning. Lecture courtesy of Szymon Rusinkiewicz Princeton University

Transcription

1 3D Scanning Lecture courtesy of Szymon Rusinkiewicz Princeton University

2 Computer Graphics Pipeline 3D Scanning Shape Motion Rendering Lighting and Reflectance Human time = expensive Sensors = cheap Computer graphics increasingly relies on measurements of the real world

3 3D Scanning Applications Computer graphics Product inspection Robot navigation Product design Archaeology Clothes fitting

4 Industrial Inspection Determine whether manufactured parts are within tolerances

5 Medicine Pre-operatively scan the patient project in real time

6 Medicine Pre-operatively scan the patient project in real time

7 Medicine Pre-operatively scan the patient project in real time

8 Medicine Pre-operatively scan the patient project in real time

9 Scanning Buildings Quality control during construction

10 Clothing Scan a person, custom-fit clothing U.S. Army; booths in malls

11 The Digital Michelangelo Project

12 The Digital Michelangelo Project

13 Why Scan Sculptures? Virtual museums Controlled interaction (lighting, proximity, etc.) Study working techniques Cultural heritage preservation

14 Goals Scan 10 sculptures by Michelangelo High-resolution ( quarter-millimeter ) geometry Side projects: architectural scanning (Accademia and Medici chapel), scanning fragments of Forma Urbis Romae

15 Why Capture Chisel Marks?? ugnetto Atlas (Accademia)

16 Why Capture Chisel Mark Geometry? 2 mm Day (Medici Chapel)

17 Side project: The Forma Urbis Romae

18 Forma Urbis Romae Fragment side face

19

20 Active Scanners Triangulation (in 2D): To figure out the position of a point we need: 1. The position of the camera 2. The position of the light source Project the light onto the surface Find where the lit point projects onto the camera Cast a ray from the lit pixel, through the camera

21 Active Scanners Triangulation (in 2D): To figure out the position of a point we need: 1. The position of the camera 2. The position of the light source Project the light onto the surface For Find the where lit point the to project The lit point position projects of the lit point onto the appropriate pixel, onto the camera it has to lie somewhere has to on be at the intersection the Cast ray. a ray from the of lit the pixel, two through rays the camera The lit point is also constrained to lie on the ray from the light source.

22 Triangulation Object Laser Camera Project laser stripe onto object

23 Triangulation Object Laser (x,y) Camera Depth from ray-plane triangulation

24 Triangulation: Moving the Camera and Illumination Moving independently leads to problems with calibration Most scanners mount camera and light source rigidly, move them as a unit

25 Triangulation: Moving the Camera and Illumination

26 Triangulation: Moving the Camera and Illumination

27 Scanning a Large Object Calibrated motions pitch (yellow) pan (blue) horizontal translation (orange) Uncalibrated motions vertical translation rolling the gantry remounting the scan head

28 Range Processing Pipeline Steps 1. manual initial alignment 2. ICP to one existing scan 3. global relaxation to spread out error 4. merging using volumetric method

29 Range Processing Pipeline Steps 1. manual initial alignment 2. ICP to one existing scan 3. global relaxation to spread out error 4. merging using volumetric method ICP( Scan1, Scan2 ) : 1. For each point on Scan1, find the nearest point on Scan2. 2. Translate/Rotate Scan1 to minimize the distance between corresponding points. 3. Go to step 1

30 Range Processing Pipeline Steps 1. manual initial alignment 2. ICP to one existing scan 3. global relaxation to spread out error 4. merging using volumetric method ICP( Scan1, Scan2 ) : 1. For each point on Scan1, find the nearest point on Scan2. 2. Translate/Rotate Scan1 to minimize the distance between corresponding points. 3. Go to step 1

31 Recall Given a matrix: M = m 11 m n1 m 1n m nn The trace is the sum of the diagonal entries: Trace M = i m ii

32 Recall 1. Given matrices P and Q, we have: P Q t = Q t P t 2. Given a square matrix P, we have: Trace P = Trace(P t ) 3. Given matrices P and Q, we have: Trace P Q = Trace(Q P) 4. Given vectors v and w, we have: v ± w 2 = v 2 + w 2 ± 2 v, w

33 Recall Given a set of points Ԧp 1,, Ԧp n R m, we can construct the n m matrix which has the points as columns: P = Ԧp 1 Ԧp n Given the transformation M R m m, the matrix defined by the transformed points is: M Ԧp 1 M Ԧp n = M P

34 Recall Given two such n m matrices P = ( Ԧp 1 Ԧp n ) and Q = ( Ԧq 1 Ԧq n ), we have: P t Q = Ԧp 1, Ԧq 1 Ԧp 1, Ԧq n Ԧp n, Ԧq 1 Ԧp n, Ԧq n In particular, we have: n Trace(P t Q) = i=1 p i, q i

35 Recall Given two such n m matrices P = ( Ԧp 1 Ԧp n ) and Q = ( Ԧq 1 Ԧq n ), we have: P t Q = Ԧp 1, Ԧq 1 Ԧp 1, Ԧq n Ԧp n, Ԧq 1 Ԧp n, Ԧq n If R is an orthogonal transformation: R t R = Id. The columns of R are orthonormal vectors R ij 1

36 Recall 1. If F Ԧp = Ԧp 2 then: F = p x 2 + p y 2 + p z 2 = 2p x, 2p y, 2p z = 2 Ԧp 2. If F Ԧp = Ԧp, Ԧq then: F = p x q x + p y q y + p z q z = q x, q y, q z = Ԧq

37 Claim Given a diagonal matrix D, the orthogonal transformation R maximizing the trace: Trace R D is the matrix: sign(d 11 ) 0 R = sign D = 0 sign(d nn ) This gives: Trace R D = sign D 11 D sign D nn D nn = D D nn

38 Proof Setting: R = R 11 R n1 D = R 1n R nn D D nn we get: Trace R D = R 11 D R nn D nn Since the columns of R have square norm equal to one, we must have R ii 1. This implies that for any choice of R we must have: Trace R D D D nn

39 ICP Registration Goal: Given points Ԧp 1,, Ԧp n R m and Ԧq 1,, Ԧq n R m, find the translation Ԧδ R m and orthogonal transform R R m m that best aligns { Ԧp i } to { Ԧq i }. That is, find Ԧδ and R minimizing the alignment energy: n E( Ԧδ, R) = i=1 R( Ԧp i + Ԧδ) Ԧq i 2

40 ICP Registration Goal: 1. Find the translation Ԧδ minimizing: n E( Ԧδ) = i=1 n = 2 ( Ԧp i + Ԧδ) Ԧq i Ԧp i Ԧq 2 i + Ԧδ 2 2 Ԧp i Ԧq i, Ԧδ i=1 n E( Ԧδ) = 2n Ԧδ 2 Ԧp i Ԧq i i=1

41 ICP Registration Goal: 1. Find the translation Ԧδ minimizing: n E( Ԧδ) = 2n Ԧδ 2 i=1 Ԧp i Ԧq i The minimizing translation must satisfy: E( Ԧδ) = 0 n Ԧδ = 1 n i=1 ( Ԧp i Ԧq i )

42 ICP Registration Goal: 2. Find the orthogonal transform R minimizing: n 2 E R = R( Ԧp i ) Ԧq i i=1 n = R( Ԧp i ) 2 + Ԧq 2 i 2 R( Ԧp i ), Ԧq i i=1 Minimizing E(R) is the same as maximizing: n E R = R p i, q i i=1

43 ICP Registration Goal: 2. Find the orthogonal transform R maximizing: n E R = R Ԧp i, Ԧq i = Trace P t R t Q i=1

44 ICP Registration Goal: 2. Find the orthogonal transform R maximizing: n E R = R Ԧp i, Ԧq i = Trace P t R t Q i=1 = Trace Q P t R t = Trace R P Q t

45 ICP Registration Goal: 2. Find the orthogonal transform R maximizing: E R = Trace R P Q t Computing the singular value decomposition: P Q t = U D V t with U and V orthogonal and D diagonal. This gives: E R = Trace R U D V t = Trace (V t R U D)

46 ICP Registration Goal: 2. Find the orthogonal transform R maximizing: E R = Trace (V t R U D) Since V t R U is orthogonal, this is maximized if: V t R U = sign(d) R = V sign D U t

47 Range Processing Pipeline Steps 1. manual initial alignment 2. ICP to one existing scan 3. global relaxation to spread out error 4. merging using volumetric method

48 Range Processing Pipeline + Steps 1. manual initial alignment 2. ICP to one existing scan 3. global relaxation to spread out error 4. merging using volumetric method

49 Range Processing Pipeline Steps 1. manual initial alignment 2. ICP to one existing scan 3. global relaxation to spread out error 4. merging using volumetric method

50 Statistics About the Scan of David 480 individually aimed scans 0.3 mm sample spacing 2 billion polygons 7,000 color images 32 gigabytes 30 nights of scanning 22 people

51 Head of Michelangelo s David Photograph 1.0 mm computer model