AIT Inline Computational Imaging: Geometric calibration and image rectification

Transcription

1 AIT Inline Computational Imaging: Geometric calibration and image rectification B. Blaschitz, S. Štolc and S. Breuss AIT Austrian Institute of Technology GmbH Center for Vision, Automation & Control Vienna, Austria

2 INLINE COMPUTATIONAL IMAGING: WORKING PRINCIPLE Multi-line scan camera Camera IlluminationConstant illumination Inspected object Inspected object Transport stage Transport stage 2

3 INLINE COMPUTATIONAL IMAGING: WORKING PRINCIPLE Multi-line scan camera Constant illumination Inspected object Transport stage 3

4 Blaschitz et al. Geometric calibration and image rectification of Inline Computational Imaging AIT ICI LIGHT FIELD: MULTIPLE VIEWING & ILLUMINATION ANGLES y x pin grid array

5 AIT INLINE COMPUTATIONAL IMAGING: INDUSTRIAL USE CASES Electronic parts Metal parts Product packaging Coins Printed circuit boards Measurement Security print / OVD 2381μm 1064μm 5

6 ICI SOFTWARE MODULES FOR 2D/3D TASKS Acquisition Stereo matching Coarse refinement Light field data acquisition Camera, illumination, and transport control High speed multi-view / light field stereo matching Fast discrete depth denoising / regularization preserving discontinuities ICI Sensor System Camera Illumination Transport stage Rectification Camera lens undistortion & rectification Transport misalignment correction Feature extraction Extraction of robust image features Extraction of local surface features Fine refinement Fast continuous depth / regularization Fusion with additional depth cues Refined 2D/3D data Fine depth model (depth map, point cloud) Depth measurement confidence Refined 2D texture images (all-in-focus, gloss / shadow suppression) Fine 2.5D surface map 6

7 MODELLING SKEW TRANSPORT Linescan image n (x 3,y 3 ) Line 1 v Sensor Line n (x 2,y 2 ) u Lens (x 1,y 1 ) u v time Input Image Stack Linescan image 1 Corresponding points in different linescan images have different u-coordinate Goal: Rectified Image Stack (for easier matching) where transport appears aligned Transport direction 7

8 GOAL: RECTIFIED IMAGE STACK (FOR EASIER MATCHING) WHERE TRANSPORT APPEARS ALIGNED Corresponding points in different linescan images should have same u-coordinate Rectification Camera lens undistortion & rectification Transport misalignment correction uncalibrated calibrated 8

9 CALIBRATION PRINCIPAL Transport direction t=(t 1,t 2,t 3 ) x v u I v u c Focal length f Sensor plane Camera center 9

10 VIRTUAL IMAGE PLANE I t=(t 1,t 2,t 3 ) x x v u H Coordinate system of new image plane I I v u Parallel to transport direction c Minimal rotation f Same distance to camera center 10

11 VIRTUAL IMAGE PLANE Project sensor lines t=(t 1,t 2,t 3 ) x x v u H v u v I Homography from I to I u c f 11

12 VIRTUAL IMAGE PLANE Resample in I x x Equal u -coordinates H R x x v u Equidistant spacing 12

13 VIRTUAL IMAGE PLANE Pull back to I t=(t 1,t 2,t 3 ) I x x x v u H R H -1 v v I u c u c f f 13

14 VIRTUAL IMAGE PLANE Interpolate color values x v H R H -1 u x x M(u,v) is the color value at position (u,v) (u-1,v) (x,v) x=(u,v) 1 - s s v u 14

15 CALIBRATION IMPROVES STEREO MATCHING Acquisition Stereo matching Coarse refinement Light field data acquisition Camera, illumination, and transport control High speed multi-view / light field stereo matching Fast discrete depth denoising / regularization preserving discontinuities ICI Sensor System Camera Illumination Transport stage Rectification Camera lens undistortion & rectification Transport misalignment correction Feature extraction Extraction of robust image features Extraction of local surface features Fine refinement Fast continuous depth / regularization Fusion with additional depth cues Refined 2D/3D data Fine depth model (depth map, point cloud) Depth measurement confidence Refined 2D texture images (all-in-focus, gloss / shadow suppression) Fine 2.5D surface map 15

16 IMPROVED CORRESPONDENCE ANALYSIS Stereo matching High speed multi-view / light field stereo matching Rectification Camera lens undistortion & rectification Transport misalignment correction Feature extraction Extraction of robust image features Extraction of local surface features Refined 2D/3D data Fine depth model (depth map, point cloud) Depth measurement confidence Refined 2D texture images (all-in-focus, gloss / shadow suppression) Fine 2.5D surface map 16

17 SCENE COMPARISON IMAGE GRADIENT Uncalibrated Calibrated Maximal pixel intensity Reconstructed z value after depth denoising Maximal pixel intensity Reconstructed z value after depth denoising 17

18 SCENE COMPARISON 3D RECONSTRUCTION Uncalibrated Calibrated 18

19 EXAMPLE: STAIRCASE INPUT LIGHTFIELD STACK CAD model of the 3d printed model staircase (print tolerance approx mm) 19

20 EXAMPLE: STAIRCASE 3D RECONSTRUCTION Distance from camera center Height difference of two steps 20

21 EXAMPLE: EURO CENT COINS IMAGE STACK 10 Euro Cent coin Diameter 19.75mm Thickness 1,93mm 2 Euro Cent coin Diameter 18.75mm Thickness 1,67mm 21

22 EXAMPLE: EURO CENT COINS - UNCALIBRATED 22

23 EXAMPLE: EURO CENT COINS - CALIBRATED 23

24 EXAMPLE: EURO CENT COINS ERROR MEASURE 24

25 TAKE-HOME MESSAGE New calibration method turns AIT s Inline Computational Imaging system, which uses a single multi-line scan camera to generate 3d light field stacks into a measurement device in µm-scale at industrial inline production speed For industrial applications, scientific work and our patents visit High speed transport 25

26 THANK YOU FOR YOUR ATTENTION! Bernhard Blaschitz 26