Tracking for the Registration of 3D and Multispectral Datasets

Transcription

1 Tracking for the Registration of 3D and Multispectral Datasets C. Simon Chane 1 2 R. Schütze 2 F. Marzani 1 F. Boochs 2 csimonchane@cite-musique.fr 1 le2i, Université de Bourgogne, France 2 i3mainz, Fachhochschule Mainz, Germany 1 / 35

2 Overview 1 Introduction Object under study Problem outline 2 Method Principle Simulations 3 Experiments Calibrations Tracking 3D registration 3D MS registration 4 Conclusion 2 / 35

3 Overview 1 Introduction Object under study Problem outline 2 Method 3 Experiments 4 Conclusion 3 / 35

4 Polychrome sandstone sarcophagus 4 / 35

5 Polychrome sandstone sarcophagus dated to the end of the 3 rd century A.D. crypt under the Friedhofs chapel of the St. Matthias abbey, Trier (Germany) discovered 50 years ago 4 / 35

6 There is no climate control in the crypt variations in humidity air flow 5 / 35

7 The surface of the sarcophagus is flaking Beginning of flaking Loss due to flaking Goal: Precisely localize and quantify the surface and polychromy degradation to highlight correlations. 6 / 35

8 Problem outline Alteration of colored surfaces must be studied spatially and spectrally. Need portable and flexible instrumentation. We use independent 3D scanner and multispectral camera. 7 / 35

9 Acquisition systems in use Commercial acquisition systems Gom Atos III Fringe projection digitization system. Resolution: 0.25 mm for a 500 mm 500 mm surface. FluxData FD 1665 MS Number of filters: 7. Acquisition range: 400 nm to 950 nm. 8 / 35

10 Problem outline Challenges Multiview and multimodal registration. With no natural feature points. Without the use of targets. Open surfaces. 9 / 35

11 Overview 1 Introduction 2 Method Principle Simulations 3 Experiments 4 Conclusion 10 / 35

12 Principle Based on photogrammetry tracking cameras acquisition system surface under study 11 / 35

13 Principle Based on photogrammetry tracking cameras acquisition system target frame surface under study 12 / 35

14 Calibrations tracking cameras acquisition systems target frame Tracking cameras interior orientation. Acquisition systems interior orientation. Target frame calibration. Target frame to acquisition system calibration. 13 / 35

15 Calibrations tracking cameras acquisition systems target frame Tracking cameras interior orientation. Acquisition systems interior orientation. Target frame calibration. Target frame to acquisition system calibration. 13 / 35

16 Calibrations tracking cameras acquisition systems target frame Tracking cameras interior orientation. Acquisition systems interior orientation. Target frame calibration. Target frame to acquisition system calibration. 13 / 35

17 Calibrations tracking cameras acquisition systems target frame Tracking cameras interior orientation. Acquisition systems interior orientation. Target frame calibration. Target frame to acquisition system calibration. 13 / 35

18 Calibrations Tracking cameras interior orientation. Acquisition systems interior orientation. Target frame calibration. Target frame to acquisition system calibration. 13 / 35

19 Acquisitions surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

20 Acquisitions tracking cameras surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

21 Acquisitions tracking cameras calibration plate surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

22 Acquisitions tracking cameras calibration plate surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

23 Acquisitions tracking cameras calibration plate surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

24 Acquisitions tracking cameras calibration plate surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

25 Acquisitions tracking cameras surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

26 Acquisitions tracking cameras acquisition system and target frame surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

27 Acquisitions tracking cameras acquisition system and target frame surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

28 Acquisitions tracking cameras acquisition system and target frame surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

29 Acquisitions tracking cameras acquisition system and target frame surface under study Tracking cameras positioning. Tracking cameras exterior orientation. Simultaneous acquisitions by the acquisition systems and tracking cameras. 14 / 35

30 Data processing Project data in previously defined coordinate system 15 / 35

31 Data processing Project data in previously defined coordinate system 15 / 35

32 Data processing Project data in previously defined coordinate system 15 / 35

33 Data processing Project data in previously defined coordinate system 15 / 35

34 Data processing Project data in previously defined coordinate system 15 / 35

35 16 / 35 Tracking and registration goals Sarcophagus configuration Area of interest 40 cm 70 cm Target accuracy 1/2 pixel 1/2 voxel Gom FluxData Acquisition distance m Target spatial accuracy mm Target angular accuracy mrad

36 Simulations Simulation goals Define tracking material. Need of target frame and characteristics. Number and type of tracking cameras and lens. Define stable setups. Evaluate achievable tracking accuracy. 17 / 35

37 Material Target frame Aluminum frame of dimensions 50 cm 50 cm 50 cm covered with 80, 1 cm diameter targets. 18 / 35

38 Material Tracking cameras 5 Mpx AVT Stingray tracking cameras with 8 mm Pentax lens. 4 cameras are necessary in this configuration 19 / 35

39 Simulations Sarcophagus configuration (a) front view (b) side view 20 / 35

40 Simulations: procedure (a) (b) (c) Simulation steps Noise (a) Scene creation. Picture coordinates. (b) Image generation. Object coordinates. (c) Noise addition. Tracking cameras position. (d) Bundle adjustment. Tracking cameras orientation. 21 / 35

41 Simulations: results Sarcophagus configuration Simulations best-case realistic Pixel coordinates µm Target coordinates mm Tracking cameras position mm Tracking cameras orientation mrad Angular tracking accuracy mm Spatial tracking accuracy mrad 22 / 35

42 22 / 35 Simulations: results Sarcophagus configuration Simulations best-case realistic Pixel coordinates µm Target coordinates mm Tracking cameras position mm Tracking cameras orientation mrad Angular tracking accuracy mm Spatial tracking accuracy mrad Gom FluxData Spatial goal mm Angular goal mrad

43 Overview 1 Introduction 2 Method 3 Experiments Calibrations Tracking 3D registration 3D MS registration 4 Conclusion 23 / 35

44 Experiment description Configuration sarcophage Objet: 500 mm 800 mm plaster bas-relief 9 3D acquisitions 4 multispectral acquisitions 24 / 35

45 25 / 35 Calibration accuracy Calibration Experiment Simulations Realistic Best-case Unit Tracking cam. I.O pixel Tracking cam. E.O mm mrad Frame calibration mm Frame to Atos III mm orientation mrad MS cam. I.O pixel Frame to MS cam mm orientation mrad

46 Internal tracking accuracy spatial accuracy (mm) 0.14 target: FluxData MSC angular accuracy (mrad) 0.12 target: Atos III target: FluxData MSC simulations: realistic 0.15 target: Atos III simulations: best case simulations: realistic simulations: best case G1 G3 G5 G7 G9 G11 FD1 FD3 0 G1 G3 G5 G7 G9 G11 FD1 FD3 26 / 35

47 27 / 35 Taget frame position and orientation Y G1 G2 G3 G4 G5 G6 G7 G8 G9 G10 G11 G12 FD1 FD2 FD3 FD X

48 28 / 35 Taget frame position and orientation Y G1 G2 G3 G4 G5 G6 G7 G8 G9 G10 G11 G12 FD1 FD2 FD3 FD X

49 3D registration 9 out of 12 meshes 29 / 35

50 3D MS registration 30 / 35

51 Overview 1 Introduction 2 Method 3 Experiments 4 Conclusion 31 / 35

52 Conlusion 4 tracking cameras Calibrated cube Acquisition system Object under study 32 / 35

53 Conclusion A method suitable for registering data with no salient features. A registration precision independent from the content of the acquired data. A registration method which works for different optical sensors. A flexible solution suitable for different applications. 33 / 35

54 Perspectives Engineering improvements. Flexible, transportable setup can be used in situ. Data fusion. Analysis of high dimension multi-modal datasets. Development of adapted interactive visualization tools. Development of specific algorithms. 34 / 35

55 35 / 35

56 Thank You 35 / 35