Oblique Image Processing in SURE - First Experiments and Results

Transcription

1 EuroSDR / ISPRS Workshop, Southampton 2015 Oblique Image Processing in SURE - First Experiments and Results Mathias Rothermel

2 Outline» General Workflow» 2.5D Meshing» Mesh Texturing» 3D Workflow» First Results Benchmark Dataset

3 Algorithm Overview» Stereo Matching (coarse-to-fine SGM)» Multi-view forward intersection

4 2.5D Surface Meshing Using RQT s - Motivation» Nadir imagery: Facades heavily slanted, challenging for 3D reconstruction» For reconstruction at least two redundant observations are required, use image information of less observed surface areas for texturing» DSM tiles already available» 2.5D fast processing times

5 2.5D Surface Meshing Using RQT s» Triangulation of DSM data» Meshing of elevation data using Restricted Quadtrees [1] Fast Matching (crack-free) triangulations Error criterion for adding nodes are guaranteed to be satisfied [1]: Pajarola, R., Large scale Terrain Visualization Using the Restricted Quadtree Triangulation, Proc. Of the Conference on Visualization, pages

6 2.5D Surface Meshing Using RQT s - Construction» Construction of RQT: Each node corresponds to elevation in DSM Top-down approach Implementation using dependency graphs Only insert elevations if they contribute to geometry (error criterion)

7 2.5D Surface Meshing Using RQT s Error Criterion» Error criterion: Each elevation node d is evaluated w.r.t its dependencies d_1,d_2 Each node is assumed to be observed with a uncertainty ε Insert node only if not contained in uncertainty/noise band

8 8

9 2.5D Surface Meshing Using RQT s Facade Re-meshing» Problem of RQT-triangulations: Badly shaped triangles at depth discontinuities Problems for further processing steps, e.g. texturing» Re-meshing of facade triangles Identification based on triangle side ratio Subdivision Collapse based on error quadrics

10 10

11 Mesh-Texturing» Problem: Each face is seen in multiple views, how to select texture?» Blending: Visible seems due to inaccurate orientation Differences in image scale cause blurred textures» Texture from one view: Criterion for best view? For example nearest non-blurred view To avoid seams: neighboring faces should be textured from the same image» Cast problem as energy optimization problem: Waechter, M., Moehrle, N., Goesele, M Let There Be Color! Large-Scale Texturing of 3D Reconstruction. Computer Vision ECCV Pages

12 Mesh-Texturing» Data term: Size of triangle projection to the images Gradients of texture in projected face» Smoothness term, Potts model evaluating to 1 if labels l are» Solution minimizing the energy can be found using Graph cuts (e.g. alpha-expansion) Loopy belief propagation

13 Mesh-Texturing» Results are patches of triangles» Problem visible seams» Global color adjustment: Estimate an additive color correction value per patch for each channel Non-linear convex optimization problem, can be solved iteratively» Local adjustment using Poisson Editing

14 Example Nadir Flight (80/80, 10cm GSD) 14

15 Example Oblique/Nadir Flight (~8-14cm GSD, IGI Penta DigiCAM )

16 IGI Example Oblique/Nadir Flight (~8-14cm GSD, IGI Penta DigiCAM )

17 3D Processing Approach» Oblique imagery allow for reconstruction of 3D structure» For depth/disparity image based methods: How can data be fused of single depth/disparity maps be fused?» Challenges: Variances of precision due to variances in image scale, number of observations Variances of precision due to errors in stereo matching (fronto-parallel effects, pixel locking, image blur, ) Inaccurate orientation

18 3D Processing - Approach» DSM generation based on median filtering delivered acceptable results» Median-based 3D Fusion» Median Filter along surface normals» Tile-based processing Normal computation on depth images Spatial sorting of oriented points Tile-based fusion

19 3D Point Fusion Octree» Sort points into multi-level octree» A point p is element of a octree node n if its bounding sphere is fully contained by n.» Radius r of sphere is chosen as a*gsd» For further operations only consider points in leaf nodes

20 3D Point Fusion - Median Filtering» For each point p derive all points q_i located in tube» Tube radius r=gsd, tube height h=b*gsd or h=b*sigma» For all q_i possessing similar orientation perform median filtering of projections d_i along normal.» Iterative approach

21 3D Point Fusion Iterative Median Filtering Original Leaf Points 1. Iter. 3. Iter. Normal Height Normal

22 3D Point Fusion Iterative Median Filtering Original 3. Iteration

23 Resulting Point Clouds of the Benchmark Data Set Zeche Zollern» Joint submission of SURE (nframes) and IGIMatch (IGI)» 81 images, IGI Penta DigiCAM, 50MP» 80/80 Nadir average, GSD ~8-14cm» Configuration for matching 2 in-strip two cross-strip» Matching (10h), Triangulation(3.25h), Fusion(3.5h), Intel 6 3.3Ghz

24

25

26 Resulting Point Clouds of the Benchmark Data Set Zeche Zollern

27 Oriented Points» Oriented points (point + normal) are useful for meshing algorithms e.g. Poisson Reconstruction» Tile-wise fusion / meshing» Stitching of mesh tiles

28 Example Dortmund2

29 Example Dortmund 1

30 Thanks for your attention.