Rectangling Panoramic Images via Warping

Transcription

1 Rectangling Panoramic Images via Warping Kaiming He Microsoft Research Asia Huiwen Chang Tsinghua University Jian Sun Microsoft Research Asia

2 Introduction Panoramas are irregular

3 Introduction Panoramas are irregular Rectangles are favored panoramas in panoramas in

4 Introduction Panoramas are irregular Rectangles are favored Rectangling the panoramas

5 Introduction Panoramas are irregular Rectangles are favored Rectangling the panoramas Cropping

6 Introduction Panoramas are irregular Rectangles are favored Rectangling the panoramas Cropping Inpainting content-aware fill

7 Introduction Panoramas are irregular Rectangles are favored Rectangling the panoramas Cropping Inpainting content-aware fill

8 Introduction Panoramas are irregular Rectangles are favored Rectangling the panoramas Cropping Inpainting Warping new our warping

9 Why Warping? distortion Panoramas are often distorted

10 Why Warping? Panoramas are often distorted Warping can be unnoticeable our warping

11 Why Warping? Panoramas are often distorted Warping can be unnoticeable Warping is robust shape manipulation image retargeting image projection video stabilization [Igarashi et al, SIGGRAPH 05] [Wang et al, SIGGRAPH Asia 08] [Carroll et al, SIGGRAPH 09] [Liu et al, SIGGRAPH 09]

12 Why Warping? Panoramas are often distorted Warping can be unnoticeable Warping is robust Rectangling via warping

13 Challenges Meshing irregular input boundary conditions?

14 Challenges Meshing irregular input boundary conditions Content-preserving boundary constraints shapes straight lines?

15 Solution: Local + Global local warping mesh global warping warped back

16 Local Warping Mesh-free

17 Local Warping longest missing boundary Mesh-free Seam Carving [Avidan & Shamir 07] known pix missing

18 Local Warping seam Mesh-free shift Seam Carving [Avidan & Shamir 07] insert a seam shift pixels

19 Local Warping seam Mesh-free shift Seam Carving [Avidan & Shamir 07] insert a seam shift pixels Seam Carving = Warping

20 Local Warping Mesh-free Seam Carving [Avidan & Shamir 07] insert a seam shift pixels Seam Carving = Warping seam carving (A video was removed when converting this ppt to pdf.)

21 Local Warping Mesh-free Seam Carving [Avidan & Shamir 07] insert a seam shift pixels Seam Carving = Warping grid mesh

22 Local Warping Mesh-free Seam Carving [Avidan & Shamir 07] insert a seam shift pixels Seam Carving = Warping warped back

23 Global Warping Mesh optimization min E(V) V: all vertexes

24 Global Warping Mesh optimization Boundary constraints E B V : hard data term

25 Global Warping Mesh optimization Boundary constraints Shape preservation E S V = V T LV as-similar-aspossible [Igarashi et al, SIGGRAPH 05] [Liu et al, SIGGRAPH 09] [Wang et al, SIGGRAPH 10] L: Laplacian smoothness term in warping

26 input boundary + shape detected lines [PAMI 10] boundary + shape + line

27 Line Preservation Lines in the same direction are rotated by the same θ [Chang & Chuang, CVPR 12] in a block detected lines

28 Line Preservation direction i Lines in the same direction are rotated by the same θ [Chang & Chuang, CVPR 12] direction j quantized directions (50 bins)

29 Line Preservation direction i Lines in the same direction are rotated by the same θ [Chang & Chuang, CVPR 12] θ i θ j direction j warped

30 Line Preservation Lines in the same direction are rotated by the same θ [Chang & Chuang, CVPR 12] Bind lines to mesh u warp e u rotate θ e bilinear V E L V, θ = V T L θ V L θ : Laplacian

31 Global Warping Mesh optimization E V, θ = E B + E S + E L Boundary constraints Shape preservation Line preservation Total energy fix θ update V fix V update θ

32 Global Warping Target rectangle input bounding box normalized scaling x : y 1:1

33 Results input

34 Results warp

35 Results input

36 Results warp

37 Results input

38 Results warp crop content-aware fill

39 Results input

40 Results warp

41 Results zoom-in output

42 Results zoom-in input

43 Results 16-Mp CPU 1-core 2s

44 Failure input

45 Failure warp

46 Conclusion New concept - rectangling via warping Unnoticeable, robust, and fast