PIXEL2BRICK: Constructing Brick Sculptures from Pixel Art

Transcription

1 PIXEL2BRICK: Constructing Brick Sculptures from Pixel Art Ming-Hsun Kuo 1 You-En Lin 1 Hung-Kuo Chu 1 Ruen-Rone Lee 1 Yong-Liang Yang 2 1 National Tsing-Hua University, Taiwan 2 University of Bath

2 LEGO A brick-based toy construction system introduced in 1940s. A cheapest and convenient tool for 3D fabrication.

3 Related Work Lego: Automated model construction [Gower et al. 1998] Six heuristics Automated brick layout [Winkler 2005] Cellular automata with cell clustering [van Zijl and Smal 2008]

4 Related Work Automatic generation of constructable brick sculptures [Testuz et al. 2013] Legolization: Optimizing LEGO Designs [Sheng-Jie et al. 2015]

5 LEGO Made from Pixel Art

6 Pixel Art Video Post-it war Game Painting Tile Art

7 Challenge Build a LEGO Sculpture from Pixel Art is not easy

8 Challenge The sculpture may be unbalanced

9 Challenge A simple solution is to add bricks in the back.

10 Introduction Pixel2Brick : Convert Pixel Art image to LEGO model Pixel Art LEGO Model Balance Appearance Stability

11 Balance Make LEGO stand stably for better exhibition.

12 Appearance LEGO bricks only have 26 basic colors. The intrinsic color distribution of pixel art images should be well approximated in the brick sculptures.

13 Stability LEGO can be built with different patterns. A poor design may cause a weak stability or even crashed.

14 System Overview Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture

15 Generating Balanced Pixel Art Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture

16 Balance Test Whether centroid can be projected onto the supporting line. Centroid Supporting line

17 Expanding The Supporting Line Deform the original shape to elongate the supporting line.

18 Expanding The Supporting Line Deform the original shape to elongate the supporting line. 1. Identify candidates at local minima. 2. Incrementally move candidates to align with supporting line. As-rigid-as-possible shape deformation 3. Evaluate using E supp = L supp /E distort

19 Adjusting The Centroid Shift the centroid to project onto the expanded supporting line. 1. Move anchor mesh 2. Evaluate using E centroid = centroid /E distort

20 Adjusting The Centroid Shift the centroid to project onto the expanded supporting line. 1. Move anchor mesh 2. Evaluate using E centroid = centroid /E distort

21 Adding A Supporting Strut Fail in above two steps.

22 Mapping Colors from Pixels to Bricks Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture

23 Color Transform Color Similarity The assigned LEGO color should as similar as possible to the original color. X O Original Color LEGO Color Original Color LEGO Color

24 Color Transform Color Smoothness The two similar adjacent colors should also keep similar after transformed. X O Original Color LEGO Color Original Color LEGO Color

25 Mapping Colors from Pixels to Bricks Flood-filling Segmentation

26 Mapping Colors from Pixels to Bricks Multi-Label Graph Cut Color Similarity Color Smoothness C = arg min c i C [E data C + E smooth (C )] n E data = r i c i c i E smooth = i=1 i,j w ij c i c j

27 Constructing Stable Brick Sculpture Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture

28 Constructing Stable Brick Sculpture Fallen

29 Resolving Dangling Parts 1. Identify dangling parts. 2. Use a 2 x 2 window. 3. Traverse and classify all the patterns.

30 Resolving Dangling Parts Pixel-Level operations Add Brick Replace Color(s)

31 Resolving Dangling Parts Select the one that minimize: E dangling = w c E pixel + w p E pattern + w s E singularity Pixels Added Pattern similarity Prevent new nonstandard brick

32 Resolving Dangling Parts

33 Resolving Dangling Parts

34 Resolving Dangling Parts

35 Optimizing Brick Layout How to convert to LEGO bricks Even a simple shape can be built in many ways.

36 Quality of Seam Lego: Automated model construction [Gower et al. 1998] Seam The middle of the brick is better to be aligned to the seam.

37 Quality of Seam L l L r E coverage = L l L r /(L l + L r ) E coverage = 1 E seam = w o E coverage + w d E dangling Prevent new dangling bricks occur.

38 Optimizing Brick Layout Randomly select 2 layers and optimize.

39 Optimizing Brick Layout Propagate to other layers.

40 Optimizing Brick Layout

41 Results

42 Original Image

43 Color Transformed

44 LEGO Sculpture

45 Original Image

46 Color Transformed

47 LEGO Sculpture

48 Original Image

49 Color Transformed

50 LEGO Sculpture

51 Original Image

52 Color Transformed

53 LEGO Sculpture

54 Original Image

55 Color Transformed

56 LEGO Sculpture

57 Evaluation

58 Comparison with previous work 3D model-based LEGO designs Beam Search [Winkler 2005] Cellular Automata [van Zijl and Smal 2008] Graph-based method [Testuz et al. 2013] Evaluation Metric Quality of vertical seams Number of bricks with different sizes

59 3D Model-Based LEGO Designs Pixel Art Image

60 3D Model-Based LEGO Designs Beam Search [Win05] Pixel2Brick(Ours)

61 3D Model-Based LEGO Designs Cellular Automata [vzs08] Pixel2Brick(Ours)

62 3D Model-Based LEGO Designs Graph-based [TSP13] Pixel2Brick(Ours)

63 3D Model-Based LEGO Designs Beam Search [Win05] Cellular Automata [vzs08] Graph-based [TSP13] Pixel2Brick(Ours)

64 Quality of Vertical Seams Count Beam Search [Win05] Cellular Automata [vzs08] Coverage Error Graph-based [TSP13] Pixel2Brick(Ours)

65 Number of Bricks Count Beam Search [Win05] Cellular Automata [vzs08] Brick Size Graph-based [TSP13] Pixel2Brick(Ours)

66 Comparison With Previous Work Image-based mosaic design. PicToBrick A tool which converts squared images to 2D brick-like mosaic. Evaluation Metric Quality of vertical seams Number of bricks with different sizes Number of dangling parts

67 Image-Based Mosaic Design PicToBrick Pixel2Brick(Ours)

68 Image-Based Mosaic Design PicToBrick Pixel2Brick(Ours)

69 Conclusion Automatically generate brick sculptures from pixel art images. Present guidelines and algorithm concerning the appearance, stability, and balance. Experiments also demonstrate we can efficiently generate brick sculptures that are appealing, stable and constructable.

70 Limitation The aspect ratio of a unit LEGO brick is not equal to one. 9.6mm 8.0mm In some rare cases, thin parts could still make the structure vulnerable. Pixel Art LEGO

71 Q&A