PIXEL2BRICK: Constructing Brick Sculptures from Pixel Art Ming-Hsun Kuo 1 You-En Lin 1 Hung-Kuo Chu 1 Ruen-Rone Lee 1 Yong-Liang Yang 2 1 National Tsing-Hua University, Taiwan 2 University of Bath
LEGO A brick-based toy construction system introduced in 1940s. A cheapest and convenient tool for 3D fabrication.
Related Work Lego: Automated model construction [Gower et al. 1998] Six heuristics Automated brick layout [Winkler 2005] Cellular automata with cell clustering [van Zijl and Smal 2008]
Related Work Automatic generation of constructable brick sculptures [Testuz et al. 2013] Legolization: Optimizing LEGO Designs [Sheng-Jie et al. 2015]
LEGO Made from Pixel Art
Pixel Art Video Post-it war Game Painting Tile Art
Challenge Build a LEGO Sculpture from Pixel Art is not easy
Challenge The sculpture may be unbalanced
Challenge A simple solution is to add bricks in the back.
Introduction Pixel2Brick : Convert Pixel Art image to LEGO model Pixel Art LEGO Model Balance Appearance Stability
Balance Make LEGO stand stably for better exhibition.
Appearance LEGO bricks only have 26 basic colors. The intrinsic color distribution of pixel art images should be well approximated in the brick sculptures.
Stability LEGO can be built with different patterns. A poor design may cause a weak stability or even crashed.
System Overview Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture
Generating Balanced Pixel Art Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture
Balance Test Whether centroid can be projected onto the supporting line. Centroid Supporting line
Expanding The Supporting Line Deform the original shape to elongate the supporting line.
Expanding The Supporting Line Deform the original shape to elongate the supporting line. 1. Identify candidates at local minima. 2. Incrementally move candidates to align with supporting line. As-rigid-as-possible shape deformation 3. Evaluate using E supp = L supp /E distort
Adjusting The Centroid Shift the centroid to project onto the expanded supporting line. 1. Move anchor mesh 2. Evaluate using E centroid = centroid /E distort
Adjusting The Centroid Shift the centroid to project onto the expanded supporting line. 1. Move anchor mesh 2. Evaluate using E centroid = centroid /E distort
Adding A Supporting Strut Fail in above two steps.
Mapping Colors from Pixels to Bricks Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture
Color Transform Color Similarity The assigned LEGO color should as similar as possible to the original color. X O Original Color LEGO Color Original Color LEGO Color
Color Transform Color Smoothness The two similar adjacent colors should also keep similar after transformed. X O Original Color LEGO Color Original Color LEGO Color
Mapping Colors from Pixels to Bricks Flood-filling Segmentation
Mapping Colors from Pixels to Bricks Multi-Label Graph Cut Color Similarity Color Smoothness C = arg min c i C [E data C + E smooth (C )] n E data = r i c i c i E smooth = i=1 i,j w ij c i c j
Constructing Stable Brick Sculpture Input Image Balanced Pixel Art Color Transform Stable Brick Sculpture
Constructing Stable Brick Sculpture Fallen
Resolving Dangling Parts 1. Identify dangling parts. 2. Use a 2 x 2 window. 3. Traverse and classify all the patterns.
Resolving Dangling Parts Pixel-Level operations Add Brick Replace Color(s)
Resolving Dangling Parts Select the one that minimize: E dangling = w c E pixel + w p E pattern + w s E singularity Pixels Added Pattern similarity Prevent new nonstandard brick
Resolving Dangling Parts
Resolving Dangling Parts
Resolving Dangling Parts
Optimizing Brick Layout How to convert to LEGO bricks Even a simple shape can be built in many ways.
Quality of Seam Lego: Automated model construction [Gower et al. 1998] Seam The middle of the brick is better to be aligned to the seam.
Quality of Seam L l L r E coverage = L l L r /(L l + L r ) E coverage = 1 E seam = w o E coverage + w d E dangling Prevent new dangling bricks occur.
Optimizing Brick Layout Randomly select 2 layers and optimize.
Optimizing Brick Layout Propagate to other layers.
Optimizing Brick Layout
Results
Original Image
Color Transformed
LEGO Sculpture
Original Image
Color Transformed
LEGO Sculpture
Original Image
Color Transformed
LEGO Sculpture
Original Image
Color Transformed
LEGO Sculpture
Original Image
Color Transformed
LEGO Sculpture
Evaluation
Comparison with previous work 3D model-based LEGO designs Beam Search [Winkler 2005] Cellular Automata [van Zijl and Smal 2008] Graph-based method [Testuz et al. 2013] Evaluation Metric Quality of vertical seams Number of bricks with different sizes
3D Model-Based LEGO Designs Pixel Art Image
3D Model-Based LEGO Designs Beam Search [Win05] Pixel2Brick(Ours)
3D Model-Based LEGO Designs Cellular Automata [vzs08] Pixel2Brick(Ours)
3D Model-Based LEGO Designs Graph-based [TSP13] Pixel2Brick(Ours)
3D Model-Based LEGO Designs Beam Search [Win05] Cellular Automata [vzs08] Graph-based [TSP13] Pixel2Brick(Ours)
Quality of Vertical Seams Count Beam Search [Win05] Cellular Automata [vzs08] Coverage Error Graph-based [TSP13] Pixel2Brick(Ours)
Number of Bricks Count Beam Search [Win05] Cellular Automata [vzs08] Brick Size Graph-based [TSP13] Pixel2Brick(Ours)
Comparison With Previous Work Image-based mosaic design. PicToBrick A tool which converts squared images to 2D brick-like mosaic. Evaluation Metric Quality of vertical seams Number of bricks with different sizes Number of dangling parts
Image-Based Mosaic Design PicToBrick Pixel2Brick(Ours)
Image-Based Mosaic Design PicToBrick Pixel2Brick(Ours)
Conclusion Automatically generate brick sculptures from pixel art images. Present guidelines and algorithm concerning the appearance, stability, and balance. Experiments also demonstrate we can efficiently generate brick sculptures that are appealing, stable and constructable.
Limitation The aspect ratio of a unit LEGO brick is not equal to one. 9.6mm 8.0mm In some rare cases, thin parts could still make the structure vulnerable. Pixel Art LEGO
Q&A