Mesh Compression CS1 - Meshing Triangle Meshes Two main parts: Connectivity: Often, triangulated graph Sometimes, polygons 3-connected graphs Geometry: Positions in space 2 Basic Definitions (I) Basic Definitions (II) Vertex: node V vertices Edge: between 2 vertices - E edges Face: face between edges - F faces Valence of a vertex: number of emanating edges Euler Formula: F E + V = 2(c-g) b c: # of connected components g: genus b: # of boundary edges Degree of a face: number of edges around 3 4 History of Multimedia Compression Success of Digital Signal Processing (DSP) Sounds, images, videos Technology followed scientific evolution Compression allows small download times MP3, JPEG, MPEG (De)Compression of 3D Meshes: Huge meshes Arbitrary topology Irregular connectivity Non-uniform sampling text 70 80 90 00 sound images video geometry I am too fat. 5 Page 1
A Common Format: VRML Geometry - Naïve Encoding V R M L Vertices Faces (geometry) (connectivity) v1 (x1;y1;z1) f1 (v1;v3;v2) v2 (x2;y2;z2) f2 (v4;v3;v1) v3 (x3;y3;z3) f3 (v4;v1;v5) v4 (x4;y4;z4) f4 (v1;v;v5) v5 (x5;y5;z5) f5 (v;v1;v7) v (x;y;z) f (v2;v7;v1) v7 (x7;y7;z7) f7 ( ) Vertices (geometry) v1 (x1;y1;z1) v2 (x2;y2;z2) v3 (x3;y3;z3) v4 (x4;y4;z4) v5 (x5;y5;z5) v (x;y;z) v7 (x7;y7;z7) Naïve: 3*float 32 = 9b/v Direct (quantization): 3*bits = 30b/v Valence n: n redundancies average 7 8 Connectivity -Naïve Encoding Faces (connectivity) f1 (v1;v3;v2) f2 (v4;v3;v1) f3 (v4;v1;v5) f4 (v1;v;v5) f5 (v;v1;v7) f (v2;v7;v1) f7 ( ) Naïve: 3*int 32 = 9 bit/face 192 b/v Direct: 3*log 2 (V) bit/face 50 b/f for 0k vertices 0 b/v And Yet Far from Optimal From a census of planar maps [Tutte 4], ( ) #triangulation(v) = 2 4V + 1! ( 3V + 2)!( V + 1)! For large V (>00), 25 V log V 25 27 3.24V #triangulation(v) 2 2 = 2 27 Theoretical connectivity bound: 3.24b/v 9 A Look at Data Compression Lossless Encoding Data compression: Storing data in a format that requires less space than usual = packing data Improves: Storage Big file Transmission Retrieval Small file Data encoding 000000001 = decoding Well suited for text, files, medical data, Goal: Remove all redundancy Data 12 Page 2
Lossy Encoding The Notion of Entropy Data 000001 Data First, decompose a data set into a sequence of events. encoding!= decoding Well suited for sound, images, videos, etc Entropy: Smallest possible expected numbers of bits needed to encode a sequence of events. Goal: Remove unnoticeable features, then all redundancy p i = probability of occurrence of event i 13 14 Entropy Encoding Run-Length Encoding The game: Run: consecutive values Find bit-rate closest to entropy Example: The idea: Give shortest codewords to most probable events. Sequence: R T A A A A A A S D E E E E E Two problems: 1. How to turn this notion into an algorithm? 2. How to benefit from statistical modeling? Replaced by: R T *A S D *5E 15 1 Dictionary methods Huffman Encoding The idea: many data types contain repeating patterns find an earlier occurrence in the input data, and only output a pointer/length. Generate codes with variable length - Higher probabilities shorter code words - Lower probabilities longer code words Extensions: - combined with Huffman, etc. [Lempel-Ziv 77] lha, arc, arj, zip Symbol Code word A 0 B C 0 D 1 Cons: - Integer code length - Must transmit probabilities Extensions: - Adaptive (non stationary data) - Extended (groups of symbols) 17 18 Page 3
Arithmetic Encoding Pros: - Non-integer code length - Optimal for large #symbols -> (entropy + ε) b/symbol Cons: - Must transmit probabilities Extensions: - adaptive (avoid transmission of probabilities) - order-n: inter-symbol probability - stack-run: coupled with detection of runs 19 Statistical Modeling Massaging the data can pay off -Delta-encoding -Prediction -Transform Prior knowledge on the data helps 20 Example of Lossy encoding Typically: 1) Transform DCT Wavelets Bloc sorting, Distance coding, etc. 2) Quantization Scalar Vector 3) [Detection of runs] 4) Entropy encoding Needs in 3D Encoding (I) Order of transmission: Single-rate coders: Encode a mesh in one single bitstream. Also called single-resolution coders. Progressive coders: Encode a mesh as successive refinements. Coarse-to-fine style. 21 22 Needs in 3D Encoding (I) Order of transmission: Single-rate Progressive T r a n s m i s s i o n 23 Needs in 3D Encoding (II) Integrity of encoding: Lossless coders: Preserve connectivity and/or geometry. For: collaborative design, games, medical data Lossy coders: Optimize the ratio rate/distortion For: geometry over the internet, e-commerce 24 Page 4
Needs in 3D Encoding (III) Part I Additional features: Resilience: robust to packet loss Efficiency: fast, or low memory requirements Guaranteed bounds: ensuring max bit rate Scalability: max speed vs min bit rate Etc Single-Rate Codecs 25 2 Redundancy and VRML Naïve Encoding Vertices (geometry) v1 (x1;y1;z1) v2 (x2;y2;z2) v3 (x3;y3;z3) v4 (x4;y4;z4) v5 (x5;y5;z5) v (x;y;z) v7 (x7;y7;z7) Valence n ~ redundancy -> average V R M L Faces (connectivity) f1 (v1;v3;v2) f2 (v4;v3;v1) f3 (v4;v1;v5) f4 (v1;v;v5) f5 (v;v1;v7) f (v2;v7;v1) f7 ( ) 27 Connectivity Geometry v1 (x1;y1;z1) f1 (v1;v3;v2) v2 (x2;y2;z2) f2 (v4;v3;v1) v3 (x3;y3;z3) f3 (v4;v1;v5) v4 (x4;y4;z4) f4 (v1;v;v5) v5 (x5;y5;z5) f5 (v;v1;v7) v (x;y;z) f (v2;v7;v1) v7 (x7;y7;z7) f7 ( ) -> 3V float -> 3F log 2 (V) ~ V log 2 (V) 9 b/v ~ 0 b/v to give an order of magnitude V = 50k 28 Single-Rate Encoding Link to Other Fields Review: 3 main approaches based on: Triangle strips Edge/gate traversal Valence Recent developments Improvement Hint of optimality In Graph Theory, some previous work Graphs can be huge, so encoding is good But mainly designed to compress (nearly) complete graphs Far away from our 3-connected graphs Different approaches using Lipton-Tarjan for instance [DL98] 29 30 Page 5
Triangle Strips Deering 95 (I) Connectivity: - Generalized triangle strips - Vertex buffer (size: 1) Idea: use stack operations to reuse vertices. reduce random access to vertices Geometry: - Quantization, Delta coding and Huffman 31 32 Deering 95 (II) Deering 95 (III) Features: -Fast -Local well suited for hardware - 8- b/v for connectivity - Geometry: highly variable Redundancy ~ 2 One symbol per face 2V symbols - Integrated in JAVA 3D 33 34 Taubin & Rossignac 98 (I) Taubin & Rossignac 98 (II) The idea: Cut a mesh by using 2 interlocked trees: - a spanning tree of vertices, - a spanning tree of triangles. Geometry: -Prediction from multiple ancestors of the vertex tree Connectivity: - Spanning trees - Marching pattern zigzag walking 35 Features: - average: 4 b/v for connectivity vertex tree 3 Page
Gumhold & Straβer 98 (I) The idea: use an active boundary with a current gate, and encode the adjacent face insertion using 8 distinct symbols. Gumhold & Straβer 98 (II) Connectivity: -"Cut-border" machine One symbol per processed edge, i.e. per conquered face F symbols ~ 2V Features: - average ~ 4.3 b/v for connectivity - Focus on speed (especially decompression) 37 38 Rossignac 99 (I) -EdgeBreaker Rossignac 99 (II) - EdgeBreaker The idea: Use an active boundary with a current gate, and encode the adjacent face insertion using 5 distinct symbols. F symbols ~ 2V Connectivity: - Spanning tree - Gate traversal Features: -4 b/v guaranteed for connectivity Overall 2V symbols (one symbol per face) V "C" symbols (creates one vertex) 1 bit (0) V other symbols among [RLSE] 3 bits (1..) Later improved 39 40 Improvements Rossignac & Szymczak 99 Wrap&Zip: Linear decompression of triangle meshes King & Rossignac 99 Guaranteed 3.7V Bit Encoding of Planar Triangle Graphs Szymczak, King & Rossignac 00 An Edgebreaker-based Efficient Compression Scheme for Regular Meshes Isenburg & Snoeyink 00 Spirale Reversi: Reverse Decoding of EdgeBreaker Encoding Gumhold 00 New Bounds on the Encoding of Planar Triangulations. (3.55 b/v for EdgeBreaker) 41 Valence-based Approaches Touma & Gotsman 98 One valence code per vertex Successive pivot vertices Isenburg & Snoeyink 99 Edge collapses One valence code per removed vertex Compression ratios equivalent to Alliez & Desbrun 01 Improvements upon Theoretical study Improvements 42 Page 7
Valence Distribution (I) Valence Distribution (II) 8000 90000 7000 80000 70000 000 0000 5000 50000 4000 40000 3000 30000 2000 20000 00 000 0 0 1 2 3 4 5 7 8 9 12 13 14 15 1 17 0 0 1 2 3 4 5 7 8 9 12 13 14 15 1 17 Calls for entropy encoding 43 44 Valence Distribution (III) Touma & Gotsman 98 (I) 18000 The idea: 00 14000 12000 000 Deterministic edge conquering from successive pivot vertices along an active edge list. Atomic edge conquest -> valence symbol 8000 000 Connectivity: 4000 2000 0 1 2 3 4 5 7 8 9 - Output one valence symbol per vertex, + ε - Huffman encoding of valences Full pivot vertex -> no code, hence V codes 45 Split(offset) code (of rare occurrence) 4 Touma & Gotsman 98 (II) Touma & Gotsman 98 (III) Geometry: decoded area - Global quantization and parallelogram prediction - Entropy encoding Features: free area current pivot - average: 2 b/v for connectivity - benefits from mesh regularity conquered area conquest list 47 - bit-rate vanishes to zero when regular - not seriously challenged since 98 Parallelogram rule 48 Page 8
Isenburg & Snoeyink 99 (I) Isenburg & Snoeyink 99 (II) The idea: Sequence of reversible edge contractions, recordable through valence codes for simple digons, and start/end for complex/trivial. Features: - Two faces conquered per valence, hence V codes - Connectivity: slightly more than TG98 [1-4] b/v Connectivity: - Also benefits from mesh regularity - Successive edge collapses - Output one valence code/v, + ε - Entropy encoding of valences Valence End Valence Start No split codes, but start/end 49 50 Alliez & Desbrun 01 (Eurographics 01) The Split Case Connectivity: -Derived from TG98 - Output one valence code per vertex, + ε - Arithmetic encoding of valences Split(3) Idea: adaptive edge conquering from successive pivot vertices along an active edge list. Features: - Minimize # split codes - Theoretical study: Hint of optimality 51 Split(offset) 52 Intuition behind Split Codes Towards Adaptive Conquest 53 One remaining degree of freedom in TG choice in the pivot valence min / average valence min 54 Page 9
Adaptive Conquest Results TG 98 [5,5,,,split(2)] AD 01 [3,,] 55 5 Discussion TG98 not really challenged since 98 valence approach seems optimal But how to prove it? Valence as Entropy (I) We want to maximize: e = p i log 2 (1/ p i ) i with the constraints: p i = v i /V, so: p i = 1 i F E + V = 1 2E = 3F 2E V, so: i i p i = Solution: Lagrange Multipliers 57 58 Valence as Entropy (II) Worst-case distribution p i is such as: Max pi log 2 (1/ pi ) + λ pi 1 + μ i pi i i i Necessary condition: log 2 ( p i ) = λ 1/ ln(2) + μi We find: p i = 1 27 4 3 i 59 Valence as Entropy (III) Now: e = p i log2 (1/ p i ) = log2(25 / 27) i Worst-case entropy: log 2 (25/27)=3.24 Matches Tutte s census! Did we reach optimality? Well, maybe Splits are a pain (best solution: Kälberer 05) Expected behavior is sublinear, though Only asymptotical 0 Page
Some connectivity results Main advantages of Valence 1.98 bit/vertex 1.02 0.024 2.20 Coding valences guarantees: Optimality in the regular case (.) Typically, less than 0.01 bit per vertex (Near) Optimality in the irregular cases Never more than 3.5bpv 2.71 2.27 1.88 1 2 Method Note Connectivity (b/v) Quad/Polygon/Tetrahedron Deering 95 Triangle strips Redundancy ~ 2 8- Taubin & Rossignac 98 Spanning trees ~ 4 Gumhold & Straβer 98 Cut-border 2V codes ~ 4.3 Touma & Gotsman 98 Valence V codes ~ 2 (~0 when regular) Rossignac 99 2V codes, upper bound 4 guaranteed King & Rossignac 99 Gate traversal Better upper bound 3.7 grntd. Gumhold 00 Even better upper bound 3.55 grntd. Isenburg & Snoeyink 99 Valence V codes ~ TG98 Valence + Alliez & Desbrun 01 adaptive conquest Proof of optimality? Tutte 2 Quads King, Rossignac & Szymczak 99 (~EdgeBreaker) Polygons Isenburg & Snoeyink 00 (Face Fixer, edge-based) Kronrod& Gotsman00(Face-based) Khodakovsky et al 03 (Face and vertex based) Tetrahedra Gumhold, Guthe & Straβer 99 -> ~2 b/tet. 3 4 Part II Progressive Mesh Encoding Definition: Progressive Codecs Produces a multi-resolution format Coarse model + refinements Concept: Creates a bit stream with decreasingly important details. 5 Page
An Ideal Progressive Encoder Review Distortion worse better For each bit-rate, max ratio: perceptual quality transmitted information Bit-rate Progressive Meshes 9 Progressive Forest Split 98 Compressed Progressive Meshes 99 Cohen-Or et al. 99 Geometry-based triangulation 00 Spectral Compression 00 Complete Remeshing 00 Valence-centered centered 01 Plus recent developments Bits must be sent in decreasing order of innovation 7 8 Common idea Encoding a mesh refinement consists in: 1. Localization "where" 2. Action "how to refine" Exemple: localisation Hoppe 9 1. Localization each vertex to split ~ log 2 (V) bits to localize 2. Action Connectivity: 2edgestosplit log 2 (Cv) 1 b/v Geometry: delta encoding ~20 b/v Encoding (decimation) Decoding (refinement) action 9 70 Taubin et al. 98 Prog. Forest Split Remeshing holes Localization : ~ forest of vertices to split (1 b/edge) ~ edge collapses grouped in batches Action : - remeshing holes ~ 3.5 b/v - prediction and pre/post smoothing for geometry Atomic operator: edge collapse Forest edges Cut by triangulating tree loops Encoding using a variation of [TR98] (single-rate) Connectivity: : b/v Geometry: : 20 b/v Triangulate tree loops Refinement done 71 72 Page 12
Pajarola and Rossignac 99 (CPM) Metrics in CPM Localization : - 2-coloring of vertices to group vertex splits into batches amortized cost: 3 b/v Action : - Indicates two edges among the valence of each vertex to split - "Butterfly" prediction for geometry Atomic operator: edge collapse Batches of edge collapses - Metrics: modified QEM (GarlandHeckbert97( GarlandHeckbert97) Idea: select a subset of the less expensive edges that do not violate independent constraints greedy edge selection by increasing approximation error (instead of priority queue used by GH97). Connectivity: : 7.2 b/v Geometry: : 15.4 b/v 73 74 Cohen-Or et al. 99 2-coloring Localization : Atomic operator: vertex removal ~ 2 or 4 face coloring (build an independent set) 2 coloring: 2 b/v (many degree 5 and >) 4 coloring: 4 b/v (degree 4 or 3) LZ encoding of face labels Action : - deterministic retriangulation (inside-z) - barycentric prediction for geometry 4- coloring Coding Decoding Connectivity: : b/v Geometry: : 15.4 b/v 2- coloring 75 CS1b Deterministic - Meshing inside-z triangulation / 2-coloring 7 Discussion Alliez & Desbrun 01 (SIGGRAPH 01) 4 coloring 4-coloring We seek: Per-vertex granularity One valence code per vertex, ideally Detection of uniformity/regularity Good rate/distortion tradeoff We need: Mesh simplification 2-coloring Long thin triangles 77» atomic decimation/refinement operator» error metric No extra how to refine data 78 Page 13
Core ideas Localization : ~ at no cost gate-based conquest Action : valence of inserted vertex, adaptive remeshing table, at no cost. Valence-centered Approach? We showed (Eurographics 01) that valence leads to optimal connectivity encoding Occurrences 8000 7000 000 5000 4000 3000 2000 00 0 1 2 3 4 5 7 8 9 121314 Valence 79 80 Main Ideas Connectivity: Only one valence per vertex Sufficient for both localization and action Close to optimal compression [AD 01] Geometry: Normal/tangential separation Separate geometry/parameterization [KSS 00] Our Method at a Glance Decimation Strategy Passes of vertex removals Automatic re-triangulation Entropy Encoding Compression of the list of symbols (essentially valences) 81 82 Basic Primitives Gate (oriented edge) Ordinary patch 1 input gate N-1 output gates 1 vertex removal Null patch 1 input gate 2 output gates 0 vertex removal 83 Decimation Strategy Gate-based deterministic conquest Vertex removal Fifo of gates Eliminate localization cost Targeting special vertices Low valences to respect balance (Denny - Sohler 97) Cosmetic decisions Automatic re-triangulation - Favor regular remeshing Look-up table 84 Page 14
Example of Conquest Conquest - Step 2 Codes Fifo Codes Fifo 5 g1 g2 g3 g4 g5 85 5 g2 g3 g4 g5 g 8 Conquest - Step 3 Conquest - Step 4 Codes Fifo Codes Fifo 5 g3 g4 g5 g g7 87 5 g4 g5 g g7 g8 88 Conquest - Step 5 Conquest - End Codes Fifo Codes Fifo 5 g5 g g7 g8 g9 89 5 4 5 90 Page 15
Decimation Strategy Gate-based deterministic conquest Vertex removal Fifo of gates Eliminate localization cost Targeting special vertices Low valences to respect balance [Denny - Sohler 97] Cosmetic decisions Automatic re-triangulation - Favor regular remeshing Look-up table 91 before after Valence Dispersion Before : V = (valences, central vertex s excluded) After : V' = (valences) remove only valences [3-] V' = V + (valence(s)-) [3-4] on boundaries 92 Vertex Selection Null patch if: Valence > Already visited Metric-related decision Valence > Already visited Decimation Strategy Gate-based deterministic conquest Vertex removal Fifo of gates Eliminate localization cost Targeting special vertices Low valences to respect balance (Denny - Sohler 97) Cosmetic decisions Automatic re-triangulation - Favor regular remeshing Look-up table 93 94 Remeshing strategy Remeshing strategy 95 9 Page 1
Remeshing strategy Remeshing strategy 97 98 Remeshing strategy Remeshing strategy 99 Retriangulation Look-up Table Regular Decimation Targeting valences [3-] 3 [Kobbelt 00] Targeting valences [3] Substitution: 3-> Page 17
Irregular Decimation Entropy Encoding Targeting valences [3-] [3-;N] [;N] [3-;N] [;N] Targeting valences [3] Adaptive arithmetic encoding [Schindler 99] Substitution: 3-> Geometry encoding Normal / tangential separation Local Frenet frame Barycentric prediction ~ [Khodakovsky et al. 00] Results Results Decimation 19851 -> 4 vertices Decimation 3450 -> 24 vertices Quantization 12 bits Quantization bits Connectivity 4.1 b/v Connectivity 0.39 b/v Geometry 1.24 b/v Geometry 3.58 b/v 20.87 b/v 3.97 b/v Page 18
Rate / Distortion Discussion Distortion (%Bounding Box Mean square error) 20 217 vertices 5 4 3 2 1 0 0 20000 40000 0000 80000 0000 120000 140000 000 180000 Optimality? - One unique valence code per vertex - Does decimation hide entropy? non-optimal optimal tiling, valence > or metric decision lead to null patch codes (unused by refinement). rate (#bits) Perspectives Devillers & Gandoin 00 (I) Entropy-driven remeshing process, minimizing occurency of null patches (seek optimal tiling). Prediction on metric decision (regular sharp edges, corners) Inverse Loop in the regular case? Localization : ~ implicitly transmitted by the order on the data, via a space subdivision. ~ progressive quantization Action : - transmit only occurences - use geometric triangulation i.e. Delaunay/Crust/Natural Neighbors Drop the order upon the vertices -> > well suited to terrains Devillers & Gandoin 00 (II) Subdivision Transmission Theoretical analysis: Vertex precision: Q bits Direct encoding: VQ Occurrence-based approach: V(Q-log 2 V+2.402) -> worst case, i.e. uniformity Karni & Gotsman 00 - Spectral Compression The idea: Apply ~ Signal Processing theory Similar in spirit to lossy JPEG requires mesh partitionning (MeTiS) Then each submesh is turned into a compact linear combination of orthogonal basis functions Basis functions: eigenvectors of the topological Laplacian of a submesh -> > Recently improved by fixed spectral bases Page 19
Khodakovsky, Schröder & Sweldens 00 Wavelet Transform Progressive Geometry Compression We are not interested in the connectivity Let s remesh the object! Coarse mesh Then subdivision connectivity (regular) Effect of wavelet transform Changes distribution of coefficients Almost all coefficients close to zero Bottom-up: loop-based wavelet decomposition Top-to-bottom: Zero tree to locate efficiently the details 0.27 0.34 0.41 0.49 0.5 0.0 0.8 1.7 2.5 3.3 Vertex coordinates Wavelet coefficients Zero-Tree Results (I) Need tree structure for coefficients wavelets live on edges 139B 34B 8517B 24B 21K Test whole tree for significance split tree isolating significant coefs 701B 1358B 2784B K 03B Results (II) Next Challenges? 95B 2004B 480B 2191B 4 b/v 1253B 2804B 482B 14844B 2.5 b/v - Progressive polygon meshes - Progressive topology encoding - Progressive animated geometry - Multi-connected components - What is entropy of geometry? - Resilient encoding - 12 Page 20