Point-Based Computer Graphics

Transcription

1 Eurograhcs 003 Tutoral T1 Organzers ETH Zürch Hanseter Pfster MERL, Cambrdge Presenters Marc Alea TU Darmstadt Carsten Dachsbacher Unverstät Erlangen-Nürnberg ETH Zürch ETH Zürch Jeroen van Baar MERL, Cambrdge Matthas Zwcker ETH Zürch

2 Contents Tutoral Schedule... Presenters and Organzers Contact Informaton...3 References...4 Project Pages...5 Tutoral Schedule Introducton () Acquston of Pont-Samled Geometry and Aearance (Jeroen van Baar) Pont-Based Surface Reresentatons (Marc Alea) Pont-Based Renderng (Matthas Zwcker) Lunch Sequental Pont Trees (Carsten Dachsbacher) Effcent Smlfcaton of Pont-Samled Geometry () Sectral Processng of Pont-Samled Geometry () Pontsho3D: A Framework for Interactve Edtng of Pont-Samled Surfaces () Shae Modelng () Pontsho3D Demo () Dscusson (all)

3 Presenters and Organzers Contact Informaton Dr. Professor Deartment of Comuter Scence Swss Federal Insttute of Technology (ETH) CH 809 Zürch Swtzerland Phone: FAX: grossm@nf.ethz.ch htt://grahcs.ethz.ch Dr. Hanseter Pfster Assocate Drector MERL - A Mtsubsh Electrc Research Lab 01 Broadway Cambrdge, MA 0139 USA Phone: (617) Fa: (617) fster@merl.com htt:// Jeroen van Baar MERL - A Mtsubsh Electrc Research Lab 01 Broadway Cambrdge, MA 0139 USA Phone: (617) Fa: (617) jeroen@merl.com htt:// Matthas Zwcker Deartment of Comuter Scence Swss Federal Insttute of Technology (ETH) CH 809 Zürch Swtzerland Phone: FAX: zwcker@nf.ethz.ch htt://grahcs.ethz.ch Deartment of Comuter Scence Swss Federal Insttute of Technology (ETH) CH 809 Zürch 3

4 Swtzerland Phone: FAX: auly@nf.ethz.ch htt://grahcs.ethz.ch Dr. Marc Stammnger Unverstät Erlangen-Nürnberg Am Wechselgarten Erlangen Germany Phone: FAX: Marc.Stammnger@nformatk.un-erlangen.de Carsten Dachsbacher Unverstät Erlangen-Nürnberg Am Wechselgarten Erlangen Germany Phone: FAX: dachsbacher@nformatk.un-erlangen.de Dr. Marc Alea Interactve Grahcs Systems Grou Technsche Unverstät Darmstadt Fraunhoferstr Darmstadt Germany Phone: FAX: alea@grs.nformatk.tu-darmstadt.de htt:// References M. Alea, J. Behr, D. Cohen-Or, S. Fleshman, D. Levn, C. Slva. Pont set surfaces. Proceedngs of IEEE Vsualzaton 001,. 1-8, San Dego, CA, October 001. C. Dachsbacher, C. Vogelsang, M. Stammnger, Sequental ont trees. Proceedngs of SIGGRAPH 003, to aear, San Dego, CA, July 003. O. Deussen, C. Coldtz, M. Stammnger, G. Drettaks, Interactve vsualzaton of comle lant ecosystems. Proceedngs of IEEE Vsualzaton 00, Boston, MA, October 00. 4

5 W. Matusk, H. Pfster, P. Beardsley, A. Ngan, R. Zegler, L. McMllan, Imagebased 3D hotograhy usng oacty hulls. Proceedngs of SIGGRAPH 00, San Antono, TX, July 00. W. Matusk, H. Pfster, A. Ngan, R. Zegler, L. McMllan, Acquston and renderng of transarent and refractve objects. Thrteenth Eurograhcs Worksho on Renderng, Psa, Italy, June 00. M. Pauly, R. Keser, L. Kobbelt, M. Gross, Shae modellng wth ont-samled geometry, to aear, Proceedngs of SIGGRAPH 003, San Dego, CA, July 003. M. Pauly, M. Gross, Sectral rocessng of ont-samled geometry. Proceedngs of SIGGRAPH 001, , Los Angeles, CA, August 001. M. Pauly, M. Gross, Effcent Smlfcaton of Pont-Samled Surfaces. IEEE Proceedngs of Vsualzaton 00, Boston, MA, October 00. H. Pfster, M. Zwcker, J. van Baar, M. Gross, Surfels - surface elements as renderng rmtves. Proceedngs of SIGGRAPH 000, , New Orleans, LS, July 000. M. Stammnger, G. Drettaks, Interactve samlng and renderng for comle and rocedural geometry, Renderng Technques 001, Proceedngs of the Eurograhcs Worksho on Renderng 001, June 001. L. Ren, H. Pfster, M. Zwcker, Object sace EWA slattng: a hardware accelerated aroach to hgh qualty ont renderng. Proceedngs of the Eurograhcs 00, to aear, Saarbrücken, Germany, Setember 00. M. Zwcker, H. Pfster, J. van Baar, M. Gross, EWA volume slattng. Proceedngs of IEEE Vsualzaton 001,. 9-36, San Dego, CA, October 001. M. Zwcker, H. Pfster, J. van Baar, M. Gross, Surface slattng. Proceedngs of SIGGRAPH 001, , Los Angeles, CA, August 001. M. Zwcker, H. Pfster, J. van Baar, M. Gross, EWA slattng. IEEE Transactons on Vsualzaton and Comuter Grahcs. M. Zwcker, M. Pauly, O. Knoll, M. Gross, Pontsho 3D: an nteractve system for ont-based surface edtng. Proceedngs of SIGGRAPH 00, San Antono, TX, July 00 Project Pages Renderng htt://grahcs.ethz.ch/surfels Acquston htt:// 5

6 Sequental ont trees htt://www9.nformatk.un-erlangen.de/persons/stammnger/research Modelng, rocessng, samlng and flterng htt://grahcs.ethz.ch/onts Pontsho3D htt:// 6

7 Surf. Res. for Grahcs Tutoral T1 Herarchcal slnes Rase degree Wavelets Dscrete (ont based) reresentatons Mesh rocessng methods Marc Alea, Carsten Dachsbacher,,, Hanseter Pfster, Marc Stammnger, Jeroen Van Baar, Matthas Zwcker Subdvson schemes Add oerators Add connectvty Trangle meshes Polynomals... Pecewse lnear aromatons Irregular samlng of the surface Forget about arameterzaton Requre arameterzaton Dscontnuty modelng Toologcal fleblty Trangle meshes Multresoluton modelng Comresson Geometrc sgnal rocessng Refne h rather than! 3 Trangles... 4 From ecewse lnear functons to Delta dstrbutons Forget about connectvty Pont clouds Sohstcated modelng s dffcult (Local) arameterzatons stll needed Comle LOD management Comresson and streamng s hghly non-trval Ponts are natural reresentatons wthn 3D acquston systems Meshes rovde an artcfcal enhancement of the acqured ont samles Remove connectvty! Trangles -> Ponts 9 Smle and effcent reresentaton 9 Hardware elnes suort 9 Advanced geometrc rocessng s beng n sght 9 The wdely acceted queen of grahcs rmtves Polynomals -> Trangles 9 Rgorous mathematcal concet 9 Robust evaluaton of geometrc enttes 9 Shae control for smooth shaes 9 Advanced hyscally-based modelng 5 6 1

8 Hstory of Ponts n Grahcs Partcle systems [Reeves 1983] Ponts as a dslay rmtve [Whtted, Levoy 1985] Orented artcles [Szelsk, Tonnesen 199] Partcles and mlct surfaces [Wtkn, Heckbert 1994] Dgtal Mchelangelo [Levoy et al. 000] Image based vsual hulls [Matusk 000] Surfels [Pfster et al. 000] QSlat [Rusnkewcz, Levoy 000] Pont set surfaces [Alea et al. 001] Radal bass functons [Carr et al. 001] Surface slattng [Zwcker et al. 001] Randomzed z-buffer [Wand et al. 001] Samlng [Stammnger, Drettaks 001] Oacty hulls [Matusk et al. 00] Pontsho3D [Zwcker, Pauly, Knoll, Gross 00]...? The Purose of our Course s I) to ntroduce onts as a versatle and owerful grahcs rmtve II) to resent state of the art concets for acquston, reresentaton, rocessng and renderng of ont samled geometry III) to stmulate YOU to hel us to further develo Pont Based Grahcs 7 8 Taonomy Mornng Schedule Renderng (Zwcker) Pont-Based Grahcs Acquston (Pfster, Stammnger) Introducton () Acquston of Pont-Samled Geometry and Arearance (Jeroen van Baar) Pont-Based Surface Reresentatons (Marc Alea) Pont-Based Renderng (Matthas Zwcker) Reresentaton (Alea) Processng & Edtng (Gross, Pauly) 9 10 Afternoon Schedule Sequental ont trees (Carsten Dachsbacher) Effcent smlfcaton of ont-samled geometry () Sectral rocessng of ont-samled geometry () Pontsho3D: A framework for nteractve edtng of ont-samled surfaces () Shae modelng () Pontsho3D demo () Dscusson (all) 11

9 Acquston of Pont-Samled Geometry and Aearance Jeroen van Baar and Hanseter Pfster, MERL The Goal: To Cature Realty Fully-automated 3D model creaton of real objects. Fathful reresentaton of aearance for these objects. Wojcech Matusk, MIT Addy Ngan, MIT Paul Beardsley, MERL Remo Zegler, MERL Leonard McMllan, MIT Hanseter Pfster, MERL 1 Hanseter Pfster, MERL Image-Based 3D Photograhy An mage-based 3D scannng system. Handles fuzzy, refractve, transarent objects. Robust, automatc Pont-samled geometry based on the vsual hull. Objects can be rendered n novel envronments. Prevous Work Actve and assve 3D scanners Work best for dffuse materals. Fuzzy, transarent, and refractve objects are dffcult. BRDF estmaton, nverse renderng Image based modelng and renderng Reflectance felds [Debevec et al. 00] Lght Stage system to cature reflectance felds Fed vewont, no geometry Envronment mattng [Zongker et al. 99, Chuang et al. 00] Cature reflectons and refractons Fed vewont, no geometry Hanseter Pfster, MERL 3 Hanseter Pfster, MERL 4 Outlne Overvew System Geometry Reflectance Refracton & Transarency Acquston System Mult-Color Montor Lght Array Cameras Rotatng Platform Hanseter Pfster, MERL 5 Hanseter Pfster, MERL 6

10 Acquston Process Acquston Process Alha Mattes Vsual Hull Alha Mattes Vsual Hull Surface Lghtfeld Surface Reflectance Felds Surface Lghtfeld Surface Reflectance Felds Hanseter Pfster, MERL 7 Hanseter Pfster, MERL 8 Acquston Process Acquston Process Alha Mattes Vsual Hull Alha Mattes Vsual Hull Surface Lghtfeld Surface Reflectance Felds Surface Lghtfeld Surface Reflectance Felds Hanseter Pfster, MERL 9 Hanseter Pfster, MERL 10 Outlne Overvew System Geometry Reflectance Refracton & Transarency Acquston For each vewont ( 6 cameras 7 ostons ) Alha mattes Use multle backgrounds [Smth and Blnn 96] Reflectance mages Pctures of the object under dfferent lghtng (4 lghts 11 ostons) Envronment mattes Use smlar technques as [Chuang et al. 000] Hanseter Pfster, MERL 11 Hanseter Pfster, MERL 1

11 Geometry Oacty Hull Geometry Eamle Vsual hull: The mamal object consstent wth a gven set of slhouettes. Hanseter Pfster, MERL 13 Hanseter Pfster, MERL 14 Aromate Geometry The aromate vsual hull s augmented by radance data to render concavtes, reflectons, and transarency. Surface Lght Felds A surface lght feld s a functon that assgns a color to each ray orgnatng on a surface. [Wood et al., 000] Hanseter Pfster, MERL 15 Hanseter Pfster, MERL 16 Shadng Algorthm A vew-deendent strategy. Color Blendng Blend colors based on angle between vrtual camera and stored colors. Unstructured Lumgrah Renderng [Buehler et al., SIGGRAPH 001] Vew-Deendent Teture Mang [Debevec, EGRW 98] Hanseter Pfster, MERL 17 Hanseter Pfster, MERL 18

12 Pont-Based Renderng Pont-based renderng usng LDC tree, vsblty slattng, and vew-deendent shadng. Geometry Oacty Hull Store the oacty of each observaton at each ont on the vsual hull [Matusk et al. SIG00]. Hanseter Pfster, MERL 19 Hanseter Pfster, MERL 0 Geometry Oacty Hull Eamle Assgn vew-deendent oacty to each ray orgnatng on a ont of the vsual hull. A B C (θ,φ) φ Photo A B C Red = nvsble Whte = oaque Black = transarent Hanseter Pfster, MERL 1 θ Hanseter Pfster, MERL Eamle Eamle Photo Photo Vsual Hull Vsual Hull Oacty Hull Hanseter Pfster, MERL 3 Hanseter Pfster, MERL 4

13 Eamle Photo Surface Lght Feld Results Pont-based renderng usng EWA slattng, A-buffer blendng, and edge antalasng. Vsual Hull Oacty Hull Hanseter Pfster, MERL 5 Hanseter Pfster, MERL 6 Results Vdeo Results Vdeo Hanseter Pfster, MERL 7 Hanseter Pfster, MERL 8 Results Vdeo Results Vdeo Hanseter Pfster, MERL 9 Hanseter Pfster, MERL 30

14 Oacty Hull - Dscusson Vew deendent oacty vs. geometry trade-off. Sometmes acqurng the geometry s not ossble. Sometmes reresentng true geometry would be very neffcent. Oacty hull stores the macro effect. Pont-Based Models No need to establsh toology or connectvty. No need for a consstent surface arameterzaton for teture mang. Reresent organc models (feather, tree) much more readly than olygon models. Easy to reresent vew-deendent oacty and radance er surface ont. Hanseter Pfster, MERL 31 Hanseter Pfster, MERL 3 Outlne Overvew Prevous Works Geometry Reflectance Refracton & Transarency Lght Transort Model Assume llumnaton orgnates from nfnty. The lght arrvng at a camera el can be descrbed as: C (, y) = W ( ω) E( ω) dω Ω C(,y) E W - the el value - the envronment -the reflectance feld Hanseter Pfster, MERL 33 Hanseter Pfster, MERL 34 Surface Reflectance Felds 6D functon: W P, ω, ω ) = W ( u, v ; θ, Φ ; θ, Φ ) ( r r r r r Reflectance Functons For each vewont, 4D functon: W ω ) = W (, y; θ, Φ ) y ( (θ,φ ) r P φ θ Hanseter Pfster, MERL 35 Hanseter Pfster, MERL 36

15 Relghtng New Illumnaton Downsamle Surface reflectance feld = V0 V1 V Vn Comresson Subdvde mages nto 8 8 el blocks. Kee blocks contanng the object (avg. comresson 1:7) PCA comresson (avg. comresson 1:10) PCA a 0 a 1 a a 3 a 4 a 5 Hanseter Pfster, MERL 37 Hanseter Pfster, MERL 38 Results The Lbrary Hanseter Pfster, MERL 39 Hanseter Pfster, MERL 40 Surface Reflectance Felds Work wthout accurate geometry Surface normals are not necessary Cature more than reflectance Inter-reflectons Subsurface scatterng Refracton Dserson Non-unform materal varatons Smlfed verson of the BSSRDF Outlne Overvew Prevous Works Geometry Reflectance Refracton & Transarency Hanseter Pfster, MERL 41 Hanseter Pfster, MERL 4

16 Acquston We searate the hemshere nto hgh resoluton h and low resoluton l. L( C(, y) = Wh ( ξ ) T ( ξ ) dξ + W ) l ( ω ) L( ω ) dω Ωh T W h Ωl W l Acquston For each vewont ( 6 cameras 7 ostons ) Alha mattes Use multle backgrounds [Smth and Blnn 96] Reflectance mages Low resoluton Pctures of the object under dfferent lghtng (4 lghts 11 ostons) Envronment mattes Hgh resoluton Use smlar technques as [Chuang et al. 000] Hanseter Pfster, MERL 43 Hanseter Pfster, MERL 44 Low-Resoluton Reflectance Feld C(, y) = Ωh W ( ξ ) T ( ξ dξ + h ) Ωl W ( ω ) L( ω ) dω l Hgh-Resoluton Reflectance Feld C(, y) = Ωh W ( ξ ) T ( ξ dξ + h ) W ( ω ) L( ω ) dω Use technques of envronment mattng [Chuang et al., SIGGRAPH 00]. Ωl l Ωl W ( ω ) L( ω ) dω l n = 1 W L for n lghts Hanseter Pfster, MERL 45 Hanseter Pfster, MERL 46 Hgh-Resoluton Reflectance Feld Aromate W h by a sum of u to two Gaussans: Reflectve G 1. W ( ) a1g 1 ag Refractve G. h ξ = + N G 1 Reroject h Project envronment mattes onto the new envronment. Envronment mattes acqured was arameterzed on lane T (the lasma dslay). We need to roject the Gaussans to the new envronment ma, roducng new Gaussans. G W h T Hanseter Pfster, MERL 47 Hanseter Pfster, MERL 48

17 Vew Interolaton Render low-resoluton reflectance feld. Hgh-resoluton reflectance feld: Match reflected and refracted Gaussans. V 1 V ~ G 1r N ~ G r Interolate drecton vectors, not colors. Determne new color along nterolated drecton. ~ Gt ~ G 1t Results Performance for 67 = 43 vewonts 337,84 mages taken n total!! Acquston (47 hours) Alha mattes 1 hour Envronment mattes 18 hours Reflectance mages 8 hours Processng Oacty hull ~ 30 mnutes PCA Comresson ~ 0 hours (MATLAB, unotmzed) Renderng ~ 5 mnutes er frame Sze Oacty hull ~ MB Envronment mattes ~ GB Reflectance mages ~ Raw 370 GB / Comressed - 4 GB Hanseter Pfster, MERL 49 Hanseter Pfster, MERL 50 Results Hgh-resoluton Ω h Low-resoluton Ωl Combned Results Hanseter Pfster, MERL 51 Hanseter Pfster, MERL 5 Results Results Ω h Hanseter Pfster, MERL 53 Hanseter Pfster, MERL 54

18 Results Ω l Results Combned Hanseter Pfster, MERL 55 Hanseter Pfster, MERL 56 Results Results Hanseter Pfster, MERL 57 Hanseter Pfster, MERL 58 Conclusons Data drven modelng s able to cature and render any tye of object. Oacty hulls rovde realstc 3D grahcs models. Our models can be seamlessly nserted nto new envronments. Pont-based renderng offers hgh magequalty for dslay of acqured models. Future Drectons Real-tme renderng Done! [Vlasc et al., I3D 003] Better envronment mattng More than two Gaussans Better comresson MPEG-4 / JPEG 000 Hanseter Pfster, MERL 59 Hanseter Pfster, MERL 60

19 Acknowledgements Colleagues: MIT: Chrs Buehler, Tom Buehler MERL: Bll Yerazuns, Darren Legh, Mchael Stern Thanks to: Davd Tames, Jennfer Roderck Pfster NSF grants CCR and EIA-9800 Paers avalable at: htt:// Hanseter Pfster, MERL 61

20 Pont-based Surface Res Marc Alea, Carsten Dachsbacher,,, Hanseter Pfster, Marc Stammnger, Matthas Zwcker MarcAlea Dscrete Geometrc Modelng Grou Darmstadt Unversty of Technology Motvaton Many alcatons need a defnton of surface based on ont samles Reducton U-samlng Interrogaton (e.g. ray tracng) Desrable surface roertes Manfold Smooth Local (effcent comutaton) Overvew Introducton & Bascs Fttng Imlct Surfaces Projecton-based Surfaces 3 4 Introducton & Bascs Terms Regular/Irregular, Aromaton/Interolaton, Global/Local Standard nterolaton/aromaton technques Trangulaton, Vorono-Interolaton, Least Squares (LS), Radal Bass Functons (RBF), Movng LS Problems Shar edges, feature sze/nose Functonal -> Manfold Terms: Regular/Irregular Regular (on a grd) or rregular (scattered) Neghborhood (toology) s unclear for rregular data 5 6

21 Terms: Aromaton/Interolaton Nosy data -> Aromaton Terms: Global/Local Global aromaton Local aromaton Perfect data -> Interolaton Localty comes at the eense of smoothness 7 8 Trangulaton Elot the toology n a trangulaton (e.g. Delaunay) of the data Interolate the data onts on the trangles Pecewse lnear C0 Pecewse quadratc C1? Trangulaton: Pecewse lnear Barycentrc nterolaton on smlces (trangles) gven d+1 onts wth values f and a ont nsde the smle defned by Comute α from = Σ α and Σ α = 1 Then f = Σ α f 9 10 Vorono Interolaton Vorono Interolaton comutevoronodagram for any ont n sace add to Vorono dagram Vorono cell τ around ntersects orgnal cells τ of natural neghbors n nterolate f() = λ () wth τ τ λ () = τ T ( f + f ( )) λ () 11 τ n τ 3 n 3 τ 1 τ τ 4 n 1 n 4 τ 5 n 5 1

22 13 Proertes of Vorono Interolaton: lnear Precson local for d = 1 f() ecewse cubc f() C 1 on doman f(, 1,..., n ) s contnuous n Vorono Interolaton 14 Least Squares Fts a rmtve to the data Mnmzes squared dstances between the s and rmtve g ( ) ( ) g y g mn ) ( c b a g + + = 15 Least Squares - Eamle Prmtve s a olynomal Lnear system of equatons ( ) T g c =,..., 1, ) ( ( ) ( ) ( ) ( ) = T j T y y c c,..., 1, 0,..., 1, mn 16 Least Squares - Eamle Resultng system ( ) ( ) = = Μ Μ Ο Μ Κ ,..., 1, 0 y y y c c c T j y c 17 Radal Bass Functons Reresent nterolant as Sum of radal functons r Centered at the data onts ( ) ( ) = w r f 18 Radal Bass Functons Solve to comute weghts w Lnear system of equatons ( ) = j j y w r ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) = Μ Μ Ο Μ Λ y y y w w w r r r r r r r r r

23 Radal Bass Functons Solvablty deends on radal functon Several choces assure solvablty r d = d log (thn late slne) ( ) d ( ) / h d r d = e (Gaussan) h s a data arameter h reflects the feature sze or antcated sacng among onts Functon Saces! Monomal, Lagrange, RBF share the same rncle: Choose bass of a functon sace Fnd weght vector for base elements by solvng lnear system defned by data onts Comute values as lnear combnatons Proertes One costly rerocessng ste Smle evaluaton of functon n any ont 19 0 Functon Saces? Problems Many onts lead to large lnear systems Evaluaton requres global solutons Solutons RBF wth comact suort Matr s sarse Stll: soluton deends on every data ont, though dro-off s eonental wth dstance Local aromaton aroaches Sheard Interolaton Aroach forr d : f()= Σ φ () f wth bass functons φ () = defne f( ) := f = lm f() j j 1 Sheard Interolaton f() s a conve combnaton of φ, because all φ (R d ) [0,1] and Σ φ () 1. f() s contaned n the conve hull of data onts for >1 f() C and φ ( ) = 0 Data onts are saddles global nterolaton every f() deends on all data onts Only constant recson,.e. only constant functons are reroduced eactly Sheard Interolaton Localzaton: Set f()= Σ µ () φ () f ν wth ( 1 ) für = < R µ () R 0 sonst for reasonable R and ν >1 no constant recson because of ossble holes n the data 3 4

24 Satal subdvsons Satal subdvsons Subdvde arameter doman nto overlang cells τ wth centrods c Comute Sheard weghts c φ () = c j and localze them usng the radus of the cell Interolate/aromate data onts n each cell by an arbtrary functon f The nterolant s gven as f()= Σ µ () φ () f j 5 6 Movng Least Squares Comute a local LS aromaton at t Weght data onts based on dstance to t Movng Least Squares The set f () t = g ( t), g : mn ( g( ) θ( t ) t t g y s a smooth curve, ff θ s smooth ( g( ) ( t ) mn θ y t g ( ) = a + b + c 7 8 Movng Least Squares Tycal choces for θ: r θ( d ) = d d ( ) / h θ d = e Note: θ ( ) s fed = θ t For each t Standard weghted LS roblem Lnear ff corresondng LS s lnear Tycal Problems Shar corners/edges Nose vs. feature sze 9 30

25 Functonal -> Manfold Standard technques are alcable f data reresents a functon Manfolds are more general No arameter doman No knowledge about neghbors, Delaunay trangulaton connects non-neghbors Imlcts Each orentable n-manfold can be embedded n n+1 sace Idea: Reresent n-manfold as zeroset of a scalar functon n n+1 sace Insde: On the manfold: f ( ) < 0 f ( ) = 0 Outsde: f > ( ) Imlcts - Illustraton Image courtesy Greg Turk Imlcts from ont samles Functon should be zero n data onts f ( ) = 0 Use standard aromaton technques to fnd f Trval soluton: f = 0 Addtonal constrants are needed Imlcts from ont samles Imlcts from ont samles Constrants defne nsde and outsde Smle aroach (Turk, O Bren) Srnkle addtonal nformaton manually Make addtonal nformaton soft constrants Use normal nformaton Normals could be comuted from scan Or, normalshave to be estmated 35 36

26 Estmatng normals Estmatng normals Normal orentaton (Imlcts are sgned) Use nsde/outsde n nformaton from scan Normal drecton by fttng a tangent LS ft to nearest neghbors Weghted LS ft MLS ft q General fttng roblem mn q, n θ q, n = 1 ( ) Problem s non-lnear because n s constraned to unt shere n q Estmatng normals Imlcts from ont samles The constraned mnmzaton roblem mn n = 1 θ q, n s solved by the egenvector corresondng to the smallest egenvalue of ( q ) θ ( q ) θ ( q ) θ y z ( q ) ( ) ( ) y θ qy θ qy θ y z ( q ) ( ) ( ) z θ qz θ y qz θ z Comute non-zero anchors n the dstance feld Use normal nformaton drectly as constrants f ( n ) = Imlcts from ont samles Comute non-zero anchors n the dstance feld Comute dstances at secfc onts Vertces, md-onts, etc. n a satal subdvson Comutng Imlcts Gven N onts and normals, n and constrants f = 0, f c = d Let + N = c An RBF aromaton f ( ) ( ) ( ) = w r ( ) leads to a system of lnear equatons 41 4

27 Comutng Imlcts Practcal roblems: N > Matr soluton becomes dffcult Two solutons Sarse matrces allow teratve soluton Smaller number of RBFs Comutng Imlcts Sarse matrces Needed: ( 0) r( ) r( ) r( 1 ) r( 0) r( 1 ) 0 r( ) r( ) r( 0) c c Comactly suorted RBFs r Μ 0 d > c r( d) = 0, r'( c) = Λ Ο Comutng Imlcts Smaller number of RBFs Greedy aroach (Carr et al.) Start wth random small subset Add RBFs where aromaton qualty s not suffcent RBF Imlcts - Results Images courtesy Greg Turk RBF Imlcts - Results Images courtesy Greg Turk Hoe s aroach Use lnear dstance feld er ont Drecton s defned by normal In every ont n sace use the dstance feld of the closest ont 47 48

28 Hoe s aroach - smoother Drecton felds are nterolated usng Vorono nterolaton PuO Imlcts Construct a satal subdvson Comute local dstance feld aromatons e.g. Quadrcs Blend them wth local Sheard weghts PuO Imlcts: Shar features Mult-level PuO Imlcts Subdvde cells based on local error Corner functon Local analyss of onts and normals Edge functon Pecewse quadrc functons Standard quadrc 51 5 Mult-level PuO Imlcts Local comutatons Insenstve to number of onts Local adataton to shae comlety Senstve to outut comlety Mult-level PuO Imlcts Aromaton at arbtrary accuracy 53 54

29 Imlcts - Conclusons Scalar feld s underconstraned Constrants only defne where the feld s zero, not where t s non-zero Addtonal constrants are needed Sgned felds restrct surfaces to be unbounded All mlct surfaces defne solds Projecton Idea: Ma sace to surface Surface s defned as fonts of mang r r Surface defnton Surface Defnton Projecton rocedure (Levn) Local olyonmal aromaton Insred by dfferental geometry Imlct surface defnton Infntely smooth & Manfold surface r r Constructve defnton Inut ont r Comute a local n reference lane H r =<q,n> Comute a local olynomal over the lane G r Project ont r =G r (0) Estmate normal r q H r G r Local Reference Plane Local Reference Plane Fnd lane H r = q, n + D mn q, n θ q q, n = 1 ( ) / d h θ d = e h s feature sze/ ont sacng H r s ndeendent of r s dstance Manfold roerty ( ) n r q Weght functon based on dstance to q, not r H r Comutng reference lane Non-lnear otmzaton roblem Mnmze ndeendent varables: Overnfor fed dstance r q Along n for fed drecton n q changes -> the weghts change Only teratve solutons ossble n r n r q H r H r q 59 60

30 Local Reference Plane Projectng the Pont Practcal comutaton Mnmze over n for fed q Egenvalueroblem Translate q so that r = q + r q n Effectvely changesr q Mnmze along n for fed drecton n Elot artal dervatve n r n r q H r H r q MLS olyonomal over H r mn q, n G G Π d LS roblem r =G r (0) ( ( ) θ( q ) Estmate normal n H r r G r q H r 61 6 Satal data structure Regular grd based on suort of θ Each ont nfluences only 8 cells Each cell s an octree r Dstant octree cells are aromated by one ont n center of mass Conclusons Projecton-based surface defnton Surface s smooth and manfold Surface may be bounded Reresentaton error manly deends on ont densty Adjustable feature sze h allows to smooth out nose 63 64

31 Pont-Based Renderng Pont-Based Renderng Matthas Zwcker Comuter Grahcs Lab ETH Zürch Introducton and motvaton Surface elements Renderng Antalasng Hardware Acceleraton Conclusons Motvaton 1 Motvaton 1 Quake, k trangles Nvda, 00 mllons of trangles Performance of 3D hardware has eloded (e.g., GeForce4: 136 mllon vertces er second) Projected trangles are very small (.e., cover only a few els) Overhead for trangle setu ncreases (ntalzaton of teture flterng, rasterzaton) A smler, more effcent renderng rmtve than trangles? 3 4 Motvaton Modern 3D scannng devces (e.g., laser range scanners) acqure huge ont clouds Generatng consstent trangle meshes s tme consumng and dffcult Ponts as Renderng Prmtves Pont clouds nstead of trangle meshes [Levoy and Whtted 1985] D vector versus el grahcs A renderng rmtve for drect vsualzaton of ont clouds, wthout the need to generate trangle meshes? 4 mllon ts. [Levoy et al. 000] trangle mesh (wth tetures) ont cloud 5 6

32 Pont-Based Surface Reresentaton Ponts are samles of the surface The ont cloud descrbes: 3D geometry of the surface Surface reflectance roertes (e.g., dffuse color, etc.) There s no addtonal nformaton, such as connectvty (.e., elct neghborhood nformaton between onts) teture mas, bum mas, etc. Surface Elements - Surfels Each ont corresonds to a surface element, or surfel, descrbng the surface n a small neghborhood Basc surfels: BascSurfel { oston; color; } y z oston color 7 8 Surfels How to reresent the surface between the onts? holes between the onts Surfels need to nterolate the surface between the onts A certan surface area s assocated wth each surfel Surfels Surfels can be etended by storng addtonal attrbutes Ths allows for hgher qualty renderng or advanced shadng effects EtendedSurfel { oston; color; normal; radus; etc... } color radus surfel dsc normal oston 9 10 Surfels Surfels store essental nformaton for renderng Surfels are rmarly desgned as a ont renderng rmtve They do not rovde a mathematcally smooth surface defnton (see [Alea 001], ont set surfaces) Model Acquston 3D scannng of hyscal objects See Pfster, acquston Drect renderng of acqured ont clouds No mesh reconstructon necessary [Matusk et al. 00] 11 1

33 Model Acquston Samlng synthetc objects Effcent renderng of comle models Dynamc samlng of rocedural objects and anmated scenes (see Stammnger, dynamc samlng) Model Acquston Processng and edtng of ont-samled geometry [Zwcker et al. 001] [Stammnger et al. 001] sectral rocessng [Pauly, Gross 00] (see Gross, sectral rocessng) ont-based surface edtng [Zwcker et al. 00] (see Pauly, Pontsho3D) Pont Renderng Pelne Pont Renderng Pelne Pont Cloud Framebuffer Projecton Shadng Vsblty Image Reconstructon Projecton Shadng Vsblty Image Reconstructon Smle, ure forward mang elne Surfels carry all nformaton through the elne ( surfel stream ) No teture look-us Framebuffer stores RGB, alha, and Z Persectve rojecton of each ont n the ont cloud Analogous to rojecton of trangle vertces homogeneous matr-vector roduct ersectve dvson Pont Renderng Pelne Pont Renderng Pelne Projecton Shadng Vsblty Image Reconstructon Projecton Shadng Vsblty Image Reconstructon Per-ont shadng Conventonal models for shadng (Phong, Torrance-Sarrow, reflectons, etc.) Vsblty and mage reconstructon s tghtly couled Dscard onts that are occluded from the current vewont Reconstruct contnuous surfaces from rojected onts (antalasng) 17 18

34 Vsblty and Image Reconstructon wthout vsblty and mage reconstructon foreground ont occluded background ont surface dscontnuty ( hole ) wth vsblty and mage reconstructon Vsblty and Image Reconstructon Goal: avod holes and dscard occluded surfels Use surfel dscs wth radus r to cover surface comletely Aly z-buffer to dscard nvsble surfels 3D object sace normal radus r surfel dsc 19 0 Quad Renderng Prmtve Rasterze a colored quad centered at the rojected ont, use z-bufferng The quad sde length s h, where h = * r * s The scalng factor s gven by ersectve rojecton and vewort transformaton Hardware mlementaton: OenGL GL_POINTS colored quad rojected ont y screen sace } h Vsblty: Z-Bufferng No blendng of renderng rmtves z1 z1 >z{ z framebuffer y z el 1 Projected Dsc Renderng Prmtve Project surfel dscs from object to screen sace Projectng dscs results n ellses n screen sace Ellses adat to the surface orentaton screen sace object sace normal y rojected surfel dsc y z surfel dsc Dscusson Quad and rojected dsc rmtve Smle, effcent Hardware suort Low mage qualty Sutable for revew renderers (e.g. Qslat [Rusnkewcz et al. 000] ) Problem: no blendng of rmtves 3 4

35 Slattng y A slat rmtve conssts of a colored ont rmtve and an alha mask colored ont rmtve c y * = alha mask w(,y) (often a D Gauss functon) y slat rmtve c * w(,y) Slattng The fnal color c(,y) s comuted by addtve alha blendng,.e., by comutng the weghted sum color of slat c(, y) = c w (, y) alha of slat at oston (,y) w (, y) Normalzaton s necessary, because the weghts do not sum u to one wth rregular ont dstrbutons w (, y) Slattng Slattng wthout normalzaton wth normalzaton Etended z-bufferng surface 1 surface varyng brghtness because of rregular ont dstrbuton no artfacts z-buffer el z-threshold accumulate slats dscard slats surfel dsc z 7 8 Etended Z-Bufferng Slattng Comarson DethTest(,y) { f (abs(slat z z(,y)) < threshold) { c(,y) = c(,y) + slat color mnf. elltcal slats crcular slats wth mn. radus surface slattng w(,y) = w(,y) + slat w(,y) } else f (slat z < z(,y)) { z(,y) = slat z c(,y) = slat color w(,y) = slat w(,y) } } magnf

36 Hgh Qualty Slattng Hgh qualty slattng requres careful analyss of alasng ssues Revew of sgnal rocessng theory Alcaton to ont renderng Surface slattng [Zwcker et al. 001] Alasng n Comuter Grahcs Alasng = Samlng of contnuous functons below the Nyqust frequency To avod alasng, samlng rate must be twce as hgh as the mamum frequency n the sgnal Alasng effects: Loss of detal More atterns, jagged edges Dsntegraton of objects or atterns Alasng n Comuter Grahcs Teture Mang Scan converson of geometry 31 3 Alasng n Comuter Grahcs Alasng: hgh frequences n the nut sgnal aear as low frequences n the reconstructed sgnal Occurrence of Alasng Satal Doman Frequency Doman Satal Doman Frequency Doman Alasng-Free Reconstructon Antalasng Satal Doman Frequency Doman Satal Doman Frequency Doman 35 Preflterng Band-lmt the contnuous sgnal before samlng Elmnates all alasng (wth an deal low-ass flter) Closed form soluton not avalable n general Suersamlng Rase samlng rate Reduces, but does not elmnate all alasng artfacts (n ractce, many sgnals have nfnte frequences) Smle mlementaton (hardware) 36

37 Resamlng dscrete nut sgnal dscrete outut sgnal Resamlng Flters Object Sace war color reconstructed nut reconstructon kernels 1. resamlng 4. oston rregular sacng Resamlng Flters Resamlng Flters Object Sace Screen Sace Object Sace Screen Sace sum of resamlng flters. War Screen Sace Screen Sace 4. Samle. War Screen Sace wared reconstructon kernel Screen Sace resamlng flters 4. Samle 3. Flter 39 low-ass flter convoluton 3. Flter 40 Resamlng Resamlng n the contet of surface renderng Dscrete nut functon = surface teture (dscrete D functon) Warng = rojectng surfaces to the mage lane (D to D rojectve mang) 41 D Reconstructon Kernels D reconstructon kernels are gven by surfel dscs wth alha masks Warng s equvalent to rojectng the kernel from object to screen sace screen sace object sace normal y wared reconstructon kernel y z surfel dsc wth alha mask = reconstructon kernel 4

38 Resamlng Flters A resamlng flter s a convoluton of a wared reconstructon flter and a low-ass flter screen sace el grd convoluton no nformaton falls nbetween the el grd Mathematcal Formulaton resamlng flter 1 c (, y) = c r ( m (, y)) h(, y) k k k wared reconstructon kernel low-ass flter (determned by el grd) resamlng flter ( blurred reconstructon kernel ) el color warng functon reconstructon kernel reconstructon kernel color low ass flter Gaussan Resamlng Flters Gaussans are closed under lnear warng and convoluton Wth Gaussan reconstructon kernels and low-ass flters, the resamlng flter s a Gaussan, too Effcent renderng algorthms (surface slattng [Zwcker et al. 001]) Mathematcal Formulaton 1 c (, y) = c r ( m (, y)) h(, y) k k k Gaussan reconstructon kernel screen sace Gaussan low-ass flter screen sace Mathematcal Formulaton 1 c (, y) = c r ( m (, y)) h(, y) k k = k c kgk (, y) k Gaussan resamlng flter Algorthm for each ont P { roject P to screen sace; shade P; determne resamlng kernel G; slat G; } for each el { normalze; } 47 48

39 Proertes of D Resamlng Flters wared reconstructon kernel low-ass flter resamlng flter mnfcaton Results Hgh qualty reconstructon and flterng magnfcaton 49 00k onts 4783k onts 50 Results transarent surfaces scanned objects Hardware Imlementaton Based on the object sace formulaton of EWA flterng Imlemented usng tetured trangles All calculatons are erformed n the rogrammable hardware (etensve use of verte shaders) Presented at EG 00 ([Ren et al. 00]) 987k onts [MERL/MIT Matusk et al.] 51 5 Surface Slattng Performance Software mlementaton slats/sec on 866 MHz PIII slats/sec on GHz P4 Hardware mlementaton [Ren et al. 00] Uses teture mang and verte shaders slats/sec on GeForce4 T 4400 Conclusons Ponts are an effcent renderng rmtve for hghly comle surfaces Ponts allow the drect vsualzaton of real world data acqured wth 3D scannng devces Hgh erformance, low qualty ont renderng s suorted by 3D hardware (tens of mllons onts er second) Hgh qualty ont renderng wth ansotroc teture flterng s avalable 3 mllon onts er second wth hardware suort 1 mllon onts er second n software Antalasng technque has been etended to volume renderng 53 54

40 Alcatons Drect vsualzaton of ont clouds Real-tme 3D reconstructon and renderng for vrtual realty alcatons Hybrd ont and olygon renderng systems Renderng anmated scenes Interactve dslay of huge meshes On the fly samlng and renderng of rocedural objects Future Work Dedcated renderng hardware Effcent aromatons of eact EWA slattng Renderng archtecture for on the fly samlng and renderng Acknowledgments Hanseter Pfster, Jeroen van Baar (MERL, Cambrdge MA),, CGL Lu Ren htt://grahcs.ethz.ch/surfels htt://grahcs.ethz.ch/ontsho3d References [Levoy and Whtted 1985] The use of onts as a dslay rmtve, techncal reort, Unversty of North Carolna at Chael Hll, 1985 [Heckbert 1986] Fundamentals of teture mang and mage warng, Master s Thess, 1986 [Grossman and Dally 1998] Pont samle renderng, Eurograhcs worksho on renderng, 1998 [Levoy et al. 000] The dgtal Mchelangelo roject, SIGGRAPH 000 [Rusnkewcz et al. 000] Qslat, SIGGRAPH 000 [Pfster et al. 000] Surfels: Surface elements as renderng rmtves, SIGGRAPH 000 [Zwcker et al. 001] Surface slattng, SIGGRAPH 001 [Zwcker et al. 00] EWA Slattng, to aear, IEEE TVCG 00 [Ren et al. 00] Object sace EWA slattng: A hardware accelerated aroach to hgh qualty ont renderng, Eurograhcs

41 Introducton Marc Alea, Carsten Dachsbacher,,, Hanseter Pfster, Marc Stammnger, Matthas Zwcker ont renderng how adat ont denstes? for a gven vewng oston, how can we get n onts that suffce for that vewer? how render the onts? gven n onts, how can we render an mage from them? Introducton how render the onts? roject ont to el, set el color hardware soluton (Radeon 9700 Pro) ~80 mo. onts er second no hole fllng software soluton ~8 mo. onts er second hole fllng hardware!= software Introducton even wth hardware: for (nt = 0; < N; ++) renderpontwthnormalandcolor ([],y[],z[],n[],ny[],nz[], ); 10 mo onts er second for (nt = 0; < N; ++) renderpont([],y[],z[]); 0 mo onts er second float * = {...} renderponts(); 80 mo onts er second best erformance wth sequental rocessng of large chunks! 3 4 Introducton whatwewant: sequental rocessng and adatve ont denstes recomuted ont lsts render contnuous segments only Herarchcal Processng Q-Slat Rusnkewcz et al., Sggrah 000 herarchcal ont renderng based on Boundng Shere Herarchy rendered segment ont lst 5 S. Rusnkewcz 6

42 Herarchcal Processng Q-Slatherarchy R R R R R R R Herarchcal Processng Q-Slat recursve renderng render( Node n ) { // comute screen sze of node s = n.r / dstancetocamera( n ); // screen sze too bg? f ( s > threshold ) // render chldren forall chldren c render( c ); else // else draw node renderpont( n.yz ); } 7 8 Herarchcal Processng notsequental no array, but tree structure most work on CPU CPU s bottleneck: ~8 mo onts er second sequental verson? Sequental Pont Trees store wth node d mn = n.r / 1 Pel render( Node n ) { // node too close? f ( dstancetocamera( n ) < n.dmn ) // render chldren forall chldren c render( c ); else // else draw node renderpont( n.yz ); } 9 10 Sequental Pont Trees node n s rendered f: n s not too close and arent s not rendered or dsttocam( n ) < n.dmn dsttocam( n.arent ) n.arent.dmn arentstooclose, butnodesfar enough Sequental Pont Trees assume dsttocam(n) dsttocam(n.arent) store wth n n.dma = n.arent.dmn then a node s rendered f n.dmn dsttocam(n) < n.dma 11 1

43 Sequental Pont Trees eamletree Sequental Pont Trees sequental verson foreach tree node n f ( n.dmn < dsttocam(n) && dsttocam(n) < n.dma ) renderpont(n); how enumerate nodes? Sequental Pont Trees Sequental Pont Trees sort nodes by d ma ont tree comute lower bound d bmn on dsttocam(n) wth boundng volume all elements wth d ma < d bmn can be sked only ref must be consdered sequental ont tree 15 ma ma ma 16 Sequental Pont Trees account for d d(arent): d ma = d mn (arent) + dstance to arent artally arent and some chldren selected no vsble artfacts from ths Sequental Pont Trees cullng by GPU necessary, because d s not constant over object 17 18

44 Sequental Pont Trees CPU does er frame: comuted bmn searchlast node ma wth d ma >d bmn send frst ma onts to GPU GPU then does for every node n comute d = dsttocam(n) fn.d mn d n.d ma render node Sequental Pont Trees CPU does frst nterval selecton by d bmn GPU does fne granularty selecton ref wth d bmn < d ma sequental ont tree culled because d < n.d mn rendered culled because d > n.d ma 19 0 Sequental Pont Trees Result cullng by GPU: only 10-40% on a,4 GHz Pentum wth Radeon 9700: CPU-Load < 0% (usually much less) > 50 Mo onts after cullng Sequental Pont Trees better error measurement n flatregons ncreased mn, d ma render larger onts 1 Sequental Pont Trees geometrc erendcular error Sequental Pont Trees eamle tangental error 3 4

45 Sequental Pont Trees also add teture crteron necessary for flat tetured regons Sequental Pont Trees f sgnfcant color varaton n chld nodes: modfy tangental error ncrease error to node dameter revents washed out colors n flat regons 5 6 Sequental Pont Trees erendcular, tangental, teture error scale wth 1/(vew dstance) fts nto sequental ont trees Sequental Pont Trees combne errors erendcular e tangental e t teturee te e com = r e + e t f teture varaton else => screen error = e com / vewdstance 7 8 Sequental Pont Trees can be combned wth olygons Sequental Pont Trees combne wth olygonal renderng for every trangle comuted ma (longest sde / d ma = ε) remove all onts from trangle wth smaller d ma sorttranglesford ma durng renderng for every object, comute uer bound d bma on dstance send trangles wth d ma < d bma to GPU on the GPU (verte rogram) test d < d ma cull by alha-test 9 30

46 Sequental Pont Trees ros verysmle! CPU-loadlow most work moved to GPU GPU runs at mamum effcency cons no vew frustum cullng currently: bad slattng suort by GPU 31

47 Overvew Effcent Smlfcaton of Pont-samled Surfaces Introducton Local surface analyss Smlfcaton methods Error measurement Comarson 1 Introducton Pont-based models are often samled very densely Many alcatons requre coarser aromatons, e.g. for effcent Introducton Eamle: Level-of-detal (LOD) renderng Storage Transmsson Processng Renderng We need smlfcaton methods for reducng the comlety of ont-based surfaces 10k 0k 60k 00k 000k 3 4 Introducton We transfer dfferent smlfcaton methods from trangle meshes to ont clouds: Herarchcal clusterng Iteratve smlfcaton Partcle smulaton Deendng on the ntended use, each method has ts ros and cons (see comarson) Local Surface Analyss Cloud of ont samles descrbes underlyng (manfold) surface We need: Mechansms for locally aromatng the surface MLS aroach Fast estmaton of tangent lane and curvature rncal comonent analyss of local neghborhood 5 6

48 Neghborhood No elct connectvty between samles (as wth trangle meshes) Neghborhood K-nearest neghbors Relace geodesc romty wth satal romty (requres suffcently hgh samlng densty!) Comute neghborhood accordng to Eucldean dstance Can be quckly comuted usng satal datastructures (e.g. kd-tree, octree, bs-tree) Requres sotroc ont dstrbuton 7 8 Neghborhood Imrovement: Angle crteron (Lnsen) Neghborhood Local Delaunay trangulaton (Floater) Project onts onto tangent lane Sort neghbors accordng to angle Include more onts f angle between subsequent onts s above some threshold Project onts nto tangent lane Comute local Vorono dagram 9 10 Covarance Analyss Covarance matr of local neghborhood N: Covarance Analyss Consder the egenroblem: wth centrod 1 1 C = Λ Λ, n n T 1 = N N j N C v l = λ v, l {0,1,} l C s a 33, ostve sem-defnte matr All egenvalues are real-valued The egenvector wth smallest egenvalue defnes the least-squares lane through the onts n the neghborhood,.e. aromates the surface normal l 11 1

49 Covarance Analyss Covarance ellsod sanned by the egenvectors scaled wth corresondng egenvalue Covarance Analyss The total varaton s gven as: N = λ + λ + λ 0 1 We defne surface varaton as: λ0 σ n( ) =, λ0 λ1 λ λ + λ + λ 0 1 Measures the fracton of varaton along the surface normal,.e. quantfes how strong the surface devates from the tangent lane estmate for curvature Covarance Analyss Surface Smlfcaton Comarson wth curvature: Herarchcal clusterng Iteratve smlfcaton Partcle smulaton orgnal mean curvature varaton n=0 varaton n= Herarchcal Clusterng Herarchcal Clusterng To-down aroach usng bnary sace artton: Slt the ont cloud f: Sze s larger than user-secfed mamum or Surface varaton s above mamum threshold Slt lane defned by centrod and as of greatest varaton (= egenvector of covarance matr wth largest assocated egenvector) Leaf nodes of the tree corresond to clusters Relace clusters by centrod D eamle covarance ellsod centrod slt lane root 17 18

50 Herarchcal Clusterng D eamle Herarchcal Clusterng D eamle 19 0 Herarchcal Clusterng Herarchcal Clusterng D eamle 43 Clusters 436 Clusters 4,80 Clusters 1 Herarchcal Clusterng Adatve Clusterng Iteratve Smlfcaton Iteratvely contracts ont ars Each contracton reduces the number of onts by one Contractons are arranged n rorty queue accordng to quadrc error metrc (Garland and Heckbert) Quadrc measures cost of contracton and determnes otmal oston for contracted samle Equvalent to QSlm ecet for defnton of aromatng lanes 3 4

51 Iteratve Smlfcaton Iteratve Smlfcaton Quadrc measures the squared dstance to a set of lanes defned over edges of neghborhood lane sanned by vectors e and e e 1 = = 1 n D eamle Comute ntal ont-ar contracton canddates e e 1 n Comute fundamental quadrcs Comute edge costs 5 6 Iteratve Smlfcaton Iteratve Smlfcaton D eamle rorty queue D eamle rorty queue edge cost edge cost Iteratve Smlfcaton Iteratve Smlfcaton D eamle rorty queue D eamle rorty queue edge cost edge cost

52 Iteratve Smlfcaton Iteratve Smlfcaton D eamle rorty queue D eamle rorty queue edge cost edge cost Iteratve Smlfcaton Iteratve Smlfcaton D eamle rorty queue edge cost orgnal model (96,850 onts) smlfed model (,000 onts) remanng ont ar contracton canddates Partcle Smulaton Resamle surface by dstrbutng artcles on the surface Partcles move on surface accordng to nter-artcle reellng forces Partcle relaaton termnates when equlbrum s reached (requres damng) Can also be used for u-samlng! Partcle Smulaton Intalzaton Randomly sread artcles Reulson F ( ) = k( r ) ( ) Lnear reulson force only need to consder neghborhood of radus r Projecton Kee artcles on surface by rojectng onto tangent lane of closest ont Aly full MLS rojecton at end of smulaton 35 36

53 Partcle Smulaton Partcle Smulaton D eamle D eamle Intalzaton randomly sread artcles Partcle Smulaton Partcle Smulaton D eamle Intalzaton randomly sread artcles D eamle Intalzaton randomly sread artcles Reulson lnear reulson force F ) = k( r ) ( ) ( Reulson lnear reulson force F ) = k( r ) ( ) ( Partcle Smulaton Partcle Smulaton D eamle Intalzaton randomly sread artcles D eamle Intalzaton randomly sread artcles Reulson lnear reulson force F ) = k( r ) ( ) ( Reulson lnear reulson force F ) = k( r ) ( ) ( Projecton roject artcles onto surface Projecton roject artcles onto surface 41 4

54 Partcle Smulaton Adatve smulaton Adjust reulson radus accordng to surface varaton more samles n regons of hgh varaton Partcle Smulaton User-controlled smulaton Adjust reulson radus accordng to user nut varaton estmaton smlfed model (3,000 onts) 43 unform orgnal selectve 44 Measurng Error Measure the dstance between two ont-samled surfaces usng a samlng aroach Mamum error: ma ( S, S ) = ma d(, S q Q q ) Two-sded Hausdorff dstance 1 Mean error: S S = d q S avg (, ) (, ) Q q Q Area-weghted ntegral of ont-to-surface dstances Q s an u-samled verson of the ont cloud that descrbes the surface S Measurng Error d ( q, S ) measures the dstance of ont q to surface S usng the MLS rojecton oerator wth lnear bass functons Measurng Error Comarson Error estmate for Mchelangelo s Davd smlfed from,000,000 onts to 5,000 onts orgnal smlfed usamled error 47 48

55 Comarson Eecuton tme as a functon of nut model sze (reducton to 1%) Comarson Eecuton tme as a functon of target model sze (nut: dragon, 535,545 onts) tme 500 (sec) 450 herarchcal clusterng teratve smlfcaton tme 70 (sec) 60 herarchcal clusterng teratve smlfcaton artcle smulaton 50 artcle smulaton nut sze target sze Comarson Summary Pont-based vs. Mesh Smlfcaton Herarchcal Clusterng Effcency + Surface Error - Control - Imlementaton + Iteratve Smlfcaton Partcle Smulaton - o + + o + o - ont-based smlfcaton wth subsequent mesh reconstructon mesh reconstructon wth subsequent mesh smlfcaton (QSlm) ont-based smlfcaton saves an eensve surface reconstructon on the dense ont cloud! 51 5 References Pauly, Gross: Effcent Smlfcaton of Pontsamled Surfaces, IEEE Vsualzaton 00 Shaffer, Garland: Effcent Adatve Smlfcaton of Massve Meshes, IEEE Vsualzaton 001 Garland, Heckbert: Surface Smlfcaton usng Quadrc Error Metrcs, SIGGRAPH 1997 Turk: Re-Tlng Polygonal Surfaces, SIGGRAPH 199 Alea et al. Pont Set Surfaces, IEEE Vsualzaton

56 Overvew Sectral Processng of Pont- Samled Geometry Introducton Fourer transform Sectral rocessng elne Alcatons Sectral flterng Adatve subsamlng Summary Introducton Idea: Etend the Fourer transform to manfold geometry Sectral reresentaton of ont-based objects Powerful methods for dgtal geometry rocessng Introducton Alcatons: Sectral flterng: Nose removal Mcrostructure analyss Enhancement Adatve resamlng: Comlety reducton Contnuous LOD 3 4 Fourer Transform Fourer Transform 1D eamle: N nk j π N X n = ke sectral bass functon k = 1 outut sgnal nut sgnal Benefts: Sound concet of frequency Etensve theory Fast algorthms Requrements: Fourer transform defned on Eucldean doman we need a global arameterzaton Bass functons are egenfunctons of Lalacan oerator requres regular samlng attern so that bass functons can be eressed n analytcal form (fast evaluaton) Lmtatons: Bass functons are globally defned Lack of local control 5 6

57 Aroach Sectral Pelne Slt model nto atches that: are arameterzed over the unt-square mang must be contnuous and should mnmze dstorton are re-samled onto a regular grd adjust samlng rate to mnmze nformaton loss rovde suffcent granularty for ntended alcaton (local analyss) rocess each atch ndvdually and blend rocessed atches 7 8 Patch Layout Creaton Patch Layout Creaton Clusterng Otmzaton Samles Clusters Patches Iteratve, local otmzaton method Merge atches accordng to qualty metrc: Φ = Φ S Φ NC Φ S atch Sze Φ NC curvature Φ B Φ Reg Φ B atch boundary Φ Reg srng energy regularzaton 9 10 Patch Layout Creaton Patch Resamlng Parameterze atches by orthogonal rojecton onto base lane Bound normal cone to control dstorton of mang usng smallest enclosng shere Patches are rregularly samled: 11 1

58 Patch Resamlng Resamle atch onto regular grd usng herarchcal ush-ull flter (scattered data aromaton) Sectral Analyss D dscrete Fourer transform (DFT) Drect manulaton of sectral coeffcents Flterng as convoluton: F( y) = F( ) F( y) Convoluton: O(N ) multlcaton: O(N) Inverse Fourer transform Fltered atch surface Sectral Flters Sectral Flters Smoothng flters deal low-ass Gaussan low-ass orgnal Mcrostructure analyss and enhancement transfer functon: sectral doman transfer functon: satal doman Sectral Resamlng Reconstructon Low-ass flterng Band-lmtaton Regular Resamlng Otmal samlng rate (samlng theorem) Error control (Parseval s theorem) Power Sectrum Flterng can lead to dscontnutes at atch boundares Create atch overla, blend adjacent atches Samlng rates Pont ostons Normals regon of overla 17 18

59 Reconstructon Tmngs Blendng the samlng rate Tme Clusterng 9% Patch Mergng 38% SDA 3% blended samlng rate n regon of atch overla dscretzed samlng rate on regular grd re-comuted samlng atterns Analyss Reconstructon 4% 6% 19 0 Alcatons Alcatons Surface Restoraton Interactve flterng Orgnal Gaussan low-ass Wener flter Patch layout 1 Alcatons Summary Adatve Subsamlng Versatle sectral decomoston of ontbased models Effectve flterng Adatve resamlng Effcent rocessng of large ont-samled models 4,18,614 ts. = 100% 87,163 ts. = 6.9% 3 4

60 Reference Pauly, Gross: Sectral Processng of Pont-samled Geometry, SIGGRAPH 001 5

61 Overvew An Interactve System for Pont-based Surface Edtng Introducton Pontsho3D System Comonents Pont Cloud Parameterzaton Resamlng Scheme EdtngOerators Summary PontSho3D Interactve system for ont-based surface edtng Generalzes D hoto edtng concets and functonalty to 3D ont-samled surfaces Uses 3D surface els (surfels) as versatle dslay and modelng rmtve Concet Parameterzaton v u Resamlng Edtng Oerator 3 4 Key Comonents Parameterzaton Pont cloud arameterzaton Φ brngs surface and brush nto common reference frame Dynamc resamlngψ creates one-to-one corresondence of surface and brush samles Edtng oerator Ω combnes surface and brush samles S = Ω( Ψ( Φ( S)), Ψ( B)) Constraned mnmum dstorton arameterzaton of ont clouds u ( u) u = ( ) u z( u) 3 [ 0,1] X ( ) y = P R modfed surface orgnal surface brush 5 6

62 Parameterzaton Parameterzaton Fnd mang X that mnmzes objectve functon: brush onts surface onts contrants = matchng of feature onts mnmum dstorton = mamum smoothness C( X ) = j M ( X ( j ) j ) + ε γ ( u) du { P { fttng constrants dstorton 7 8 Parameterzaton Parameterzaton Measurng dstorton Dscrete formulaton: γ ( ) X ( θ, r) dθ r u = u θ r u θ ~ C( U ) = n ( j u j ) + ε j M = 1 j N ( ) ( ) U U ~ v j v j Integrates squared curvature usng local olar re-arameterzaton cos( θ ) X u( θ, r) = X u + r sn( θ ) Aromaton: mang s ecewse lnear 9 10 Parameterzaton Parameterzaton Drectonal dervatves as etenson of dvded dfferences based on k-nearest neghbors Multgrd solver for effcent comutaton of resultng sarse lnear least squares roblem ~ C( U ) b n = j j = 1 a j, u = b Au 11 1

63 Reconstructon Reconstructon Parameterzed scattered data aromaton fttng functons Φ ( u) r ( u) X ( u) = r ( u) weght functons normalzaton factor Fttng functons Comute local fttng functons usng local arameterzatons Ma to global arameterzaton usng global arameter coordnates of neghborng onts reconstructon wth lnear fttng functons weght functons n arameter sace Reconstructon Samlng Reconstructon wth lnear fttng functons s equvalent to surface slattng! we can use the surface slattng renderer to reconstruct our surface functon (see chater on renderng) Ths rovdes: Fast evaluaton Ant-alasng (Band-lmt the weght functons before samlng usng Gaussan low-ass flter) Dstortons of slats due to arameterzaton can be comuted effcently usng local affne mangs Three samlng strateges: Resamle the brush,.e., samle at the orgnal surface onts Resamle the surface,.e., samle at the brush onts Adatve resamlng,.e., samle at surface or brush onts deendng on the resectve samlng densty Edtng Oerators Edtng Oerators Pantng Teture, materal roertes, transarency Scultng Carvng, normal dslacement teture ma dslacement mas carved and teture maed ont-samled surface 17 18

64 Edtng Oerators Flterng Scalar attrbutes, geometry Summary Pontsho3D rovdes sohstcated edtng oeratons on ont-samled surfaces onts are a versatle and owerful modelng rmtve Lmtaton: only works on clean models suffcently hgh samlng densty no outlers lttle nose requres model cleanng (ntegrated or as rerocess) 19 0 Reference Zwcker, Pauly, Knoll, Gross: Pontsho3D: An nteractve system for Pont-based Surface Edtng, SIGGRAPH 00 check out: 1

65 Motvaton Shae Modelng 3D content creaton elne Motvaton Motvaton Surface reresentatons Imlct surfaces Level sets Radal bass functons Algebrac surfaces Parametrc surfaces Polygonal meshes Subdvson surfaces NURBS + Etreme deformatons + Changes of toology + Shar features + Effcent renderng + Intutve Edtng Surface reresentatons Imlct surfaces Level sets Radal bass functons Algebrac surfaces Parametrc surfaces Polygonal meshes Subdvson surfaces Nurbs Hybrd Reresentaton Elct cloud of ont samles Imlct dynamc surface model 3 4 Motvaton Interactve Modelng Pont cloud reresentaton Mnmal consstency requrements for etreme deformatons (dynamc re-samlng) Fast nsde/outsde classfcaton for boolean oeratons and collson detecton Elct modelng and renderng of shar feature curves Integrated, ntutve edtng of shae and aearance Interactve desgn and edtng of ont-samled models Shae Modelng Boolean oeratons Free-form deformaton Aearance Modelng Pantng & teturng Embossng & engravng 5 6

66 Boolean Oeratons Boolean Oeratons Create new shaes by combnng estng models usng unon, ntersecton, or dfference oeratons Powerful and fleble edtng aradgm mostly used n ndustral desgn alcatons (CAD/CAM) 7 8 Boolean Oeratons Boolean Oeratons Easly erformed on mlct reresentatons Requres smle comutatons on the dstance functon Dffcult for arametrc surfaces Requres surface-surface ntersecton Toologcal comlety of resultng surface deends on geometrc comlety of nut models Pont-Samled Geometry Classfcaton Insde-outsde test usng sgned dstance functon nduced by MLS rojecton Samlng Comute eact ntersecton of two MLS surfaces to samle the ntersecton curve Renderng Accurate decton of shar corners and creases usng ont-based renderng 9 10 Boolean Oeratons Boolean Oeratons Classfcaton: gven a smooth, closed surface S and ont. Is nsde or outsde of the volume V bounded by S? S V Classfcaton: gven a smooth, closed surface S and ont. Is nsde or outsde of the volume V bounded by S? 1.fnd closest ont q on S S V q 11 1

67 Boolean Oeratons Boolean Oeratons Classfcaton: gven a smooth, closed surface S and ont. Is nsde or outsde of the volume V bounded by S? 1.fnd closest ont q on S.d=(-q) n defnes sgned dstance of to S n q S V Classfcaton: gven a smooth, closed surface S and ont. Is nsde or outsde of the volume V bounded by S? 1.fnd closest ont q on S.d=(-q) n defnes sgned dstance of to S 3.classfy as nsde V, f d < 0 outsde V, f d > 0 n q S V Boolean Oeratons Boolean Oeratons Classfcaton: reresent smooth surface S by ont cloud P m P S V Classfcaton: reresent smooth surface S by ont cloud P m 1.fnd closest ont q n P S P n q V Boolean Oeratons Boolean Oeratons Classfcaton: reresent smooth surface S by ont cloud P m 1.fnd closest ont q n P.classfy as nsde V, f (-q) n < 0 outsde V, f (-q) n > 0 S P n q V Classfcaton: aly full MLS rojecton for onts close to the surface Ψ P () n q 17 18

68 Boolean Oeratons Samlng the ntersecton curve Boolean Oeratons Newton scheme: 1.dentfy ars of closest onts 19 0 Boolean Oeratons Boolean Oeratons Newton scheme: 1.dentfy ars of closest onts Newton scheme: 1. dentfy ars of closest onts. comute closest ont on ntersecton of tangent saces r q 1 q q 1 q 1 Boolean Oeratons Boolean Oeratons Newton scheme: 1. dentfy ars of closest onts. comute closest ont on ntersecton of tangent saces 3. re-roject ont on both surfaces q 1 r q Newton scheme: 1. dentfy ars of closest onts. comute closest ont on ntersecton of tangent saces 3. re-roject ont on both surfaces 4. terate q 1 r q q 1 q 3 4

69 Boolean Oeratons Renderng shar creases reresent onts on ntersecton curve wth two surfels that mutually cl each other Boolean Oeratons Renderng shar creases 5 6 Boolean Oeratons Boolean Oeratons Renderng shar creases easly etended to handle corners by allowng multle clng Boolean oeratons can create ntrcate shaes wth comle toology A + B A B A B B A 7 8 Boolean Oeratons Boolean Oeratons Sngulartes lead to numercal nstabltes (ntersecton of almost arallel lanes) Shar creases can be blended usng orented artcles (Szelsk, Tonnesen) 9 30

70 Free-form Deformaton Free-form Deformaton Smooth deformaton feld F:R 3 R 3 that wars 3D sace Can be aled drectly to ont samles 31 3 Free-form Deformaton Free-form Deformaton How to defne the deformaton feld? Pantng metahor How to detect and handle selfntersectons? Pont-based collson detecton, boolean unon, artcle-based blendng How the handle strong dstortons? Dynamc re-samlng Intutve edtng aradgm usng antng metahor Defne rgd surface art (zero-regon) and handle (one-regon) usng nteractve antng tool Dslace handle usng combnaton of translaton and rotaton Create smooth blend towards zero-regon Free-form Deformaton Free-form Deformaton zero-regon Defnton of deformaton feld: Contnuous scale arameter t t = β (d 0 / (d 0 + d 1 )) d 0 : dstance of to zero-regon d 1 : dstance of to one-regon d 0 d 1 one-regon Blendng functon β : [0,1] [0,1] β C 0, β (0) = 0, β (1) = 1 orgnal surface deformed surface t = 0 f n zero-regon t = 1 f n one-regon 35 36

71 Free-form Deformaton Free-form Deformaton Defnton of deformaton feld: Deformaton functon F () = F T () + F R () d 0 Translaton for three dfferent blendng functons Translaton F T () = + t v Rotaton F R () = M(t ) d 1 blendng functon deformed surface Free-form Deformaton Free-form Deformaton Rotatonal deformaton along two dfferent rotaton aes Embossng effect btma mage orgnal surface color-coded scale arameter deformed surface zero- and one-regons deformed surface Collson Detecton Collson Detecton Deformatons can lead to selfntersectons Aly boolean nsde/outsde classfcaton to detect collsons Restrcted to collsons between deformable regon and zero-regon to ensure effcent comutatons Elotng temoral coherence 41 4

72 Collson Detecton Interactve modelng sesson collson detected boolean unon erformed artcle-based blendng Dynamc Samlng Large model deformatons can lead to strong surface dstortons Requres adataton of the samlng densty Dynamc nserton and deleton of ont samles Dynamc Samlng Dynamc Samlng Surface dstorton vares locally 1. Measure local surface stretch from frst fundamental form. Slt samles that eceed stretch threshold 3. Regularze dstrbuton by relaaton 4. Interolate scalar attrbutes color-coded surface stretch surface after dynamc re-samlng Dynamc Samlng D llustraton Free-form Deformaton Interactve modelng sesson wth dynamc samlng orgnal surface wth zero- and one-regons ntermedate stes of deformaton fnal surface 47 48

73 Results 3D shae modelng functonalty has been ntegrated nto Pontsho3D to create a comlete system for ont-based shae and aearance modelng Boolean oeratons Free-form deformaton Pantng & teturng Scultng Flterng Etc. Results Ab-nto desgn of an Octous Free-form deformaton wth dynamc samlng from 69,706 to 95, onts Results Results Modelng wth synthetc and scanned data Combnaton of free-form deformaton wth collson detecton, boolean oeratons, artcle-based blendng, embossng and teturng Boolean oeratons on scanned data Irregular samlng attern, low resoluton models 51 5 Results Concluson Interactve modelng wth scanned data nose removal, free-form deformaton, cut-andaste edtng, nteractve teture mang Ponts are a versatle shae modelng rmtve Combnes advantages of mlct and arametrc surfaces Integrates boolean oeratons and freeform deformaton Dynamc restructurng Tme and sace effcent mlementatons 53 54