CS 231 Motion Capture Data I The Pipeline Bodenheimer et al 1
Marker Magnetic Optical Marker placement On limbs vs joints neither is ideal Over tight clothing or thin skin In repeatable 'landmarks' Using standard marker sets 2
Triangulation from multiple calibrated cameras Sources of noise Outliers Joints are approximated as pivots Simplifications (like a rigid back) Markers move on the skin, clothing Errors may accumulate 3
Filtering Gets rid of outliers Smoothes data But removes important details! All data is filtered What s s next? Processing the data Correcting errors Modifying data to fit your character Edit the data to do something new Connecting data sequences Generalize a pool of data 4
Using marker data - skeleton estimation O Brien et al Video 5
Mapping data to your character (Proprietary solutions used in practice) Mapping data to your character Simulation is used offline to compute postures Internal torque actuators allow the simulation to act as a flexible ragdoll Force springs pull 'ragdoll' ragdoll' to reach the data, marker by marker Contact (e.g. ground) may be added through force 6
Approach overview Basic Algorithm foreach (data sample) { update [yellow] markers while (not still) { compute torques compute body forces if (active) compute contact forces update simulation }//while record posture }//for 7
Spring body forces Force-driven virtual 'landmarks' placed by hand guide the simulated bodies to follow the markers F marker Springs pull the simulation to the marker data F marker = -kf X error F damping Body motion is damped F damping = -bf V body t Note, markers near joints affect both nearby bodies Internal torque control PD-servo's control 3D ball joints at each articulation point to resist bending = = k( q d q ) b( q ) q d from rest position k and b are chosen by hand t No joint limits 8
Additional constraint forces Avoiding foot/ground penetration and foot skate Normal ground forces flatten the foot on ground via a penalty method Marker data is used to tag when each foot is sliding or not Horizontal friction forces (not shown) resist in opposite direction of the simulated point velocity when in slip 9
10
Raw vs sim foot position 11
Editing data for reuse Modifying individual sequences Combining by creating transitions and cyclification Parametric motions from set of data Re-ordering automatically Modifying individual sequences Displacement mapping q' = a(t) ) q + b(t) 12
Modifying individual sequences Witkin & Popovic - Motion warping Modifying individual sequences Witkin & Popovic - Motion warping 13
Hierarchical Edits - Lee & Shin Multi-level level B-Splines Hierarchical Edits 14
Combining by creating transitions Combining by creating transitions Rose et al use min energy to create transition 15
More automatically based transitions When to transition? Kovar and Gleicher - Registration Curves Parametric motions from set of data 16
Parametric motions from set of data Wiley and Hahn - interpolation synthesis Interpolation Synthesis for Articulated Figure Motion 17
Editing data for reuse Modifying individual sequences Combining by creating transitions and cyclification Parametric motions from set of data Re-ordering automatically Re-ordering automatically (Mocap( Soup) Various dataset Automatic transitions are generated 18
Walk Cycle Start Stop Left Turn Right Turn Combining by creating transitions 19
Re-ordering automatically Motion capture Virtual environment Sketched path Obstacles Re-ordering automatically Motion capture Virtual environment 20
Re-ordering automatically Motion graph Re-ordering automatically Motion graphs Schoedl et al. 2000 Arikan and Forsyth; Kovar et al. 2002; Lee et al. 2002; Li et al. 2002; Control Schoedl and Essa 2002 Arikan et al 2003, Reitsma and Pollard 2004, Lau and Kuffner 2005, Beck and Gleicher 2007 Perception Reitsma and Pollard 2003, Ren et al. 2005, Ikemoto et al. 2007, Wang and Bodenheimer 2003, 2004 Clean-up (foot skate) Kovar and Gleicher 2002; Ikemoto et al. 2005 21
Re-ordering automatically Unstructured Input Data A number of motion clips Each clip contains many frames Each frame represents a pose Re-ordering automatically Unstructured Input Data Connecting transition Between similar frames 22
Search - Frames and Windows Frame distance (normalized) n d ( f 1, f 2) ( w p ( f 1 pb b 1) p b( f 2) w b b( f 1) b( f 2)) b position error orientation error Window distance e DW (, W) w d( f, f ) 1 2 i s i 1i 2i n+m... ṇ+m n+2 n n+1 Re-ordering automatically Distance between Frames D( i, j) d( pi, p j ) d( vi, v j ) Weighted differences of joint angles Weighted differences of joint velocities 23
Re-ordering automatically Distance Matrix Re-ordering automatically Graph Construction 24
Re-ordering automatically Pruning Transition Contact state: Avoid transition to dissimilar contact state Likelihood: User-specified specified threshold Similarity: Local maxima Avoid dead-ends: ends: Strongly connected components Re-ordering automatically Motion capture region Virtual environment 25
Video Break Re-ordering automatically Control how to determine transition to follow? User key press AI - search 26
Re-ordering semi-automatically Gleicher et al - "Snap together motion" 27