Style-based Inverse Kinematics

Transcription

1 Style-based Inverse Kinematics Keith Grochow, Steven L. Martin, Aaron Hertzmann, Zoran Popovic SIGGRAPH 04 Presentation by Peter Hess 1

2 Inverse Kinematics (1) Goal: Compute a human body pose from a set of constraints. Problem: The problem is inherently underdetermined! There are many possible solutions, but most of them don't look natural. 2

3 Inverse Kinematics (2) Solution: Restrain the algorithm to a certain "style" of natural looking poses constrained optimization problem Previous approachs: Use a reference pose and define a distance metric Mass displacement Energyconsumption Drawbacks: Not every metric is good for every style. Finding a good metric for a certain style is difficult. 3

4 Overview Scaled Gaussian Process Latent Variable Model (SGPLVM) Learning Synthesis Optimizations / Extensions Annealing Active Set Style Interpolation Applications Performance Conclusions 4

5 SGPLVM: Setting (1) Main idea Learning Take a set of motion capture poses for training Search a lowdimensonal representation of these data Create a probability distribution function (PDF) in latent space Create new poses (synthesis) For a set of constraints, use the PDF to find the pose which is most likely compared to the input. 5

6 SGPLVM: Setting (2) Pose q i : Input poses are represented by skeletons Joint angles, root position and root orientation 42 dim Feature vector y i : Joint angles from q i Vertical orientation Velocity and acceleration 100+ dim 6

7 SGPLVM: Setting (3) Scaling matrix W W = diag{w 1,, w D } Scales the dimensions of y according to their importance: Wy Latent variables x i x i are linked with y i by a Gaussian Process 2-3 dim Lowdimensional representation of y i Dimension reduction is only possible, because natural movements are very structured. 7

8 SGPLVM: Setting (4) Kernel k γ k( x, x') = αexp( x x' 2 2 Parameters α, β, γ initially unknown Value of k mainly depends on distance between x and x' Shows the correlation between y and y', based on their corresponding x and x' ) + δ x,x' β 1 8

9 SGPLVM: Setting (5) The mapping between x i and y i is defined by a Gaussian Process. The likelihood for y i,k is: 1 1 T 1 p({ yi, k} { xi},α,β, γ) = exp( Y N k K Y (2π ) K 2 k ) K isthecovariancematrixof thefeaturevectors: k(x i, x j ) = cov(y i, y j ) distance between x i and x j small y i and y j are simillar 9

10 SGPLVM: Learning (1) Known: {q i }, {y i } Unknown: {x i }, {w i }, α, β, γ Maximize posterior probability: max p({ x },{w },α,β, γ { y }) { xi },{wi },α,β,γ i i i 10

11 SGPLVM: Learning(2) Minimizing L GP will learn all the model parameters you need: min L = D ln K T 1 2 w Y K Yk + xi + k k GP { x N i },{w i },α,β,γ 2 2 k 2 i w k k 1 ln αβγ Minimizing L GP arranges {x i }, so that similar poses are nearby and dissimilar poses are far apart. Remember: k(x i, x j ) = cov(y i, y j ) Likelihood for x is high near the example poses. 11

12 SGPLVM: Synthesis (1) For new poses y(q) the according x can be found by minimizing L IK : W( y f ( x)) D min LIK ( x, y) = + lnσ ( x) + x x 2 2σ ( x) prediction error 2 neg. likelihood of x 12

13 SGPLVM: Synthesis (2) For synthesizing new poses y is unknown a optimize L IK for x and q min x, q L IK ( x, y( q)) s.t. Constraints( q) = 0 13

14 Annealing(1) L IK is a complicated function with many local minima the numerical optimizer may get trapped in a poor minima Avoid local minima by smoothing L IK 14

15 Annealing(2) Learning: use original setting: minimize L GP {x i }, {w i }, α, β, γ Keep {x i }, {w i } fixed and add noise to {y i }: minimize L GP α', β', γ' Synthesis: Optimize L IK : Use α', β', γ' for first steps Switch to interpolated parameters Finish with using α, β, γ 15

16 Active Set For the optimization of L IK K is needed Poor scaling, because K grows quadratically with the number of learning examples. Simplify L IK by only using a subset of all the training poses. The set of most representative input poses may be calculated during the learing step. 16

17 Style Interpolation Interpolation for two styles for a given interpolation parameter s. L (, x, y) = (1 s)l ( x, y) sl ( x, y) s x0 1 IK0 0 + IK1 1 17

18 Motion capturing with missing markers Optical Motion Capturing: Occluded markers can be reconstructed with style-based IK Works even when 50% of the markers are missing 18

19 Trajectory keyframing Create sequence of poses by defining a trajectory for one part of the body. 19

20 Posing from 2D images Reconstruct 3D pose from 2D image 20

21 Performance Computer 2.8 GHz P4 Model input sequence: 500 frames active set: 100 poses latent space: 3D Performance precomputation time:? synthesis: 23 fps 21

22 Conclusion(1) Good things: + Realtime + Intuitive & easy to use + Usefull for different kind of problems + No parameter tuning (except optimizations) 22

23 Conclusion(2) Drawbacks / possible problems: - Scaling behaviour for more complex skeletons? - No physics - Only indirect temporal relation between inputs no keyframing - Sensitive to input: How much can the input differ from the output? 23

24 Future Now: 1 image, 1 person Future: whole film, n persons? 24