Animation by Adaptation Tales of Motion Use and Re-Use

Transcription

1 Animation by Adaptation Tales of Motion Use and Re-Use Michael Gleicher And the UW Graphics Gang Department of Computer Sciences University of Wisconsin Madison

2 Tales of Motion Use and Re-Use Can we use retargeting tools to create animation? Spacetime-based retargeting (1998) sounded good, what happened? Hey Mike what have you been doing for the past 3 years?

3 What have I been doing? Welcome to Wisconsin! (yes the snow is up to the mailbox)

4 Motion Retargeting in Practice Why haven t the methods taken off? Retargeting solves only one piece of the character animation puzzle The results aren t as good as we might like The methods are a pain to deal with Maybe motion re-use isn t the right problem

5 The Talk Agenda Help! Show you the complete story Our whole motion re-use pipeline How we are dealing with issues Where the old stuff fits in Show you how far we re getting And where we might be going

6 The Original Plan Create animation by re-use Easier to beg, borrow, steal, buy,

7 Retargeting Compendium A retrospective Some old stuff Indulge me it s 47 seconds Media Clip Some new examples of the old stuff Notice what s different Media Clip Found examples

8 What did you just see Found Animation Found characters on the web Found motions on the web Put them together! Animating Characters Not just skeletons Simple skinning methods All examples from motion capture

9 What did you see? Plenty of problems! Many relating to the fact that we have characters not just skeletons We ve been lucky to get some good motion, but not all of it is great

10 Motion is Tough! And it s not just motion capture s problem Motion Capture Sample every frame No No structure. No No intent. lots lots of of data! Dense samples can represent rich frequency content Noise, sensor errors Hand Animation Data at at Keyframes Structured? Remember why? Maybe lots lots of of data. Good animators create complex frequency content Quality takes work Hard for academics to obtain quality hand animation

11 Mocap Pipeline Plan Shoot Process Apply Motion Capture is a misunderstood technology Retargeting is in the Process box Found Motion, so no control of Shoot Distinction between Process and Apply might be blurred Similar story for hand animation

12 Processing Pipeline Raw Data identify clean Clean Solve to Marker Data Skeleton cleanup Clean Skeletal Data Define Character Define Character Controls Define Skeleton Application Animation Data Retargeting& Control Mapping Clean Skeletal Data

13 Why change a perfectly good motion? Animation: bring something to life means we want to change something about the performance Actor vs. character Restrictions of realism Performers aren't perfect Need for usable data (loops, reference poses,...) Studio is not virtual world Motion re-use sometimes we are stuck with what we have

14 So You Want a Dancing Duck. Our motion pipeline Driven by need to use found stuff An overview of the process includes: Gather model and motion Match model and motion poses Bind the model to the motion Tweak the results using constraintbased motion editing techniques

15 How did we do that? Get Motion (from library) Annotate Constraints Compute Adapted Motion Get from library (stock CD or Web) Our Software Convert Skeleton Get Character (from library) Devise skeleton for character Lighting Commercial Animation System (Maya or 3D Studio MAX) Render

16 How to connect? Find a skeleton for character Use topology of original motion Find deformation parameters Skinning Vertex Weights (since we re doing SSD) This is backwards! Pros design controls with characters Map animation data on those controls

17 Skeleton Matching But how does the skeleton connect to the character? What are the character s sizes? Where should the joints go? What about orientations?

18 Find Character s Skeleton Not simply a geometric operation Spine down center or back? How many joints? Which d.o.f. where? Skeleton is an abstraction

19 Skeletons for humans Animals are very complex We understand humans pretty well

20 We have favorite abstractions 206 bones, muscles, fat, organs, clothing, 206 bones, complex joints 53 bones Kinematic joints

21 This is not how humans work Representation of complex human structure with varying degrees of simplification Simple Pin Joint Complex tendon and bone system

22 This can t be automated! Artistic decisions in how the character s geometry responds to movement! Need to design the set of controls for moving the geometry Get the right expressitivity Get the right response Get the right ease of use We re limiting ourselves to given format

23 Our Approach. Instead of a skeleton, create a point cloud of joints in the mesh Easy for the user Findskel finds skeleton

24 Now Are We Done? Most certainly not! The model is not interacting with its environment as it should Its feet are nowhere near the ground

25 Transformation Basics Change what isn t important, retain what is Hard to define what is important high-level properties motion specific Stick to what s easy to define geometric constraints signal characteristics framework for better metrics later

26 Transformation as Constrained Optimization Find a motion that 1. Meets any specific requests 2. Keeps any specific characteristics of the original 3. Is as similar as possible to the original Naturally posed as constrained optimization subject to meeting the constraints (1 and 2) minimize some objective (3)

27 Spacetime Constraints Consider all constraints simultaneously NOT frame at a time Solve for motions best motion that meets constraints

28 Basic Idea 1: Constraints Exact parameter values may not be important Geometric constraints often are important

29 Basic Idea 2: Frequency Content High frequencies (abrupt changes) are significant and noticeable Altering high-frequencies changes motions Adaptations should avoid disturbing high frequencies

30 Retargetting Motion to New Characters Goal: one motion, a cast of characters Focus on similar structure

31 Retargetting Recipe 1. Define Constraints 2. Apply to new character

32 Retargetting Recipe 3. Approximate Answer 4. Solve constraints (band-limited adaptation)

33 What s Right with this story?

34 What s wrong with the story? Philosophically misguided Force old controls onto new character Requires convention in controls Randomly chose hard to use controls Ignores MoCap Wisdom Begin with the ends in mind Solve problems earlier, not later Dealing with clips

35 Processing Pipeline Raw Data identify clean Clean Solve to Marker Data Skeleton cleanup Clean Skeletal Data Define Character Define Character Controls Define Skeleton Application Animation Data Retargeting& Control Mapping Clean Skeletal Data

36 What s wrong with the story? Results aren t as good as we d like Get better constraints Solver is too hard to implement Find new solution paradigm High-level control still hard Build abstracted representations Off-line See upcoming ToG paper Now for the quick tour of the first 3

37 Better Constraints/Objectives Feature of constraint-based approach Just need to find those right ones Dates back to Witkin&Kass 88 We still don t have the grace functional I expected to do more of this Lots of mileage from using a few constraints well Figure out how to use what you have

38 Want Better Results? Devise Better Constraints! Self-intersection Higher-level properties Style???? Form????????

39 Collision Constraints Get your hands out of your pockets! Media Clip

40 Constraints don t fix everything This arm does well This arm doesn t go through the body, but

41 Wins and Losses of Spacetime Fast, practical Linear complexity? Solves real problems Flexibility in: Spatial Constraints Objective Functions? Temporal Constraints? Widely applicable Nice results Hard to implement Poor integration Off-line No guarantees Spatial constraints not enforced Flexibility not exploited Rely on constraints

42 Is there an alternative? Need to deal with spatial and temporal constraints Don t want the messiness of whole motion computation Handle spatial and temporal constraints separately! Per-Frame IK Plus Filter Lee & Shin first version Gleicher 00 & generalization

43 Per-Frame IK + Filter (PFIK+F) IK per frame to solve spatial constraints But this messes up temporal constraints Filter changes to enforce temporal constraints But this messes up spatial constraints Iterate until converges, or

44 PFIK+F vs. Spacetime Fast, practical Solves real problems Flexibility in: Spatial Constraints Objective Functions (?) Temporal Constraints (?) Widely applicable Nice results Yes! (requires fast IK) Yes! (depends on IK) (depends on IK) (limited, unexplored) Yes! Um, it s a matter of taste, and IK quality

45 PFIK+F vs. Spacetime Use standard pieces! Use standard pieces! Choice in which last Solve spatial constraints last Need good IK Hard to implement Poor integration Off-line No guarantees Spatial constraints not enforced Flexibility not exploited Rely on constraints

46 My PFIK+F solver Use pieces I have lying around Non-linear optimizing solver for IK FIR linear filters for temporal constraints Propagation of results for temporal coherence

47 Downsides of PFIK+F? No global decisions Doesn t handle don t cares as well Order dependence Media Clip No interframe constraints Reliance on quality of IK solver Not necessarily faster (or slower)

48 Higher Level Control Need to find abstractions of motion Can t just work with details Even constraints aren t enough better Path Editing (a factoring approach) Gleicher 01 Motion Graphs Kovar & Gleicher??

49 The Idea Original Motion New Path

50 What is Path Editing? Factor Motion into Path And Detail Edit Path as Needed (use curve editing tools) Apply Detail to new Path

51 Why Edit Paths? (because previous work can t do it) Can t Capture All Possibilities Can t Synthesize All Styles Can t Simulate Complex Motions Current Editing Tools Don t Apply Need tools for sculpting motion INTERACTIVELY! Need easier to implement tools!

52 How to Edit a Path Determine what the path is Easy to manipulate, conceptual representation Factor motion into path and detail Edit the path Put path and detail back together Clean up the details Media Clip

53 Motion Use Ideal Capture a large corpus of motion Everything you need is there Somewhere but where Maybe it needs some adaptation Have some specification of needs Generate motion from this

54 Current approaches Move Trees Specially designed motions that connect Requires planning to create Requires tricks to fit together Games use 1000s (yes thousands) of motions! Graphs/motions done manually Style Machines???

55 Goal Corpus of of Motions Preprocessing Requests Abstracted Representation Generate motion, on demand, using an abstracted representation

56 First (working) Example A few notable failures Best Linear Unbiased Estimators Dense tables Simple seems to work State matching Rigid factoring (timescaling soon) Graph structures are useful Today: random walks Tomorrow:??? Media Clip

57 Tricks up our sleeves General Transitions are hard Can be arbitrarily hard Can be really easy (if similar motions) Identify when basic techniques work Factored representations More things to match More ways to apply things

58 Thanks! UW Graphics Gang Mainframe Entertainment, House of Moves Studios, Digital Domain This research is supported by: Microsoft, Intel, Pixar, IBM NSF Career Award