Motion Capture and 3D Animation

Transcription

1 Motion Capture and 3D Animation Prof. Marcos Fernández Instituto de Robotica UVEG

2 Motion Capture recording of motion for immediate or delayed analysis or playback - David J. Sturman The creation of a 3d representation of a live performance - Alberto Menache is a technique of digitally recording movements for entertainment, sports, and medical applications. - Wikipedia

3 History of Motion Capture Eadweard Muybridge ( ) first person to photograph movement sequences

4 History of Motion Capture Eadweard Muybridge ( ) first person to photograph movement sequences whether during a horse's trot, all four hooves were ever off the ground at the same time. the flying horse Sequence of a horse jumping (courtesy of E. Muybridge)

5 History of Motion Capture Eadweard Muybridge ( ) first person to photograph movement sequences the flying horse animal locomotion (20k pictures about men, women, children, animals, and birds). Woman walking downstairs (courtesy of E. Muybridge)

6 History of Motion Capture Eadweard Muybridge ( ) Father of the motion picture several cameras successive pictures photographs of human and animal motion zoopraxiscope (zoogyroscope, zoetrope) a device for playing still images in sequence

7 History of Motion Capture Etienne Jules Marey( )

8 Etienne Jules Marey( ) First Mocap Suit History of Motion Capture

9 History of Motion Capture Harold Hedgerton( ) high speed and stop motion photography exposures as small as a millionth of a second electronic flash stroboscope

10 Rotoscope Allow animators to trace cartoon character over photographed frames of live performances invented by Max Fleischer in 1915

11 Rotoscope Allow animators to trace cartoon character over photographed frames of live performances invented by Max Fleischer in D manual motion capture A horse animated by rotoscoping from Muybridge s photos

12 Rotoscoping rotoscoping can be thought of as a primitive form or precursoro to motion capture, where the motion is captured painstakingly by hand. - Sturman

13 Allow animators to trace cartoon character over photographed frames of live performances invented by Max Fleischer in D manual capture Rotoscope the first cartoon character to be rotoscoped -- Koko the clown the human character animation -- snow white and her prince (Walt Disney, 1937)

14 Digital Motion Capture Technologies 3D Rotoscoping : measuring 3D positions, orientations, velocities or accelerations more players commercial players multiple uses 70 s: development of magnetic systems 80 s: development of optical systems 90 s: mocap is hot, 00 s: mocap is used more frequently for feature films

15 Overview of MoCap Systems Types of Mocap systems: Outside-In» sources (e.g., reflective markers) on body» external sensors (e.g., cameras)» optical systems Inside-Out» sensors on body» external sources» magnetic systems Inside-In» sources and sensor on body» mechanical systems

16 Electromagnetic Mocap Each sensor record 3D position and orientation Each sensor placed on joints of moving object Full-body motion capture needs at least 15 sensors Popular system:

17 Pros Electromagnetic Mocap measure 3D position and orientation no occlusion problems can capture multiple subjects simultaneously Cons magnetic perturbations (metal) small capture volume cannot capture deformation (facial expression) hard to capture small bone movement (finger motion) not as accurate as optical mocap system

18 Electromechanical Mocap Each sensor measures 3D orientation

19 Electromechanical Mocap Each sensor measures 3D orientation Each sensor placed on joints of moving object Full-body motion capture needs at least 15 sensors Popular systems:

20 Pros Electromechanical Mocap measure 3D orientation no occlusion problems can capture multiple subjects simultaneously large capture volume Cons getting 3D position info is not easy cannot capture deformation (facial expression) hard to capture small bone movement (finger motion) not as accurate as optical mocap system

21 Fiber Optic Mocap/ Prosthetic Measures 3D position and orientation of entire tape Binding the tape to the body Popular systems:

22 Pros Fiber Optic Mocap/ Prosthetic measure 3D orientation and position no occlusion problems can capture multiple subjects simultaneously go anywhere mocap system can capture hand/finger motion Cons intrusive capture cannot capture deformation (facial expression) not as accurate as optical mocap system

23 Optical Mocap Multiple calibrated cameras (>=8) digitize different views of performance Wears retro-reflective markers Accurately measures 3D positions of markers

24 Optical Mocap Vicon mocap system:

25 Pros Optical Mocap Cons measure 3D position data also orientation the most accurate capture method very high frame rate can capture very detailed motion (body, finger, facial deformation, etc.) has occlusion problems hard to capture interactions among multiple ppl limited capture volume

26 MoCap Planning Mocap Pipeline Introduction to Motion Capture Sensor Calibration Compute the position and orientation of camera/transmitter Subject Calibration Place markers on body, compute skeleton structure(lengths of links) Record Movements Marker positions (and orientations) varying over time Mapping to characters Direct Mapping or retargetting Convert to Joint Angles Occlusion and correspondence problems Marker Data Cleanup Occlusion and correspondence problems

27 Planning Motion capture requires serious planning Anticipate how the data will be used Garbage in garbage out Shot list Games motions need to be able to blend into one an another capture base motions and transitions which motions transition into which other transitions cycles/loops

28 Movement Flowchart for Games Planning and Directing Motion Capture For Games By Melianthe Kines Gamasutra January 19, 2000 URL:

29 Character/prop set up Planning - character skeleton topology (bones/joints number, Dofs for each bone) - location and size of props Marker Setup - the number of markers - where to place markers

30 Camera Calibration: Calibration determine the location and orientation of each camera determine camera parameters (e.g. focal length) Subject calibration - determine the skeleton size of actors/actresses (.asf file) - relative marker positions in terms of bones - determine the size and location of props

31 Processing Markers Each camera records capture session Extraction: markers need to be identified in the image determines 2d position problem: occlusion, marker is not seen use more cameras Markers need to be labeled which marker is which? problem: crossover, markers exchange labels may require user intervention Compute 3d position: if a marker is seen by at least 2 cameras then its position in 3d space can be determined

32 Data Process 3D marker positions (.c3d file) Fill in missing data Filter mocap data Complete 3D marker trajectories (.c3d file)

33 Data Process

34 Animation with Mocap the character is controlled by skeleton to control the skeleton, need to specify joint rotations

35 Animation with Mocap capture motion on performer positions of markers are recorded retarget motion on a virtual character motion is usually applied to a skeleton a skeleton is hierarchical linked joints need rotation data! need to convert positions to rotations

36 Animation with Mocap the actor is used to convert marker positions to rotational data» markers are handles on the actor» actor should have similar proportions as the performer joint rotations of the actor are applied to the character there are still issues with proportions

37 Animation with Mocap Some Retargeting problems Performer ->Actor->Character

38 Animation with Mocap Some Retargeting problems Performer ->Actor->Character

39 Motion Capture Data Files Each sequence of human motion data contains two files: Skeleton file (.asf): Specify the skeleton model of character Motion data file (.amc): Specify the joint angle values over the frame/time Both files are generated by Vicon software

40 Human skeletal file Described in a default pose

41 Human skeletal model This is still a tree!

42 Human skeletal file (.asf) individual bone information - length of the bone - direction of the bone - local coordinate frame - number of Dofs - joint limits bone hierarchy/connections

43

44

45 Motion data file (.amc) i // frame number root // root position and orientation lowerback // joint angles for lowerback joint upperback // joint angles for thorax joint thorax lowerneck upperneck head rclavicle e e-014 rhumerus rradius rwrist rhand rfingers rthumb lclavicle e e-014 lhumerus

46 Data Process 3D marker positions (.c3d file) Fill in missing data Filter mocap data Complete 3D marker trajectories (.c3d file) Inverse Kinematics Joint angle data (.amc file)