Virtual Production for the Real World Using Autodesk MotionBuilder 2013

Transcription

1 Virtual Production for the Real World Using Autodesk MotionBuilder 2013 Daryl Obert Autodesk Hein Beute- Xsens DG3148 This class will give attendees a firm understanding of the concepts and workflows involved in motion capture for virtual production. We will provide a basic overview of working with Autodesk MotionBuilder software and describe some of the options available for getting data in on a low budget. Learning Objectives At the end of this class, you will be able to: Explain how motion capture works List options for doing motion capture on a low budget Set up assets for use in MotionBuilder Work around common re-targeting issues About the Speaker Daryl is a 20-year CG veteran who started working for Alias in 1995 as an application engineer. His current role at Autodesk is Senior Technical Marketing Manager. Daryl is responsible for building many of the presentations used at events such as Siggraph, IBC, GDC, and NAB. He has spent his career working with clients, ensuring their success at implementing Autodesk technology. In addition to his role at Autodesk, Daryl was an adjunct faculty member at NYU's Tisch School of the Arts, where he taught rendering and special effects.

2 Motion Capture 101 Motion capture or mocap is the process of recording the movement of objects or people. It is used in military, entertainment, sports, and medical applications. In filmmaking and video game development, it refers to recording actions of human actors, and using that information to animate digital character models in 2D or 3D computer animation. When it includes face and fingers or captures subtle expressions, it is often referred to as performance capture. Motion capture offers several advantages over traditional computer animation of a 3D model: More rapid, even real time results can be obtained. In entertainment applications this can reduce the costs of keyframe-based animation. The amount of work does not vary with the complexity or length of the performance to the same degree as when using traditional techniques. This allows many tests to be done with different Complex movement and realistic physical interactions such as secondary motions, weight and exchange of forces can be easily recreated in a physically accurate manner. The amount of animation data that can be produced within a given time is extremely large when compared to traditional animation techniques. This contributes to both cost effectiveness and meeting production deadlines. Methods of getting the data There are many types of motion capture systems that have a range of price and capabilities. For this class we will focus our discussion on three different types of systems that are commonly used. Optical systems Optical systems utilize data captured from image sensors to triangulate the 3D position of a subject between one or more cameras calibrated to provide overlapping projections. The cameras will emit red or inferred light that will illuminate special reflective markers attached to an actor or object being tracked. These systems produce data with 3 degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of three or more markers; for instance shoulder, elbow and wrist markers providing the angle of the elbow. This conversion of maker cloud data to a solved skeleton can either be handled by MotionBuilder or the mocap system s native software like Natural Point s Arena. 2

3 Natural Point has a starter system based on six cameras with an entry price of around 7k. Tracking a large number of performers or expanding the capture area is accomplished by the addition of more cameras. You can also increase the quality of the camera to have a higher frame rate, resolution, and range. When you move up into the hi-end cameras, the ability to capture more than just the body becomes a reality. Games tech You can use low cost games technology like the Xbox or PlayStation cameras. The consumer level systems based on gaming technology do come with some compromises. The quality of the data is not as high, the range of motion that can be captured is limited, and the capture size is small. Microsoft's Kinect system, released for the Xbox 360, projects an invisible infra-red pattern for depth recovery motion acquisition. Kinect relies on a pair of depth sensors to measure the distance of objects in the room in three dimensions by emitting infrared 'structured light' beams which project a specific grid pattern that is distorted based on the person's distance from the emitter. These images are measured by an 11-bit 640 x 480 pixel monochrome CMOS sensor providing 2,048 levels of grey, which builds a map showing the distance from the sensor of every point in the image. The Kinect uses a skeletal model that breaks down the human body into dozens of segments and joints. Using the skeleton model programmers have built some great hacks. Brekel-Kinect is an application using a Microsoft Kinect, and PrimeSense s OpenNI and NITE to preform skeleton tracking and stream the data into MotionBuilder. 3

4 Inertial based systems Xsens MVN Motion Capture uses 17 state of the art MEMS inertial sensors attached to the body by a lycra suit. Using sensor fusion algorithms and validated biomechanical models, highly accurate kinematic data is generated and passed on to MotionBuilder. Xsens MVN gives you freedom of movement and no limit to the capture size because MVN uses no cameras. It is a flexible and portable Motion Capture system that can be used indoors and outdoors, not only on-set in the studio but also outside. Xsens MVN requires minimal clean-up of captured data as there is no occlusion or marker swapping. The ease of use and short calibration time allows you to set up the system in less than 15 minutes. Working with MotionBuilder MotionBuilder is a real-time 3D animation solution with a focus on interactive real-time workflows for feature film, video game, and broadcast productions. MotionBuilder is designed for 3D data acquisition, manipulation and visualization, which makes it not only an animation productivity tool, but also a tool that drives the iterative process of creativity. At the core of Motion builder is HumanIK, a full-body inverse kinematics (FBIK) solver that that uses a biomechanical model to realistic move your characters. Motion Builder UI MotionBuilder has a UI that is built with QT so it is very customizable; its layout is a little different than most 3d content creation tools but once you know where everything is you will be able to work quickly. 4

5 Getting assets into MotionBuilder MotionBuilder s native file format.fbx is supported by most animation packages. FBX assets may include a model created in another 3D software package and exported using an FBX plug-in, animation data, or any other assets created and saved using MotionBuilder including lights, cameras, control rigs, and so on. In addition to exporting.fbx files Maya and 3ds Max has the ability to use the Single Click workflows to add assets to MotionBuilder. Character mapping When you bring a character model into MotionBuilder, you must define the structure of that character by completing the Mapping list. This process is referred to as Character mapping. Essentially, Character mapping tells MotionBuilder that on this character skeleton, these are the arms, these are the legs, this is the head, and so on. The action of activating the defined Character map is called characterizing. This is done after you have mapped all of the joints and clicked the Lock button. When you characterize a model, it is no longer just a mesh and a skeleton, MotionBuilder recognizes it as a character that can be animated. Before characterizing the model make sure it is in the T-pose facing down the Z-axis. This makes an offset map that zeros out the local joint rotations and gives all of the characters a common starting point. To set up a character fallow these steps in the Character Controls window. Character is in T-pose facing down the Z-axis Create a Character Double click on joint in the Definition tab 5

6 Select matching joint on the character s skeleton in the Viewer (crtl a = hotkey for x-ray) After all joints are mapped Lock the character Once you have a characterized character, you can select what type of motion source you want to control it, such as a Control rig, Actor, or other character. Control Rig The Control Rig really is where the magic happens in Motion Builder, it is a FBIK custom rig that is based on the HumanIK solver. I always create a Control Rig with IK (Inverse Kinematics) and FK (Forward Kinematics) effectors. This give the most control to work around common retargeting issues that will come up. When working with the Control rig we can chose between three keying modes in the Character Control window. The modes control how the FBIK solver will interact with the character as well as what nodes will get keyframes. The joints that will key keyed are stroked in the Character Control window. Body Part will key the select limb(like the arm or leg) IK will stop at the end of a body part Full body this will key all joints and enables HumankIK Selected keys only the current selected nodes When setting keyframes on the Control rig, FK and IK get keyed at the same time. You can use the Reach sliders to adjust how much contribution each result will affect the character. This ability to blend between FK and IK is extremely useful when retargeting mocap data onto dissimilar sized characters. 6

7 When posing the character you can easily pin any body part to stop its movement in translation and rotation. At any time you can return your character back to its Stance pose by clicking the stance icon in the Character Control window. Whatever joints that are highlighted in the Character Control window will be affected. Devices A device is either a special software component (such as the Voice and Sound devices) or a separate piece of hardware (joystick, mouse, or keyboard) that connects to your computer using a serial, ethernet, or other connection. Devices react to either recorded data or live input directly from the device itself. An input device feeds data to MotionBuilder, and an output device receives data from MotionBuilder. For example, the keyboard and the mouse are input devices, and the MIDI device is both an input and output device. Most of the mocap systems have developed custom devices that will stream their data into Motion Builder in real time. Depending on the system the type of data will change Kinect-Brekel device plugin brings in the motion data as a solved skeleton Xsens device plugin brings in the motion data as a solved skeleton Natural Point device plugin brings the motion data in as an optical data markers that need to be connected with MotionBuilder s Actor to do the skeleton solving before being used as an input source for a character 7

8 Recording live data coming in Motion Builder can record the live data coming in from multiple devices and use different frame rates for each data stream. For example you may be recording the body movement at 48fps, the face at 60fps, and a video reference at 24fps. If needed an external time code can be provide and used to help track all the data that is recorded. Motion builder 2013 has ability to record live data directly to disk using Motion Builders Story tool. The ability to record to disk becomes crucial when dealing with very large scene files that can easily outgrow a system's memory limit. Instead of storing the data into the memory buffer, you can store the data in real time to disk and also choose to record discrete Animation track, Character track, and Camera track clip files to disk. The steps to record to disk are: Create a Character Track in Story by right clicking Set the path for the data in the Character Track Record Options Click the Record button in the Timeline Hit Play As the recoding happens you can see the clip getting added to Story. In the Character track s naming option that is accessible in the Properties tab you change the naming tokens used for the clips. The naming tokens will allow you to integrate the recorded clips with a pipeline management tool like Shotgun. This metadata is important when doing performance capture due to the volume of data that is generated. Fixing common retargeting problems By default Motion Builder tries to match the FK, this means that the characters joint angles that are being retargeted will match the Source mocap. Motion Build looks to the length of the leg and auto scales the translation to adjust for differences in character size; this is called Action Space Compensation. For simple movements this is all that is needed but for more complex moves sometimes we will want the retargeted character s body part to reach the same world position as the source mocap. This is easily 8

9 accomplished with the IK Blend slides. When you start to increase the IK Blend T value you are actually using IK to translate the characters position on that body part. If the retargeted body part that is trying to reach the sources world position is too short you can use the IK Pull slide to enable HumanIK on the body part, this will allow the solver to pull past the body part and affect all of the joints in the rig. Plotting Animation Plotting is sometimes referred to as baking or transferring, because it transfers all animation in your scene to selected models. There are many times when you would need to plot animation, here are some examples. Plotting FK skeleton animation on a character to the Control rig. Plotting Story Character track clips of recorded mocap data onto the Control rig. When you are finished creating or modifying the animation on the Control rig, before you save, you can plot the animation back to the model s skeleton. This prepares your animation data to be exported so you can work with it in other software packages. When you have keyframes on multiple layers, you can also plot to merge the animation from all layers to the Base Layer. Smart Plot options let you determine the number of keyframes that are added during the plotting process, to simplify editing the plotted animation. Animation layers You can create and blend multiple layers of animation in a scene. You can create layers to organize keyframe animation, or to keyframe on top of animation without overwriting the original curves. Animation layers hold animation in your scene, with each layer containing the keyframes you set. The animation layers appear stacked in the Animation Layer editor, and depending on various settings, blend to create the result animation that plays in your scene. I often use animation layers to fix simple problems in the mocap data or to add offsets on arms so they do not pass through the character s body. Layers are super useful to make incremental changes, and compare and preview the results on different layers. Auxiliary Effectors and Pivots Auxiliary effectors are supplementary effectors in a Control rig that correspond to existing IK effectors. Every IK effector in a Control rig can have multiple Auxiliary effectors. Auxiliary effectors are useful in many situations to provide an additional level of IK control. For example, you can use Auxiliary effectors to ensure that an arm in the character always reaches towards a prop, such as a weapon. 9

10 You can also use Auxiliary effectors to stabilize a feet in the character so that they do not slide on the floor. To do this, you can create an Auxiliary effector for the primary IK foot effectors, then place the Auxiliary effector on the floor where the foot slides. Define the Blend values to maximum reach (100%). Auxiliary pivots are objects that let you quickly define and animate multiple rotation pivot points for IK Control rig effectors. You can use Auxiliary pivots for any character animation, and they are particularly useful if you want to manipulate the feet or hands of a character using more than one rotational point. By creating multiple Auxiliary pivots for an IK effector in the foot of a character, for example, the feet can rotate around multiple independent pivot points in order to create naturallooking walk cycles. To add Auxiliary Effecters or Pivots right click on any of the existing IK effectors that are represented by round dots in the Character Control window. 10