Adapting Motion Capture Data to Follow an Arbitrary Path

Transcription

1 Adapting Motion Capture Data to Follow an Arbitrary Path Submitted in partial fulfilment of the requirements of the degree Bachelor of Science (Honours) of Rhodes University Mark Whitfield 8th November 2004

2 Abstract This paper describes the implementation of a method to adapt motion capture data to follow an arbitrary path. The central idea is to create a path which represents an abstraction of the motion. The motion is then represented relative to the path. Thus altering the path alters the motion. This method is successful in getting the avatar to follow an arbitrary path. However, results using this method (which does not incorporate constraints) exhibit undesirable side-effects like footskate. However, if the path corresponds closely to the actual motion we can minimize these effects.

3 Acknowledgements Thanks to my supervisors, Shaun Bangay and Adele Lobb, for their support and advice throughout the year. I acknowledge the financial and technical support of this project by Telkom SA, Business Connexion, Comverse SA, and Verso Technologies through the Telkom Centre of Excellence at Rhodes University.

4 Contents 1 Introduction Problem Statement Motivation Document Structure Related Work Altering a Base Motion Synthesis Cyclic Motions BVH Files Least Squares Fitting Euler Step Integration Summary Design Paths Creating the Original Path Representing the Motion Relative to the Path Absolute Residuals Residuals Relative to the Path Altitude Adjusted Paths Arc-Length Reparameterization Calculating Arc-Length Transformation End of Path Handling Cycling

5 CONTENTS Editing the Path Summary Implementation Original Path Residuals Calculating the Residuals at each Frame Orientating the Avatar at a Frame on the New Path Translating the Avatar at a Frame on the New Path Arc-length Reparameterization Calculating Arc-Length on Original Path Finding the Co-Ordinates on a New Path End of Path Handling Cycling User Interface Rendering Objects Arc-Length Reparameterization Cycling Speed Original Path New Path Tracking the Path Camera Views Summary Results Arc-Length Reparameterization End of Path Handling Cycling Original Path Degree Converting a Simple Path to a Complex Path Converting a Complex Path to a Simple Path Converting a Complex Path to another Complex Path Uphill and Downhill Paths Summary

6 CONTENTS 4 6 Conclusion Conclusions Future Work References 58

7 Chapter 1 Introduction 1.1 Problem Statement Getting an avatar to follow an arbitrary path is a trivial problem whether we are using motion capture data or not. Getting the motion to look realistic as well as follow the path is not a trivial problem. A tool for editing the path of motion capture data needs to be built using a suitable method. Michael Gleicher [Gleicher 2001] introduces a method for editing the path of a motion. This method is implemented. The characteristics of the original motion should be preserved. For example we do not want the feet of the avatar to slide around in the adapted motion if they do not slide in the original motion. The speed of the avatar should also be preserved. The tool must be programmed in Linux using C++. This is because the tool must be compatible with the virtual reality system already being used in the Computer Science Department. This system was built in C++ using the Linux environment. A suitable user interface needs to be built to allow for the easy manipulation of motion capture data. 1.2 Motivation The subtle details of human motion are present in motion capture data. These subtle details are key to the realism of the motion. Furthermore, a variety of locomotion styles may be captured. 1

8 CHAPTER 1. INTRODUCTION 2 Thus motion capture data provides a good basis for creating realistic avatar movement. Motion capture is an expensive and time-consuming process. Therefore, it is desirable to be able to alter previously recorded data. For example we may have motion capture data of a man walking in a straight line, but we need him to walk around an obstacle. Instead of spending time and money on recording more data, a motion capture data manipulation technique can be employed. 1.3 Document Structure Chapter 2 deals with related work. We will look at some differing views on how to get an avatar to follow an arbitrary path using motion capture data. Chapter 3 details the design of the system proposed by Gleicher. Chapter 4 relates to the implementation of the system described in Chapter 3. Chapter 5 deals with the results of experiments done to test the system. Chapter 6 discusses the conclusions reached after analyzing the results of the experiments.

9 Chapter 2 Related Work There are three main schools of thought for getting an avatar to follow an arbitrary path using motion capture data: The desired path is followed by editing a base motion. A new motion is synthesized that fits the desired path. A cyclic motion is applied over and over again as the avatar is displaced along the path. 2.1 Altering a Base Motion Gleicher [Gleicher 2001] uses a method based on creating a path which represents an abstraction of the motion. The motion is then represented relative to the path. Thus altering the path alters the motion. This method forms the basis of this document, and will be discussed in detail in the following chapters. The motion of an avatar can be specified by the root node translation and orientation at each frame, as well as each joint angle at each frame. Each of these parameters (e.g. knee joint) can be represented as a function of time. A motion curve is any one of these parameters as a function of time. Witkin and Popovic [Witkin and Popovic 1995] use a technique based on warping motion curves. Keyframes are used as constraints in the motion warp. Although Witkin and Popovic do not directly address the problem of path transformation, we could warp the motion curves so that the avatar follows the desired path by setting certain keyframes which the avatar must pass through. 3

10 CHAPTER 2. RELATED WORK 4 Methods which involve altering a base motion only require a single suitable motion clip to work with. It is still desirable to have a fairly large library of motion capture data. The desired clip can be chosen from this library, and then altered accordingly. Blending may be used to create the base motion by joining two or more motions together, thus creating a longer path. Constraints may be required to prevent undesirables like footskate. 2.2 Synthesis With synthesis methods, a sequence of motion capture clips is put together so as to follow the desired path. This sequence is extracted from a library of clips, usually stored in a graph structure. Kovar et al. [Kovar et al. 2002] construct a directed graph from a library of motion capture data. This graph consists of nodes which represent possible transitions from one motion capture clip to another. The user specifies the transition threshold which will determine the connectivity of the graph. A higher threshold leads to a greater number of possible transitions. A cost value is associated with each possible transition, and motion is generated by building walks on the graph. A branch and bound search is used to find sequences which satisfy user demands such as sketched paths. An interesting feature of this paper is that the style of locomotion can be specified for the sketched path. This is achieved by labelling clips. For example certain clips may be labelled as sneak. This allows for constraints to be set such that only a certain locomotive style is used in certain intervals. Lee et al. [Lee et al. 2002] use a similar technique to Kovar et al. They also construct a directed graph from motion clips containing possible transitions. A cost formula incorporating joint angles, velocities and positions is used to identify the cost of possible transitions, and this data is stored in a matrix. The sketched path is considered as a sequence of goal positions to be traversed. The tracking algorithm used by Lee et al. is based on a best-first search through the directed graph. Motion synthesis methods require a large database of motion capture data if they are to cater for a wide variety of paths while still maintaining the quality of the motion. It is possible to work with a smaller database and lower the standard for selecting transitions. However, this will be evident in the quality of the motion. Synthesis methods also require constraints to avoid undesirables like footskate. Blending is an integral part of synthesis methods as the two frames associated with a transition may not be similar enough for a smooth transition to occur.

11 CHAPTER 2. RELATED WORK Cyclic Motions A common technique used by animators is to define locomotion cycles and loop these cycles while the character is moving along. This technique has been used since the earliest productions [Lutz 1920]. This idea can be extended to motion capture data. Motion capture sequences which transition onto themselves are used. This single cycle is applied over and over as the avatar is displaced along the path [Rose et al. 1998]. This method does not provide a way to alter an existing complex path. 2.4 BVH Files BioVision Hierarchy (BVH) is a format for storing motion capture data. BVH files are separated into two main sections: HIERARCHY and MOTION. The first section begins with the word HIERARCHY and defines a hierarchy of bones (example in figure 2.1). For example the LeftHip and RightHip fall just under the Hips in the hierarchy. The LeftKnee falls under the LeftHip. The offset of each bone from the bone just above it in the hierarchy is given. A channel describes an aspect of the translation or orientation of a bone for each frame. For example the Hips has 6 channels: one for each of the translation co-ordinates and one for each of the rotation co-ordinates. The second section starts with the word MOTION (example in figure 2.2). The number of frames as well as the time interval between frames is given. Finally the value of each channel at each frame is given. The BVH specification is given below. In the specification pseudonyms for the names of the root and the joints may be used. For example Hips may be known as Pelvis. HIERARCHY ROOT Hips { OFFSET Xoffset Yoffset Zoffset CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotat JOINT LeftHip { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT LeftKnee

12 CHAPTER 2. RELATED WORK 6 { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT LeftAnkle { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation End Site { OFFSET Xoffset Yoffset Zoffset } } } } JOINT RightHip { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT RightKnee { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT RightAnkle { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation End Site { OFFSET Xoffset Yoffset Zoffset } } } } JOINT Chest

13 CHAPTER 2. RELATED WORK 7 { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT LeftCollar { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT LeftShoulder { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT LeftElbow { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT LeftWrist { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation End Site { OFFSET Xoffset Yoffset Zoffset } } } } } JOINT RightCollar { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT RightShoulder { OFFSET Xoffset Yoffset Zoffset

14 CHAPTER 2. RELATED WORK 8 CHANNELS 3 Zrotation Xrotation Yrotation JOINT RightElbow { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT RightWrist { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation End Site { OFFSET Xoffset Yoffset Zoffset } } } } } JOINT Neck { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation JOINT Head { OFFSET Xoffset Yoffset Zoffset CHANNELS 3 Zrotation Xrotation Yrotation End Site { OFFSET Xoffset Yoffset Zoffset } } } } } MOTION

15 CHAPTER 2. RELATED WORK 9 Figure 2.1: An example of part 1 of a bvh file. Frames: number_of_frames Frame Time: time_between_frames frame1_channel1_value... frame1_channeln_value frame2_channel1_value... frame2_channeln_value... framen_channel1_value... framen_channeln_value 2.5 Least Squares Fitting Least squares fitting to a polynomial involves finding the best fitting curve to a set of n points: (x 1, y 1 ),(x 2, y 2 ),(x 3, y 3 ),...,(x n, y n ). The residual for a point is its vertical offset from the curve:

16 CHAPTER 2. RELATED WORK 10 Figure 2.2: An example of part 2 of a bvh file.

17 CHAPTER 2. RELATED WORK 11 y (x,y ) 1 1 R 1 R (x,y ) 2 2 (x,y ) R 3 (x,y ) 5 5 R 4 (x,y ) 4 4 R 5 R 6 (x,y ) 6 6 x Figure 2.3: Least squares fit to a polynomial of degree 2.

18 CHAPTER 2. RELATED WORK 12 R i = y i f(x i ). The best fit for a given polynomial degree is the fit where the sum of the squares of the residuals ( n i=1 Ri 2 ) is at a minimum. Figure 2.3 shows a least squares fit to a polynomial of degree 2 using the points in the figure. Refer to [Weisstein 1999] for a detailed explanation of the mathematics involved in finding the best fitting curve using the least squares method. 2.6 Euler Step Integration The idea behind Euler s method is to approximate a curve with a series of straight lines. Euler step integration assumes that the rates computed at a given time are constant throughout the timestep. This is why it is not very accurate. y i+1 = y i + hg (x i ) (2.1) Equation 2.1 shows how we can approximate the curve g(x) using Euler s method. In this equation, y i is the current y value and y i+1 is the next y value. Sampling occurs at time intervals of h. The gradient of the line (m) is calculated at x i by finding the value of g (x i ). This value is then multiplied by the timestep (h). This will give us the change in y (equation 2.2). The change in y is then added to y i to give us the next value of y i+1. and m = y i+1 y i x i+1 x i h = x i+1 x i Therefore y i+1 y i = hm (2.2) Figure 2.4 uses a very large timestep and thus gives a poor approximation of g(x). Figure 2.5 uses a timestep of half the size. This gives a better approximation of g(x). A better approximation of g(x) can be achieved by taking a smaller timestep.

19 CHAPTER 2. RELATED WORK 13 g(x) y y 3 y y 2 1 h h x x x x Figure 2.4: Euler s method using a very large timestep.

20 CHAPTER 2. RELATED WORK 14 g(x) y y 5 y 4 y 2 y 3 y 1 h h h h x x x x x x Figure 2.5: Euler s method using a smaller timestep.

21 CHAPTER 2. RELATED WORK Summary Motion synthesis methods require a larger database of motion capture data to work with than methods which alter a base motion. Constraints are required in both of these methods to prevent undesirable effects like footskate. Blending is an integral part of synthesis methods, and is also applicable to methods which alter a base motion if a longer base motion is desired. Methods which involve altering a base motion provide a way to alter an existing complex path, whereas cyclic methods do not. The BVH specification is a popular format for storing motion capture data. Least squares fitting to a polynomial provides us with the best fit of a set of data points to a curve. Euler step integration approximates a curve with a series of straight lines.

22 Chapter 3 Design This section details the design of the system. The system must provide the user with a means to adapt motion capture data to follow an arbitrary path. Furthermore, the new motion should look realistic. The concepts of paths, residuals and arc-length reparameterization as well as how they fit together in the system is dealt with in this chapter. 3.1 Paths Motion capture data does not explicitly contain a path, but rather a series of translations and orientations. In order to give meaning to the motion for the user, a path is constructed based on the translations in the motion capture data. The path can be thought of as an abstraction of the motion, where smaller details are disregarded Creating the Original Path The root node translation data is extracted from the motion capture data. The original path is created by applying a least squares fit of the root node translation data to a polynomial Representing the Motion Relative to the Path Absolute Residuals We can describe the avatar s position relative to the path s position. We calculate the translation transformation from the original path s position at a given time to the position of the original motion at that time. This is known as the translation residual. The direction of the path at a 16

23 CHAPTER 3. DESIGN 17 t R A t R B tangent direction R C tangent direction Figure 3.1: R represents the residual at time t on the path. On the left is the original zig-zag motion and corresponding path. On the right is the new path. If we represent the residuals in the absolute sense, then we will transform to point B. If we represent the residuals relative to the path, then we will transform to point C.

24 CHAPTER 3. DESIGN 18 (a) (b) (c) Figure 3.2: Original zig-zagging motion and original path (a). New path and corresponding motion with residuals represented in the absolute sense (b). New path and corresponding motion with residuals represented relative to the path (c). given time is defined as its tangent vector. We can also calculate the rotation transformation from the original path s direction to the direction of the motion at that time. This is known as the orientation residual. The path is then changed. To calculate the avatar s new position we transform onto the path and then transform using the residuals. orientation residual = transformation from the direction of the path to the direction of the avatar at a given time; translation residual = transformation from the position of the path to the position of the avatar at a given time; This method for representing the motion is not recommended. Figure 3.1 shows how the residuals will be applied if we represent them in the absolute sense. Figure 3.2b shows the undesirable effects when the direction of motion is changed. The method detailed in section is the preferred method.

25 CHAPTER 3. DESIGN 19 Y Y Y X X X Z Z Z (a) (b) (c) Figure 3.3: Initially the avatar is at the origin of the moving co-ordinate system facing down the z-axis (a). We rotate the avatar using the orientation residual for the frame (b). Next we translate the avatar using the translation residual for the frame (c) Residuals Relative to the Path Figure 3.1 shows the result of representing the avatar s position relative to the path as opposed to representing it in the absolute sense. We use a moving co-ordinate system to implement this technique. This is the preferred method for representing residuals. We use a y-up moving co-ordinate system: the y-vector of the co-ordinate system faces upwards. This ensures that the avatar is will remain upright even if the altitude of the path is changed (section 3.1.3). The z-axis points in the direction of the tangent projected on the horizontal. The x-axis is their cross product. In order to position and orientate the avatar correctly for a frame, first we transform using the residuals in this co-ordinate system (figure 3.3). Then we transform to place this co-ordinate system in the world co-ordinate system (figures 3.4, 3.5 and 3.6). In order to calculate the residuals we simply reverse this process using the original path and original motion. Assuming the original avatar is correctly positioned and orientated in the world co-ordinate system, first we move the origin of our moving co-ordinate system back to the origin of the world co-ordinate system (moving the avatar with it). Then we rotate our moving coordinate system so that the z-axis of the moving co-ordinate system is lined up with the z-axis of the world co-ordinate system (rotating the avatar with it). Now we can calculate the residuals. The translation residual will be the translation required to get from the origin of the world co-

26 CHAPTER 3. DESIGN 20 Y Y X t X T Z Z Figure 3.4: Initially the origin of our moving co-ordinate system is positioned at the origin of our world co-ordinate system. The x-axis, y-axis and z-axis face in the same direction as those in the world co-ordinate system. The frame we are going to render occurs at time t on the path. T represents the tangent projected on the horizontal at time t on the path. ordinate system to the position of the avatar. The orientation residual will be the rotation required to get from the direction of the z-axis of the world co-ordinate system to the orientation of the avatar Altitude Adjusted Paths If we represent the motion relative to the tangent, then when we adapt the path to go uphill or downhill the posture of the avatar will change (figure 3.7b). It now looks as if the avatar should fall over backwards, but he will continue to move up the path. This results in a highly unrealistic looking motion. If we represent the motion relative to the tangent projected on the horizontal, then even when we adapt the path to go uphill or downhill, the avatar will retain his original posture (figure 3.7c). This results in a more realistic looking motion. This method will not be able to handle an original path or an new path which has segments where the tangent vector faces vertically upwards or downwards. This is because the tangent vector projected on the horizontal will be 0.

27 CHAPTER 3. DESIGN 21 Y Y X Z t X T Z Figure 3.5: Next we rotate our moving co-ordinate system such that the z-axis is facing in the same direction as the tangent projected on the horizontal at time t on the path. Y Y X t X T Z Z Figure 3.6: Finally we translate our moving co-ordinate system so that the origin is positioned on the path at time t. Our avatar is now in the desired position and orientated correctly.

28 CHAPTER 3. DESIGN 22 (a) (b) (c) Figure 3.7: Original motion and path (a). New motion representing the motion relative to the tangent (b). New motion representing the motion relative to the tangent projected on the horizontal (c). Figure 3.8: Original path (left) and new path (right) parameterized by time. Figure 3.9: Original path (left) and new path (right) parameterized by arc-length.

29 CHAPTER 3. DESIGN Arc-Length Reparameterization Since the path is parameterized by time, altering the path may also alter the speed of the avatar. For example if we lengthen the path, the speed of the avatar will increase (Figure 3.8). This is generally undesirable. By parameterizing the path by arc-length, the original speed is maintained (Figure 3.9) Calculating Arc-Length The arc-length of frame f is the distance along the path from the start of the path to the position on the path at frame f. We calculate the arc-length along the original path for each frame. Frame f in the new path must have the same arc-length as frame f in the original path Transformation The user alters the original path. First we transform in the moving co-ordinate system using the residuals. Then we transform in the world co-ordinate system. We already have the arc-length at each frame on the original path. We will position and orientate the moving co-ordinate system at the point on the new path with the same arc-length End of Path Handling If the new path is shorter than the original path, and we are using arc-length reparameterization, the end of the new path will be reached before all the frames in the motion have been rendered. We reset the motion back to the first frame and back to the beginning of the path when the end of the path is reached Cycling If the new path is longer than the original path, and we have arc-length reparameterization enabled, then the new motion may only take up a small portion of the new path. Cycling enables us to repeat the motion until the end of the new path is reached.

30 CHAPTER 3. DESIGN Editing the Path The path is represented as a B-spline with a number of control points. The knots are designed so that the spline passes through the first and last control points. The user has direct control over the control points. This allows the user to alter the path quickly and easily. 3.4 Summary The path is an abstraction of the motion. The original path is created by applying a least squares fit of the root node translation data to a polynomial. The motion is represented relative to the path. Thus altering the path alters the motion. Arclength reparameterization is used to ensure that altering the path does not alter the speed of the avatar. Motions may be looped until the end of the path is reached. In the following chapter the implementation of this design is detailed.

31 Chapter 4 Implementation This section relates to the implementation of the system specified in the previous chapter. The system was implemented in Linux using C++. This chapter begins with the implementation of the original path and the residuals. The implementation of arc-length reparameterization is then discussed. We will also deal with handling new paths which are longer or shorter than the original path. Finally we will look at issues relating to the user interface. 4.1 Original Path Code for a least squares fit in two dimensions was obtained [Kavanagh 2003]. However, we need a least-squares fit in three dimensions. Three separate functions are used for the fitting: f(t) represents the least-squares fit of the root node x translation vs. time data points to a polynomial g(t) represents the least-squares fit of the root node y translation vs. time data points to a polynomial h(t) represents the least-squares fit of the root node z translation vs. time data points to a polynomial The position of the original path at time t is the point [f(t), g(t), h(t)]. The orientation of the original path at time t is taken to be the direction of the tangent vector [f (t), g (t), h (t)]. 25

32 CHAPTER 4. IMPLEMENTATION 26 Figure 4.1: Least squares fit of x vs. t data points using a polynomial of degree 3.

33 CHAPTER 4. IMPLEMENTATION Residuals Calculating the Residuals at each Frame for each frame translate and orientate the avatar as in the original motion; p = the point on the original curve at the same time as the frame; t = translation transformation that would move point p to the origin; apply transformation t to the avatar; r = rotation transformation that would line up the tangent at p projected on the horizontal with the positive z-axis; apply transformation r to the avatar; orientation residual = rotation transformation from the positive z-axis to the current orientation of the avatar; translation residual = translation transformation from the origin to the current position of the avatar; next frame

34 CHAPTER 4. IMPLEMENTATION Orientating the Avatar at a Frame on the New Path note: initially the avatar is at the origin facing down the z-axis. rotate the avatar by applying the orientation residual at the frame; p = the point on the new curve corresponding to the frame; t = the tangent at p projected on the horizontal; r = rotation transformation required to get from the orientation of the z-axis to the orientation of t; apply transformation r to the avatar; Translating the Avatar at a Frame on the New Path translate the avatar by applying the translation residual at the frame; rotate the avatar by applying the rotation residual at the frame; t = translation transformation from the origin to the point on the new path corresponding to the frame; apply transformation t to the avatar;

35 CHAPTER 4. IMPLEMENTATION Arc-length Reparameterization We calculate the arc-length at each frame on the original path. Given a new path, we can find the point on the new path that corresponds to the arc-length of the same frame on the original path. This is the point we transform onto when transforming onto the path Calculating Arc-Length on Original Path We use Equation 4.1 [Okikiolu 2003] to calculate the arc-length between time a (time 0 which is the start of the path) and time b (the time of the frame). We calculate the arc-length at each frame on the original path. L (t) = b a (dx/dt) 2 + (dy/dt) 2 + (dz/dt) 2 dt (4.1) Euler-step integration [Fitzpatrick 2003] is used to perform the integration required in equation 4.1. l i+1 = l i + hl (t i ) (4.2) In equation 4.2, l i represents the current sampled arc-length and l i+1 represents the next sampled arc-length. Sampling occurs at time intervals of h. The first sample, l 1, is the arc-length from t 1 to t 1 which is 0. The second sample, l 2, is the arc-length from t 1 to t 2, where t 2 is equal to the previous time (t 1 ) plus the timestep (h). Sample l i, is the arc-length from t 1 to t i, where t i is equal to the previous time (t i 1 ) plus the timestep (h) Finding the Co-Ordinates on a New Path Given a new path, we can find the point on the path that corresponds to a given arc-length. To find this point we sample the arc-length along the path at small time intervals. The new path is a spline. I was unable to represent the spline in the form required by equation 4.1. Equation 4.1 requires 3 separate functions for representing the x(t), y(t) and z(t). Therefore an alternative technique was employed for finding the arc-length along the new path. L = (x 1 x 2 ) + (y 1 y 2 ) + (z 1 z 2 ) (4.3) Equation 4.3 gives us the distance between two points (x 1, y 1, z 1 ) and (x 2, y 2, z 2 ). We can approximate the arc-length by taking the distance between points on the new path at very small

36 CHAPTER 4. IMPLEMENTATION 30 intervals and finding their sum. We can also sample the arc-length at time intervals of h similar to what we did with Euler-step integration. l i+1 = l i + (x(t i + h) x(t i )) 2 + (y(t i + h) y(t i )) 2 + (z(t i + h) z(t i )) 2 (4.4) The sampling occurs in much the same way as equation 4.2. In equation 4.4 l i represents the current sampled arc-length and l i+1 represents the next sampled arc-length. Sampling occurs at time intervals of h. The first sample, l 1, is the arc-length from t 1 to t 1 which is 0. The second sample, l 2, is the arc-length from t 1 to t 2, where t 2 is equal to the previous time (t 1 ) plus the timestep (h). Sample l i is the arc-length from t 1 to t i, where t i is equal to the previous time (t i 1 ) plus the timestep (h). We sample arc-lengths until the arc-length along the path becomes greater than the desired arc-length. We then take the co-ordinates of the path at that time. Although the final sampled arclength is unlikely to be exactly the same as the desired arc-length, a sufficiently small timestep provides enough accuracy for our purposes. Equation 4.4 is used for the sampling. CalculateCoOrdinates (timestep h) t = 0; sampled arc-length = 0; while (sampled arc-length < desired arc-length) t = t + h; sample the arc-length at time t on the new path; endwhile return the point at time t on the new path; 4.4 End of Path Handling The method used for calculating when the end of the path is reached, is very similar to the method used to find the co-ordinates on a new path (section 4.3.2). We sample the arc-length at small time intervals. If the sampled arc-length becomes greater than the desired arc-length then we have not reached the end of the new path. If our value of t + h increases beyond the end of the spline (i.e. past time 1.0) then we have reached the end of the new path.

37 CHAPTER 4. IMPLEMENTATION Cycling Equation 4.5 is used to calculate the desired arc-length for a frame. d = f c + a (4.5) d = desired arc-length, f = arc-length of the final frame on the original path, c = current cycle number (initially 0), a = arc-length at the current frame. The cycle number is incremented whenever the end of the motion sequence is reached. When the end of the new path is reached, then the cycle number will be reset to 0. The frame number will also be reset back to the first frame. The pose of the first frame and the last frame of the sequence have to be very similar for the transition from one cycle to the next to look authentic. Furthermore, the residual for the first frame and the last frame of the motion sequence will have to be very similar. 4.6 User Interface The user interface should allow the user to alter the path. It should also provide options to aid the user in the process of altering the path Rendering Objects We may not want to render all the objects all the time. For example, we may want to view the new motion on its own. The rendering of any of the objects may be turned on/off Arc-Length Reparameterization The user may wish to retain the speed of the original avatar in the new motion. On the other hand, there may be instances where the user wishes to increase/decrease the speed of the avatar. Arc-length reparameterization may be enabled/disabled by the user.

38 CHAPTER 4. IMPLEMENTATION 32 Figure 4.2: The original path is the straight white path. The new path is the curved white path. The control points are all purple besides the currently selected control point is pink. The original skeleton is red and the new skeleton is green.

39 CHAPTER 4. IMPLEMENTATION 33 Figure 4.3: Tabs Cycling In some cases the user may wish to use cycling, in other cases he may not. For example the user may have a fairly short segment of a person skipping, but he wants the new motion to skip for longer. In this case he would require cycling. In other cases the original motion may be the length of motion that he desires, so he does not want the new motion repeating itself if the end of the path has not been reached. Therefore cycling may be enable/disabled by the user Speed A motion may appear unrealistic, but it may not be apparent why it is so. Slowing the motion down can often reveal subtle imperfections which detract from the realism of the motion. Therefore the speed of the avatar may be adjusted.

40 CHAPTER 4. IMPLEMENTATION Original Path As already mentioned, the original path is created by applying a least squares fit to a polynomial of the root node translation data of the original motion. A higher degree original path generally leads to a tighter fit (i.e. smaller residuals). The user may select the degree of the original path New Path The new path is represented as a B-spline. The user may choose how many control points are used to determine the spline. Furthermore, the user may define the degree of the spline. The user has direct control over the control points and selects and moves the control points using the keyboard. The distance that a control point moves each time it is moved may also be set. The control points are initially set up in a straight line. A control point is put at the same position as the first point on the original path. Another control point is put at the same position as the last point on the original path. Further control points are placed at even intervals between. This provides a neutral starting point for the user to create the new path. The knots are set up so that the B-spline passes through the first and the last control points. The first knot (0.0) and the last knot (1.0) are repeated a number of times equal to the degree of the spline plus one. Since the number of knots must be equal to the number of control points plus the degree of the B-spline minus one, we fill in the remaining knots at even intervals between the first and last knot Tracking the Path The user may choose to track the original path and/or the new path. A block is moved along the path following the translation onto the path for each frame. We can compare the position of the path tracker to the actual position of the avatar. This gives gives a graphical display of the effect of the residuals. If the avatar is behaving unexpectedly, tracking the path may often provide insight Camera Views Although a motion may look realistic from one camera view, it may not look realistic from another camera view. Four camera views are defined and may be edited by the user. The user can easily flick between camera views. This allows for a quick assessment of the new motion after the user alters the path.

41 CHAPTER 4. IMPLEMENTATION Summary The original path is implemented by taking a separate least squares fit of the x translation vs. time data points, y translation vs. time data points, and z translation vs. time data points. The three are combined to give the original path. Residuals are applied and calculated using a moving co-ordinate system. Euler-step integration is used to calculate the arc-length at each frame. The arc-length at each frame is required for arc-length reparameterization. New paths which are shorter than the original path (when arc-length reparameterization is being used) are reset when the end of the path is reached. With new paths which are longer than the original path (when arc-length reparameterization is being used), we calculate the desired arc-length using equation 4.5. The user interface allows the user a high degree of control over the path, as well as display options. In the next chapter we will look at the results of testing the system.

42 Chapter 5 Results This section deals with experimental results. Experiments were conducted to test the effectiveness of arc-length reparameterization, end of path handling and cycling. Experiments were also done using different original path degrees on the same original motion, and observing the results in the new motion. Changes to the altitude of a path were tested. Furthermore, a simple original motion was adapted to a complex new motion. A complex original motion was adapted to a simple new motion. A complex original motion was adapted to a complex new motion. All original motion is rendered in red, and all new motion is rendered in green. Arc-length reparameterization was enabled and cycling was disabled for all the experiments after section Arc-Length Reparameterization The results achieved from applying arc-length reparameterization were as expected. The original motion in figure 5.1 was used for the experiments conducted in sections 5.1, 5.2, 5.3 and 5.8. If the new path is longer than the original path, and we are not using arc-length reparameterization, then the speed of the avatar will increase (figure 5.2). A new path which is longer that the original path (without using arc-length reparameterization) results in footskate. If the new path is shorter than the original path, and we are not using arc-length reparameterization, then the speed of the avatar will decrease (figure 5.3). If the new path is longer than the original path, and we are using arc-length reparameterization, then the speed of the new avatar will be the same as the original avatar (figure 5.4). 36

43 CHAPTER 5. RESULTS 37 Figure 5.1: Original motion and original path. Figure 5.2: New motion and new longer path without using arc-length reparameterization. This results in footskate. Also, note how the speed of the avatar has increased.

44 CHAPTER 5. RESULTS 38 Figure 5.3: New motion and new shorter path without using arc-length reparameterization. Note how the speed of the avatar has decreased. Figure 5.4: New motion and new longer path using arc-length reparameterization. Note how the speed of the avatar is the same as the original avatar in figure 5.1.

45 CHAPTER 5. RESULTS 39 Figure 5.5: New motion and new shorter path using arc-length reparameterization. The motion is reset when we reach the end of the path. 5.2 End of Path Handling The end of path handling worked as expected. When the avatar reached the end of the path before the end of the motion sequence, the system reset back to the beginning of the motion and the beginning of the path (figure 5.5). 5.3 Cycling The method implemented for cycling yielded poor results. As already discussed, the pose and the residual of the first and last frames need to be very similar for a smooth transition (from one cycle to the next) to occur. At the very least we need to incorporate some method of blending to create a smoother transition (from one cycle to the next) if a cycled motion is to look authentic. Figure 5.6 shows the result of cycling. 5.4 Original Path Degree The degree of the original path effects the tightness of the original path to the actual motion (figure 5.7). This in turns effects how closely an avatar will follow a new path. If the residuals are very small, then the avatar will follow the new path very closely. On the other hand, if the residuals are very large then the avatar will not follow the new path very closely. This will make the task of altering the path more difficult for the user. For example,

46 CHAPTER 5. RESULTS 40 Figure 5.6: New motion using arc-length reparameterization and cycling. Note how the rest of the path is filled up by repeating the motion. a very complex motion with an original path of relatively low degree will result in the new path being vastly different from the actual path followed by the avatar. Since the new path has very little correlation to the actual path followed by the avatar, it will be difficult to gauge how and where to change it to obtain the desired change in the actual path. Graphs were constructed to show how the translation residuals become smaller as the degree of the original path increases (figures 5.8, 5.9, 5.10 and 5.11). These graphs correspond to the motion and original paths in figure Converting a Simple Path to a Complex Path Figure 5.12 shows a simple motion where the avatar is walking in a fairly straight line. Figures 5.13, 5.14, 5.15 and 5.16 show some new motions using the original motion from figure The quality of the new motions were good, but not totally realistic. Footskate was noticeable around the sharper bends. Footskate was not as noticeable around less severe bends, and in some cases it was not noticeable at all. The way the avatar moves around bends looks contrived. The main cause of footskate appeared to be from the rotation of the avatar while the foot was planted as opposed to the translation of the avatar, although both were evident. Apart from these issues, the motions looked good.

47 CHAPTER 5. RESULTS 41 Figure 5.7: Original motion (red) with corresponding original path (white). The degree of the original paths are (clockwise from top left) 2, 4, 13, 45.

48 CHAPTER 5. RESULTS 42 Figure 5.8: The residuals are very large. This is not surprising considering we are only using an original path degree of 2. Figure 5.9: The residuals are smaller than the previous graph, but these results are still poor. The original path degree is still far too low.

49 CHAPTER 5. RESULTS 43 Figure 5.10: In this graph, it is starting to look like the original path is fitting a lot better to the original motion. This is expected as we are using a much larger original path degree than the previous graph. Figure 5.11: These residuals are small. The path degree is large (45). The new motion should follow the new path very closely.

50 CHAPTER 5. RESULTS 44 Figure 5.12: Original motion with original path.

51 CHAPTER 5. RESULTS 45 Figure 5.13: New motion walking in a circle

52 CHAPTER 5. RESULTS 46 Figure 5.14: New motion zig-zagging.

53 CHAPTER 5. RESULTS 47 Figure 5.15: New motion walking in a figure of eight.

54 CHAPTER 5. RESULTS 48 Figure 5.16: New motion going walkabout.

55 CHAPTER 5. RESULTS Converting a Complex Path to a Simple Path Figure 5.17 shows the results of converting a complex path to a simple path with different original path degrees. Not surprisingly motion where the original path degree is higher shows better results in terms of the avatar following the path, as well as the quality of the motion. If the avatar performs a tight turn in the original motion, and we straighten this motion out, then the section where we performed the turn in the original motion will be slower in the new motion. This is because people take smaller steps and move slower when they take a bend. We can see the effect of in figure Look closely at the new motion at the bottom left of this figure. Towards the end of the motion the avatar slows down. This section of motion corresponds directly to a very tight turn in the original motion. Figure 5.18 displays this new motion from a different viewpoint. Unsurprisingly, the new motion based on an original path degree of 45 followed the simple new path the best (figure 5.18). Footskate is more rife in new simple motions which have been converted from complex original motions, than new complex motions which have been converted from original simple motions. The main problem areas were sections where the original motion went around a bend, and the new path required him to walk straight. Footskate was the primary problem in these sections. 5.7 Converting a Complex Path to another Complex Path Figure 5.19 and 5.20 shows the results of converting a complex path to another complex path. These conversions from a complex path to another complex path posed similar problems to conversions from a complex path to a simple path. Footskate was most evident on tight turns. 5.8 Uphill and Downhill Paths We started with the original motion in figure 5.1. Figure 5.21 shows the new motion when the motion is represented relative to the tangent project on the horizontal. A side effect is that the avatar slides vertically upwards on the uphill, almost as if he were walking up inside a lift. On the downhill the avatar slides downwards. Figure 5.22 shows the new motion when the motion is represented relative to the tangent. It is clear from the figure that this motion is highly unrealistic. It looks as if the avatar should fall backwards, but he continues to move up the hill. On the downhill it looks as if he should fall flat

56 CHAPTER 5. RESULTS 50 Figure 5.17: The results of changing a complex original path to a simple new path using different original path degrees. The original complex motion used can be seen in figure 5.7. The degree of the original paths are (clockwise from top left) 2, 4, 13, 45.

57 CHAPTER 5. RESULTS 51 Figure 5.18: Another view (corresponding to the bottom left screenshot of figure 5.17) of the new motion with original path degree of 45.

58 CHAPTER 5. RESULTS 52 Figure 5.19: A new complex motion. The original complex motion can be seen in figure 5.7.

59 CHAPTER 5. RESULTS 53 Figure 5.20: Another new complex motion. The original complex motion can be seen in figure 5.7.

60 CHAPTER 5. RESULTS 54 Figure 5.21: Uphill/downhill path with motion represented relative to the tangent projected on the horizontal. on his face. 5.9 Summary This chapter has dealt with the results achieved from experiments designed to test certain aspects of the system. Arc-length reparameterization, end of path handling and cycling were all tested. Further experiments were performed on adapting the path of the avatar. The conclusions drawn from these results will be discussed in the next chapter.

61 CHAPTER 5. RESULTS 55 Figure 5.22: Uphill/downhill path with motion represented relative to the tangent.

62 Chapter 6 Conclusion 6.1 Conclusions A tool for editing the path of motion capture data was successfully built. The user interface contains numerous options to aid the user in adapting the motion to follow an arbitrary path. The method implemented is successful in getting the avatar to follow an arbitrary path. Provided the original motion corresponds tightly to the original path, the new motion follows the new path very well. The speed of the avatar in the original motion is successfully maintained in the new motion using arc-length reparameterization. Footskate was an issue primarily when sharp corners were involved or when arc-length reparameterization was disabled. Footskate was the main cause of unrealistic looking motion. Simple motions converted to complex motions look more realistic than complex motions which have been converted to follow a different path. The method used for cycling is very crude, and generally produces poor results. Gleicher s method is not realistic for paths which have a large variance in altitude, unless we are trying to simulate motion in an elevator. 56