Advanced Physics in a Film Animation Environment

Transcription

1 Advanced Physics in a Film Animation Environment Tor Sterner August 8, 2011 Master s Thesis in Computing Science, 30 credits Supervisor at CS-UmU: Niclas Börlin Examiner: Fredrik Georgsson

2 Umeå University Department of Computing Science SE UMEÅ SWEDEN

3 Abstract AgX Multiphysics is a toolkit for performing physics-based simulations, developed by Algoryx Simulations AB. AgX provides stable physics simulations and is used for industrial and engineering simulations around the world. This thesis examines the possibilities for AgX to be used in the visual effects market for films and commercials. The thesis includes a survey of opinions of professionals from the visual effects company Digital Domain. The three most wanted capabilities from AgX was according to the survey; stiff constraints, fluid-rigid body interaction and hair simulation. Furthermore, all new tools must improve or be equal on all aspects compared to the old tools, and have good scalability. Several simulations were run to compare the stiff constraints and scalability of AgX with the currently used tools. The results show that AgX handles stiff constraints a lot better than the old tool and is also highly scalable. In summary, for saving time from doing tedius tasks and for using physics to an extent earlier not possible, AgX would be a good addition to the current use of physics in visual effects.

4 ii

5 Contents 1 Introduction Background Aim Related Work Organization of thesis Theory Computer Animation Early animation techniques Walt Disney s Principles of Animation Animation Techniques Computer Physics Animation versus Physics Physics in Houdini DROP FSim AgX Survey Wanted capabilities Stiff constraints Fluid-rigid body interaction Cloth and hair Rigid body vector fields Fluid tool simplicity Future features Simulation tool properties Scalability iii

6 iv CONTENTS Determinism Data Features over simplicity No step back Development scenarios Replace the DROP physics engine with AgX Build a completely new rigid body system tool with AgX, to replace DROP Add AgX constraint fluids to DROP Build a AgX constraint fluid tool outside DROP Add AgX wire functionality to DROP Use cloth built in AgX Use hair built in AgX Add physics to modelling environments Results of the survey Ranking Experiments Simulation scenarios Brick wall Convex objects Results Brick wall Convex objects Discussion Simulations discussion Conclusions Limitations Future work Summary Acknowledgements 43 References 45

7 List of Figures 2.1 Clay stop motion in form of Wallace in Wallace and Gromit Animation of a horse in gallop Soft ball landing with squash and stretch properties Blue screen used with Chroma Key technique Motion capture transfered to animated character Simulation of ball Node structure in a Houdini network A small system built with Houdini s own tools Node network for a DROP setup AgX fluid interacting with rigid boxes Several rigid objects constrained together A possible fluid setup with several node systems Constrained objects for hair DROP brick wall simulation scene DROP convex simulation scene Simulation of a lock constrained wall getting hit by a light ball. 34 (a) AgX brick wall frame (b) Original DROP brick wall frame (c) AgX brick wall frame (d) Original DROP brick wall frame (e) AgX brick wall frame (f) Original DROP brick wall frame (g) AgX brick wall frame (h) Original DROP brick wall frame Simulation of a lock constrained wall getting hit by a light ball. 35 v

8 vi LIST OF FIGURES (a) AgX brick wall frame (b) Original DROP brick wall frame (c) AgX brick wall frame (d) Original DROP brick wall frame (e) AgX brick wall frame (f) Original DROP brick wall frame (g) AgX brick wall frame (h) Original DROP brick wall frame Simulation time graph, no lock constraints Simulation time graph, lock constraints

9 Chapter 1 Introduction To animate means to give life to an inanimate object (Kerlow, 2009). In animation, movement and actions of objects are illusions, achieved by displaying many images rapidly in sequence. The craft of animation began over a century ago and grew during the first half of the twentieth century with the popularity of hand-drawn cartoons. In the later part of the twentieth century, computers were introduced to the animation art, and have since then formed several new techniques of animation. The need for physical realism in animation greatly increased in the later part of the twentieth century with the use in live action films. When animation is used in visual effects for live action films the visual effects has to visually match realistic motion. This, and the ever-increasing use of computers led to the start of the field physics-based animation. 1.1 Background Algoryx Simulation 1 is a company in Umeå started 2007 that focuses on delivering stable and robust simulation solution for the professional simulator and engineering market. The main product of Algoryx is the AgX MultiPhysics Toolkit, which is a physics engine that comes with functionality suitable for developing training simulators in heavy vehicles, marine anchor handling as well as engineering simulations. Two of the main properties of AgX are high precision and scalability (Algoryx, 2011). Furthermore, AgX supports unified physics, i.e. combination of different forms of physical behavior for example wires and rigid bodies. One of the business areas of Algoryx Simulation is the visual effects market. The current use of physics in visual effects could then be improved with the use of AgX

10 2 Chapter 1. Introduction Houdini is an authoring environment for 3D animation and visual effects developed by Side Effects Software Inc. 2. It covers all the major areas of 3D production including modeling, animation and rendering. Houdini also supports several modules including dynamical physical simulation of rigid bodies, wires, fluids and cloth. 1.2 Aim The aim of this thesis is to examine the current role of physics in animation and suggest how AgX can be used for further improvement. Furthermore, a prototype integration of AgX in Houdini will be performed. Part of this thesis is an initial analysis of the current use of advanced physics in film animation. The results will be used to select scenarios interesting for visual effects professionals. Some examples of scenarios and features where AgX Multiphysics is believed to have potential are: unified physics, i.e. several different physical areas are simulated and solved simultaneously, e.g. wires and rigid bodies (Servin et al., 2011), incompressible fluids, and mechanical systems, e.g. stiff and jointed mechanical systems, such as trees, robots and biomechanics. The scenarios will be graded based on a survey of film animators and visual effects artists. At least one scenario should be implemented and evaluated as part of the integration of AgX MultiPhysics in Houdini. The integration of AgX MultiPhysics will be directed towards 3D artists and animators rather than programmers or physicists. This means that it will focus more on increasing artistic expression rather than high precision. A more general aim is to expand and simplify the use of physics in film animation. 1.3 Related Work Physics in visual effects has become an important part in creating movies (Erleben, 2005). Water, hair, smoke, explosions, cloth, and falling objects are all examples of simulated physics common in contemporary film production. At the same time, visual effects in films and games have been a motivation for the development of physics in computer animation (Erleben, 2005). The big issue with physics-based simulation for visual effects in films is the conflicting attributes of physical realism and artist control (Pentland and Williams, 1989; Liu, 1996; Bridson et al., 2007; Barbič et al., 2009). Physical realism is required from visual effects because our eyes and brain is sensitive to the behavior of certain objects, e.g. shape and motion of fabric (Goldenthal et al., 2007). Artist 2

11 1.4. Organization of thesis 3 control may be seen as the animators way to decide the outcome of a simulation. For example to decide where a thrown dice should land and which side should come up even though the motion of the object is physically simulated. The conflict is further complicated by the wish to exaggerate certain events, the so called The Hollywood Effect (von Borries et al., 2007). The conflicting attributes of realism and control is the reason for several improvements of physics simulation methods. The improvements are directed towards giving animators better control of a physics-based simulation. An example is the use of space-time constraints (Ngo and Marks, 1993). With certain techniques animators are able to use regular animation methods along with rigid and deformable body physics (Popović et al., 2000, 2003; Barbič et al., 2009), as well as fluid simulation as in Treuille et al. (2003). 1.4 Organization of thesis Chapter 2 covers basic theoretical parts of animation and physics as well as more specific information about the software to be used. Chapter 3 contains the wanted capabilities and properties of the visual effects professionals participating in the survey. Furthermore, it includes the participants opinions of several development scenarios and a ranking among the scenarios. Chapter 4 describes some experiments in form of simulation scenarios and results of the simulations are then presented in chapter 5. Chapter 6 concludes by discussing the results, limitations, future work, and giving a summary of the thesis.

12 4 Chapter 1. Introduction

13 Chapter 2 Theory This chapter explains the basics of animation and computer physics and compares the different requirements of animation and physics. Furthermore, information about the software used in the thesis is presented. 2.1 Computer Animation In animation, pictures rapidly shown in sequence are perceived as a motion by the viewer. Pictures, or frames, are the most basic entities of an animation (Kerlow, 2009) Early animation techniques There are plenty of techniques for making animated frame sequences, e.g. stop motion and hand-drawn animation. In stop motion, each frame consists of still objects or characters moved by hand by an animator. In a frame sequence it gives the illusion of movement. Several stop-motion sequences uses clay as its objects, as in Fig 2.1. Hand-drawn animation requires the animator to draw moving objects each frame. An example of hand-drawn animation is shown in Fig Both stop motion and hand-drawn animation is very time-consuming because of the many frames the animator had to produce. The number of frames per second needed have also been increased since the start of animation history, from 12 to (Erleben, 2005) Walt Disney s Principles of Animation The art of animation was pioneered by the Walt Disney Studios, during the first half of the twentieth century. Part of their legacy is their self-developed Principles of Animation (Thomas and Johnston, 1981). There are twelve principles, of which three are directed towards adding physicality to the animation; squash 5

14 6 Chapter 2. Theory Figure 2.1: Clay stop-motion in form of Wallace in Wallace and Gromit. Used with permission. Figure 2.2: Animation of a horse in galopp. (Animation by Jan-Eric Nyström, Helsinki, Finland)

15 2.1. Computer Animation 7 Figure 2.3: Soft ball landing with squash and stretch properties. and stretch, follow-through and overlapping action, and arc motion (Kerlow, 2009). The main point of squash and stretch is to maintain the volume of an object even if it is deformed. This means that a soft ball that is dropped into the ground and thereby squashed should also be stretched to preserve the same volume during the whole motion as seen in Fig In the follow-through and overlapping action principle, the follow-through part adds physicality to the animation. It discourages animators from making motions and actions halt instantaneously. Instead objects decelerate, as in the real world. The arc motion principle intends to give animation of characters a natural and physically real look. In nature, most motions are in curved paths, rarely in straight lines. The rest of the principles goes through other animation guidelines, e.g. how to use timing and anticipation to enhance certain motions and exaggeration to give the animation a cartoony feel. The principles were used by Walt Disney Studios to decrease the amount of training that new animators needed to go through. Additionally, the principles were used as guides in production of animated sequences.

16 8 Chapter 2. Theory Animation Techniques One of the most fundamental techniques in animation is keyframe animation. In keyframe animation the animator sets up two or more key frames, where the characters or objects are in some important states, e.g. the beginning and end of a motion. An example of this is the throwing of a ball, where in one keyframe the ball is in the hand of the thrower and in the next keyframe it has hit a wall. The keyframes are linked together with in-betweening i.e. creation of images between keyframes, to complete a full motion between the states. In the example of the thrown ball, the animator would create all images of the ball sailing in an arc through the air. The introduction of computers increased the production speed and the number of produced sequences. In the beginning, computers were used to set together the hand-drawn images and to add colors. As the computers and software got better, more and more of the animation production was done with them. Computers are able to do the in-betweening in animation, using interpolation i.e. a way of constructing new points between already given points. In the example of the thrown ball, the computer would calculate the path the ball should take in the air and draw in-betweening frames. Three important techniques in computer animation are green screen, motion capture and physics-based simulation. The green screen technique, with the technical term Chroma key compositing, is used in many contemporary film productions. The technique is used to merge two pictures. It uses a clear colored background in one picture and later makes the color transparent. The partially transparent picture is added to a background picture to form the final picture. The final picture gives the impression that the objects in the foreground is in the environment of the background, the use of chroma key can be seen in Fig 2.4. Motion capture is a way to increase the realism of animated characters. Movements of real actors are recorded by sensors and computers(kerlow, 2009), as in Fig 2.5. The recorded movements are applied to the animated character by the animation software. This gives the animated character the realistic movements of a real actor but can still be in the form and environment of an animation. 2.2 Computer Physics Physical simulations has been used for some decades as a way to understand physical occurrences in nature and to predict physical reactions, e.g. weather forecast. Simulations can be done by stepping through mathematical formulas for several steps and watch how data change. Each such step is called a timestep and represents a short period of time in the simulation, often as a part of a second. An example of a moving ball simulated in two timesteps is shown in Fig 2.6.

17 2.2. Computer Physics 9 Figure 2.4: Blue screen used with Chroma Key technique. Used with permission. Figure 2.5: Motion capture transfered to animated character. Used with permission.

18 10 Chapter 2. Theory Figure 2.6: Ball simulated in two timesteps, velocity is calculated from previous velocity and forces. Position is calculated from previous position and velocity. A common simulation rate is 100Hz, i.e. 100 timesteps per second. However, unless the simulation needs to be in real-time, each timestep can take an arbitrary time. The determinism and theoretical ground makes simulations great to run on computers. Which is why physics-based simulations are almost exclusively run on computers. For a real-time application the time to calculate one second of simulation should be less than one second. Physics-based simulation on computers has granted a way for very large complex systems to be simulated with little effort. This has led to specialized software directed towards performing physics-based simulation, so called physics engines. A physics engine can be used as means to either perform a precise mathematical simulation or a good enough approximation of a physical occurrence. Several physics engines require a high simulation rate, i.e. a small timestep, to get good results. The choice between precise or good enough depends on what the simulations will be used for. Two different situations in which a different choice is prefered are industrial simulations, e.g. the simulation of a robotic arm on a conveyour belt, and physics in games, where it s more important that the simulation is fast and produce a nice looking result. One field that uses physics-based simulation is computer animation. The use of physics-based simulation for animation is called physics-based animation and is the field around which this thesis will revolve. Physics-based animation can be divided in several areas of animation dynamics (Coutinho, 2001; Erleben, 2005). In this thesis, the physics-based animation field will be divided in two main areas, solid dynamics and particle dynamics, i.e. the motion of solids and the motion of particles. Solid dynamics simulations uses objects with several attributes, e.g. mass, volume, velocity and rotation. Solid dynamics are further divided into two main categories; rigid body dynamics i.e. dynamics of hard objects that do not change shape, and deformable body dynamics i.e. soft bodies whose shape can vary. Rigid body dynamics is very common in physics-based animation. Falling objects, shattering bricks, mechanical systems and character skeletons are all common examples of rigid body animation. Deformable body dynamics is also used in animation, e.g. for simulating skin and hair. Deformable body dynamics does have some methods for simulating fluids, although particle dynamics is the more common way.

19 2.3. Animation versus Physics 11 Particle dynamics consists of points, without volume and rotational attributes, and can be used to simulate a great variety of effects. Most methods for fluid simulations are based on particles, and fluids can by itself simulate things commonly not recognized as fluids e.g. smoke or fire. Particles is also used in the simulation of cloth as well as more obvious effects like snow. A regular physics engine takes input in forms of objects or particles with attributes and can then simulate how the input will behave over time. A regular physics engine does not support stepping backwards in time or control the future outcome of a simulation. New input can however be given while the simulation is running between steps. The progress of physics-based animation has been immense. About two decades ago a simulation with just a few objects with links between them was hard and took a long time per frame. Since then, several methods and algorithms that speeds this process up has been developed. Furthermore, several methods that simulates new physical occurrences have been created and used. This progress is greatly amplified by the progress of computers in general. With faster computers, simulations can be made better, e.g. by running with shorter timesteps, and to a larger scale than before. 2.3 Animation versus Physics Animation and physics have different advantages. Animation allows for great artistic expression where as physics simulations makes it easy and fast to produce large and realistic sequences. The conflict between artistic expression and realism in animation physics lies in the level of control in physics-based simulations (Pentland and Williams, 1989; Liu, 1996; Bridson et al., 2007; Barbič et al., 2009). Simulations simplifies producing physical occurences. However, most simulations has no way to decide the behaviour and outcome of the simulation except for setting initial conditions and giving input during the simulation. This makes it hard for physics-based simulations to follow the wishes of a director or an artist without tedious tweaking of initial condition parameters. For visual effects in many contemporary movies, control over animation is more important than simplicity. However, controlling a large system animation in great detail is a slow and expensive process. Furthermore, the human eyes and brain is sensitive to the realism of certain objects behavior (Goldenthal et al., 2007), which makes it hard to animate without help from physics-based simulations. Objects like fluids, building debris and cloth often require physics-based simulation to acquire the physical properties for a realistic look. 2.4 Physics in Houdini The current use of physics in Houdini ranges from simulations of small constrained systems to large-scale fluid simulations and rigid body systems with

20 12 Chapter 2. Theory Figure 2.7: Node structure in a Houdini network. tens of thousands objects. Small systems of rigid bodies, hair, fluids, and particle systems can be simulated with built-in features included in the distribution of Houdini. Additionally, Houdini allows users to implement custom made solutions that can be used inside Houdini with all its features. Houdini s own built-in tools give artists an easy to use physics-based simulation that is easy to setup, is intuitive and have good visual feedback. A small system with one constraint is shown in Fig 2.8. Houdini uses several node structures, the most releveant is a SOP network (SideFX, 2011), seen in Fig 2.7. The structure is intuitive and allows for good customization and test of data. However Houdini s tools are in most cases too slow for large systems, and allow for too little customization. Instead, several externally designed tools have been constructed for large systems. These tools are often significantly faster than Houdini s own built-in solutions. What these externally made solutions often lack is simplicity, more time is spent optimizing and implementing additional features DROP One externally designed tool constructed for simulating large rigid body systems is DROP. Created by visual effects company Digital Domain 1, DROP allow physics-based simulation of rigid body systems using a regular physics engine. DROP is made for simulating large numbers of objects with good control while also allowing for high detailed objects. 1

21 2.4. Physics in Houdini 13 Figure 2.8: A small system built with Houdini s own tools. A ball is hinged to a bar and will move like a pendulum. One example of what makes DROP able to simulate a large number of objects is that it optimizes the amount of data that needs to be sent between nodes in the network. DROP separates the visual representation of an object and its physical properties and shape. While sending data used for simulating the object inside the DROP structure, DROP do not need the visual representation. The visual representation of an object is added after the simulation is complete at the position simulated. A DROP setup is shown in Fig FSim As with the rigid body simulations it is common for visual effects companies to use other implementations than Houdini s own for fluids. Houdini s own fluids can simulate small systems with Smoothed-particle hydrodynamics (Erleben, 2005) and Fluid implicit particle (Bridson, 2008) fluids. Those simulations are good for small and simple volumes of water and can be used with simple control from the Houdini environment. However, for large bodies of water other solutions are often used for visual effects. As with rigid bodies, visual effects company Digital Domain, uses an own implementation for large bodies of water. The implementation uses a particle level set method described in Enright et al. (2002) for simulating fluids. The level set method is however not enough for most fluid simulation as it do not work well with small amounts of fluids or splashes. In this case a combination of particle level set fluids and SPH fluids is used. The main body of fluid is simulated using the particle level set method until it is satisfactory. Then the SPH fluids is added given velocities from the level set fluids where splashes should be.

22 14 Chapter 2. Theory Figure 2.9: Node network for a DROP setup.

23 2.5. AgX 15 Figure 2.10: Fluid interacting with rigid boxes stacked on top of each other, simulated by AgX. A mass of fluid flows on the stack and knocks them down. The implementation made by Digital Domain is used as one single node in Houdini. The node loads several files describing the whole scene with fluid and also files describing other areas affecting the fluid, e.g. certain areas where the fluid should flow faster or slower than regular. In addition, the fluid node also specifies paths to export the whole simulation to external files during the simulation. The fluid simulation is often saved each frame to be able to restore a faulty simulation and to view each simulated frame separately. The fluid node can only input rigid bodies in the form of a static environment. 2.5 AgX The AgX MultiPhysics Toolkit, a physics engine developed by Algoryx Simulations (Algoryx, 2011), contains several features relevant for visual effects simulations, e.g. unified physics, incompressible fluids, and stiff and jointed mechanical systems. Unified physics, i.e. the unification of different forms of dynamic systems, e.g. fluids and rigid bodies, chould be of benifit for visual effects simulations. Interaction between rigid bodies and water will always be present in movies in both small scale (something is dropped into water) and large scale (a huge wave crushes everything in its path). Today such simulations are often simulated separately from each other, and the effects of interaction added manually from an animator. AgX has support for full interaction between rigid bodies and its incompressible fluid, seen in Fig 2.10.

24 16 Chapter 2. Theory Figure 2.11: Several rigid objects constrained together to form a beam. The red boxes are constrained together and forms a beam resting on two static platforms. The incompressible fluid in AgX, called Constraint fluid, is based on a regular SPH model with added constraints for density and other stability improving measures (Bodin et al., 2011). This makes it near incompressible and able to solve together with other dynamic systems, e.g. rigid body systems. The near incompressibility of the fluid adds further realism to the simulation as a real fluid is practically incompressible. Another example of the unified physics in AgX is the integration of wires and rigid bodies, i.e. it can simulate interaction between objects and wires (Servin et al., 2011). AgX is able to simulate completely stiff and jointed mechanical systems (Lacoursière, 2007). Stiffness of constraints in physics-based simulations is how hard it is to break the properties of a certain constraint, e.g. if the constraints are completely stiff several objects constrained together would act as a single rigid body, e.g. a building constructed by several different parts. One of the advantages of using several constrained objects as one instead of a single object is that a user can extract the forces applied to each constraint, and also remove constraints when needed, e.g. a building that collapses after collision with another object. The stiffness of a constrained object can also be set weaker, e.g. to represent different kinds of materials and object softness, an example of a constrained beam with a little weaker stiffness is seen in Fig A weaker stiffness could be used to construct softer objects, e.g. rubber-like objects or swaying trees. In constrast to the Houdini physics, an AgX simulation are set up in text files. The files describe the physical objects and environment to be simulated. Furthermore, specific functionality can be described in form of callback functions. AgX uses the same simulation setup pipeline as Colosseum3D (Backman, 2005). For representation of the environment and objects the files can be in form of code (C++ and Lua) or a simple in-house data format. Callback func-

25 2.5. AgX 17 tions need to be in the form of code.

26 18 Chapter 2. Theory

27 Chapter 3 Survey A survey was performed with visual effects professionals at Digital Domain during a three week visit to Los Angeles, California, in May Digital Domain has produced visual effects for big movies like Titanic, Transformers: Dark of the Moon, and Tron: Legacy. The survey intended to examine how AgX could be integrated in the visual effects production to improve the current use of physics in visual effects. Four interviews were performed with people from different areas of visual effects production; A creative director of software, a software developer, a computer graphics supervisor and a commercials visual effects supervisor. Furthermore, several meetings were held to discuss the production pipeline for physics in visual effects and general physics-based simulations. Before and during meetings and interviews, the participants were shown several videos of AgX MultiPhysics to be able to compare and use as a reference during discussions. Simulations made by Digital Domain with the current use of physics were also used as reference. Furthermore, after interviews and meetings were held and evaluated, some possible development scenarios were produced. The development scenarios were later discussed with the participants. This chapter will describe discussion topics about the use of physics simulations in visual effects. Several physics capabilities requested by the participants of the survey are discussed, as well as some properties they believe are important for all simulation tools. Furthermore, the chapter contains a section describing discussions about several development scenarios and the resulting ranking among them. 3.1 Wanted capabilities Several features of the current physics simulations in visual effects and also features that the participants thought interesting were discussed. The most 19

28 20 Chapter 3. Survey prominent are described in following sections Stiff constraints The physics engine used in DROP cannot handle stiffly constrained objects, only really small systems (10-15 bodies) are able to act as one body. One example is a collapsing building, where the building should act as one single body during several frames and then collapse from the impact of another object. Currently this sequence could not be simulated with the same input during the entire scene. Instead, the initial building has to be simulated separately from the collapsing building. All participants saw big benefits from having the stiffness of constraints that exists in AgX. The example building can then be initially constrained with some of the constraints vanishing when the impact takes place Fluid-rigid body interaction Another major point was fluid rigid body interaction, the current physics simulation method in Fsim has no support for interaction between fluids and dynamic rigid bodies, i.e. simulated rigid bodies. Instead a customized process for handling such scenes have been developed. The process alternates between running the fluid simulation and the rigid body simulation. The fluids uses the rigid bodies as static bodies in its simulation, and the rigid body simulation extracts forces from the fluids simulation and adds them to the dynamic bodies. The alternating process is slow and inefficient. However, one benefit is that some artistic control remains in the rigid body simulation. In sequences where the interaction is in focus and the effects on both fluid and objects must be clear, the participants saw the use of AgX fluid as a great solution. The example scenario discussed was where a helicopter crashes into an ocean. The scenario requires great detail on the interaction between fluid and object, which was not satisfactory in their current physics methods Cloth and hair Cloth was another interesting system discussed during both interviews and meetings. While AgX did not have an implementation of it at the time, some properties of the physics engine indicates that it would yield good results, e.g. the stiffness of constraints. The human eye and brain is very sensitive for all defects with simulated cloth (Goldenthal et al., 2007), and the main problem is to make it as flexible as real fabric while keeping it inextensible, e.g. it does not stretch from its own weight. Similar to the discussions about cloth were the discussions about hair. Hair is one of the most complicated areas of visual effects in contemporary film production. No solution has solved all issues hair brings, e.g. hair collision,

29 3.2. Simulation tool properties 21 huge numbers of hairs and natural look. As in the case with cloth, AgX has properties the participants found interesting during the discussions Rigid body vector fields Some participants expressed the opinion that DROP lacked a good way to add and configure vector fields, to influence the bodies. An example sequence is the effect of a shockwave, which could send all objects in a certain direction. The requested effects is available for fluids in Fsim Fluid tool simplicity The participants discussed the simplicity of Fsim for using their current fluid simulation. While their tool was highly configurable, it was not visually represented in an intuitive way inside Houdini and it was hard to test different parts of the simulation without running the entire simulation Future features Two additional features was briefly discussed during the conversations; aerodynamics and large scale fire/smoke. Aerodynamics would allow rigid objects to interact more with the air surrounding it, e.g. creating a gust of wind or flying a helicopter. The large scale fire/smoke feature was described with the comment We want to set a city on fire. 3.2 Simulation tool properties During discussions and interviews several general points came up. These did not directly specify certain capabilities, instead it was general things to think about concerning the current and future use of physics. Many participants in the discussions consider these properties more important than new features Scalability In visual effects, everything is about the scalability of the simulated systems. Everything new has to be bigger than the old 1. Several parts of this concept is important to visual effects artists and other professionals. For rigid body systems there are two major aspects that are important to rate in scalability; total number of bodies and total number of constraints. Rigid body simulations used in contemporary film production often simulates large breaking objects with lots of debris and the general opinion of the visual effects professionals is that more is better. Today the visual effect of a building collapsing in several pieces is often simulated in several steps, with large pieces simulated 1 Or as one of the participants put it: We want to blow up bigger things in greater detail

30 22 Chapter 3. Survey first and small debris added from them at a later stage. This makes large pieces unaffected by collisions with small debris. While this is not a major issue, a lot of time could be saved simulating everything at the same time. For constraints the same reasons applies, with many constraints in a simulation the amount of rigid bodies have to be decreased with a factor of ten. This makes it hard to simulate large constrained systems, e.g. twenty storey buildings collapsing. Scalability is also important in fluid simulations, not only do visual effects professionals want to increase the size of fluids simulated, they also want to increase the resolution of the fluid. The size in this case can be thought of as the amount of fluid simulated, e.g. a glass of water compared to a tsunami, and the resolution is the detail of the simulated fluid. In the future the fluids simulations should be able to simulate a large fluid system, for example a part of an ocean while still splash with small drops from the impact of a tennis ball. A potential solution other than simply increasing the amount of data for larger systems exists, such as a varying resolution. In the ocean example the simulation would have a low resolution everywhere except around where the ball strikes Determinism Determinism is important for all types of simulations in visual effects. Determinism in physics-based simulations imply that the simulation will behave exactly the same given the same input. For visual effects it is desired that a change of a small independent part of a simulation does not influence the other parts. For example, if a director has approved a certain simulation with the exception of a small part that needs to be changed, a new run of the simulation with the changes applied should not alter the whole simulation Data Two aspects of data handling from physics-based simulations is interesting according to the participants in the survey. First of all, it is very important that all information about a simulation is possible to save and load from an external file. This is useful when simulating large systems which need to be paused and resumed, or when errors occur, e.g. if a simulation crashes after four hours the user must be able to resume from precisely the moment before the crash. Secondly, as much data as possible should be possible to extract from the simulation about everything inside it. When visual effects artists runs physics simulations they want as much information as possible. One example is if an artist wants to produce a sequence of a collapsing building. In this case it is useful to know where the largest forces appears on the building to know where it should break first. This information must be possible to extract from the simulation. After the simulation, smoke and small debris could appear where the building has shattered and should have a similar velocity and direction as the part it is appearing from. This information must also be possible to extract.

31 3.3. Development scenarios 23 All types of stress data that the user can extract from the simulation can help the artist to create an as realistic and nice looking simulation as possible Features over simplicity One key opinion among all participants in the survey is that features are always more important than simplicity. The visual effects in contemporary film production is always looking for the next step to take in new features and in sheer size and detail. To be able to produce something that no other company can do is highly regarded in the business, and they allow for advanced tools to make it so. Furthermore, configurability is another very important aspect of all tools used in film production according to the participants. The artist want to be able to tweak all parameters in the simulation and add limitations and enhancers to make the simulation behave exactly the way the director wants. The trade-off between features and simplicity is slightly different for visual effects production for commercials. The commercial clip visual effects production time is much shorter than the production time for films, which makes simplicity and intuitivity important. While configurability is still the most important property, it is also important that a simple scene is easy to set up. When producing a sequence for three weeks it can not take one week of learning the tool and setting up the first version of the simulation. This makes having good default configurations or templates, e.g. a template for using a water fluid with viscosity and density parameters already set, a great benefit No step back The final and most important point made by the participants in the survey is that new tools for physics simulations can only be improvements, nothing can be allowed to become worse than the previously used tools. A new tool can not, even if it is an improvement in some aspect or adds some features, be slower or unable to handle something previously possible. 3.3 Development scenarios The development scenarios were constructed after discussions and interviews with visual effects professionals, and later reviewed with the same participants. Listed are some scenarios, including the most interesting and discussed ones. Replace the DROP physics engine with AgX. Build a completely new rigid body system tool with AgX, to replace DROP. Add AgX constraint fluids to DROP. Build a AgX constraint fluid tool outside DROP.

32 24 Chapter 3. Survey Add AgX wire functionality to DROP. Use cloth built in AgX. Use hair built in AgX. Add physics to modelling environments Replace the DROP physics engine with AgX This scenario would require a implementation of DROP with AgX as physics engine. The implementation would keep the interface for DROP and all its features, while taking advantage of AgX s stiff constraints and other features. The advantages of doing the AgX implementation this way is that no new work procedures need to be taught. Furthermore, it would be a fast and simple implementation to do, and would utilizes several features of DROP not directly related to physics engine. However, DROP is a pretty specialized implementation and could restrict an intuitive use of AgX. All participants viewed this development scenario as the first step to a full implementation of AgX. The fact that this scenario should be reasonably fast and simple to do means that testing and further development of AgX in DROP could begin sooner than in other scenarios. However, several participants expressed the fear of being stuck in this implementation. Which is why this kind of implementation should be regarded as a first step, with either additions in form of features or a completely new implementation as the next Build a completely new rigid body system tool with AgX, to replace DROP As implied in previous section, replacing DROP is seen as a harder and slower process than integrating AgX in DROP. However, several advantages to building a new rigid body system interface with AgX exists, e.g. the implementation would allow for intuitive use of new AgX features such as wires and fluids. Some participants expressed their wish to have a new rigid body system interface replacing DROP. This would allow for a implementation that could be maintained and upgraded more easily. However, this was seen as a step after changing DROP to use AgX as physics engine Add AgX constraint fluids to DROP This implementation would require AgX integrated in DROP as explained above. The addition of the constraint fluid of AgX would allow for full integration of rigid bodies and fluids, with the features of DROP and recognition from the users. The big advantage of doing this implementation would be the simplicity of using ridig bodies and fluids together. A disadvantage is that

33 3.3. Development scenarios 25 Figure 3.1: A possible fluid setup. All three inputs are small networks of nodes. DROP never was intended for fluid functionality and thus do not have an easy and intuitive way of setting up fluid simulations. If an implementation with AgX in DROP were performed, the addition of AgX constraint fluid seemed natural according to the participants. The constraint fluid was also the most desired addition to the current physics effects, with the integration between rigid bodies and fluids in focus. However, the participants discussed the scenario in regards to how the fluid could be integrated. Two options were debated; that the fluid integration should be used as their current fluid, or if the fluids should be used in a more intuitive way in the DROP structure. Their current fluid interface consists of one node with all functionality in one place, which sometimes can be difficult and unintuitive. A better solution is to implement the constraint fluid interface as several nodes that fits together with a rigid body system. A possible interface can be seen in Fig Build a AgX constraint fluid tool outside DROP If no integration of AgX in DROP is made, the constraint fluid could be implemented by itself or in the new rigid body system setup made for AgX. If the constraint fluids was implemented by itself, it would lack the integration with rigid bodies, with the exception of static bodies, and could be used in the same manner as FSim. An implementation of the constraint fluid in a new rigid body system interface made for AgX would not have any limitations in structure or features, but would require much work. The discussion about a fluid implementation in a new rigid body system interface covered the same aspects as with an implementation inside DROP, how the integration should be done. The same options as with the DROP implementation were discussed and the same conclusions were drawn. An implementation of AgX constraint fluid outside any rigid body system interface would have the same features as the current fluid interface has, and should therefore work the same way to make the transition for current users easy.

34 26 Chapter 3. Survey Figure 3.2: Several constrained objects in a line or following a curve can be hair Add AgX wire functionality to DROP This implementation would allow AgX wire objects to be integrated with regular rigid body physics. It would allow scenes were wires affected objects in a way current methods for wires or ropes will not. The participants of the survey saw no real use of this feature in current visual effects. Wires and ropes simulated in visual effects are mostly not load bearing Use cloth built in AgX An implementation of cloth in AgX would be able to interact with rigid bodies and other AgX features, e.g. if fluids was implemented. A cloth interface could be implemented in DROP which would be the fastest and most effective way of implementing AgX cloth that would integrate with rigid bodies. If an implementation of AgX cloth should be implemented, the participants agreed that it should be unified with a rigid body interface. The participants emphasized that if such implementation should be done, it is important that the implementation also supports fluids. If AgX cloth could interact with fluids it would be able to simulate sequences other cloth simulations can not, e.g. a piece of cloth getting wet after being dragged in water. A fluid-cloth interaction is required for AgX cloth to be interesting Use hair built in AgX The specialized request of creating hair in AgX came from one of the participants and is seen as one possible development. The hair would in this case consist of several constrained objects in a line, see Fig 3.2 With an AgX implementation in DROP, a hair feature would not require a lot of effort. The advantages is clear, hair that can collide with other objects as well as itself. To use hair built in AgX in visual effects is one of the most wanted developments. Having AgX s collision stability and stiff constraints in hair simulation would be of great benefit in visual effects.

35 3.4. Results of the survey Add physics to modelling environments This development scenario is different from the rest. It would not run any of the simulations shown in films or clips. Instead, it would be used when modelling and setting up an environment for any scene in visual effects, made in e.g. Houdini or Maya. With this implementation, an artist setting up an environment of objects could run a short simulation of the current scene and objects would fall in place by gravity. This would result in all objects lying in physical correct places with no gaps or intersections. This kind of feature would have to be really fast and simple to be of help for artists creating environments in visual effects. The participants find this idea really interesting as it would simplify the process of setting up environments particularly for physics simulations. It would however also need to be configurable, e.g. allowing some objects not to be affected by gravity. 3.4 Results of the survey From the survey an internal ranking among the development scenarios was created. The participant s opinion about wanted features has influenced the ranking the most, but also their opinion about AgX features. One or more development scenarios will be chosen for prototyping using this ranking Ranking 1. Replace the DROP physics engine with AgX, i.e. get AgX functionality e.g. stiff constraints. 2. Add AgX constraint fluid to DROP. 3. Use hair built in AgX. 4. Build a completely new rigid body system tool with AgX, to replace DROP. 5. Build a AgX constraint fluid tool outside DROP. 6. Use cloth built in AgX. 7. Add physics to modelling environments, e.g. Houdini or Maya. 8. Add AgX wire functionality to DROP.

36 28 Chapter 3. Survey

37 Chapter 4 Experiments A prototype implementation of the highest ranked development scenario was performed during the thesis. The highest ranked development scenario was Replace the DROP physics engine with AgX. In the prototype, the physics engine that did the actual simulation in DROP was replaced with AgX. The DROP node structure was retained. However, the prototype does not support every DROP configuration and setting. Having AgX as the physics engine in DROP was seen as a base implementation which could then be extended with more AgX specific capabilities. 4.1 Simulation scenarios The prototype allows for some basic property testing, mainly the constraint stiffness and the general speed of the simulation. Simulation scenarios were constructed to evaluate these properties. Two different scenarios was set up to test the stiffness and speed; simulating a brick wall constructed from constrained objects and simulating large amounts of convex objects. Convex objects are the basis for all rigid body simulations at Digital Domain. Two different solvers are used in AgX; a direct solver and an iterative solver. A solver is the part of a multibody physics engine that actually calculates the behavior in the simulation. The direct solver is able to simulate infinitely stiff constraints and contacts while the iterative solver is faster in systems not requiring completely stiff constraints. The simulations were set up as a node network with DROP nodes as described in Chapter Furthermore, several parameters and attributes were set in some of the nodes to make the simulation behave as intended, e.g. objects weights and solver timestep. 29

38 30 Chapter 4. Experiments Figure 4.1: DROP brick wall scene, it contains 80 rigid body objects in form of bricks in a wall and a sphere with an initial velocity towards the wall Brick wall To evaluate the stiffness of constraints a brick wall simulation was set up. The brick wall simulation consists of 80 bricks constrained as a wall standing on the ground. Each brick is 0.1m in height, 0.2m in width and 0.5 in length and weighs 2kg. The wall is set together with 640 constraints to create a solid wall that should not break from impact. A constraint between two objects is meant to keep the objects in exactly the same position and rotation in relation to each other, this is called a lock constraint. A ball with radius 0.3m will be thrown towards the wall. The ball weighs 2kg and will be sent orthogonal towards the wall at a initial velocity of 10m/s. The whole scene can be seen in Fig 4.1. The expected realistic behavior of the setup is that the ball should bounce of the wall with little or no effect on the wall since it is very light in comparison to the wall. The wall should behave as a single flat object Convex objects To compare the simulation speed of AgX and DROP two scenarios with a large number of objects was simulated. The only difference between the two scenarios is that lock constraints were added in the second between close objects. The scenario is built with octahedrons. Each octahedron has an edge length of 0.3m and weighs 13.5kg. The objects are organized in small clumps of

39 4.1. Simulation scenarios 31 Figure 4.2: DROP convex scene, it contains rigid body objects in form of octahedrons (8-sided dice) falling on a flat surface. objects in eight large sphere shapes. In the constraints scenario each small clump will be constrained together to act as one body. The total number of lock constraints was All objects are organized over a large flat surface, like a tabletop. The entire simulation setup can be seen in Fig 4.2 The expected behavior of the first scenario is that all objects should fall independently on the surface, colliding with it or other bodies. Objects should bounce and finally come to rest on the surface. The behavior of the second scenario should be similar, but each of the clumps of objects should act as one single object.

40 32 Chapter 4. Experiments

41 Chapter 5 Results This chapter shows results of runs of the simulation scenarios. In the brick wall scenario, several images are shown for subjective comparison between AgX and the original DROP. In the convex objects simulations, speed graphs are presented. 5.1 Brick wall The brick wall scenario was simulated with a timestep of 1/120s and produced 24 frames for each simulated second. The scenario was simulated for 3 seconds. The ball impacted the wall at frame 8 and bounced back. As shown in figures 5.1 and 5.2, the AgX brick wall stood still during and after the impact of the ball. The original DROP brick wall began to wobble elastically from the impact of the ball and fell down. The AgX wall behaved like a single solid object, where as the original DROP wall behaved like a rubber object. Several different masses for the ball were simulated apart from the simulation described here. The walls behaved the same for almost all masses on the ball. However, if the ball has a high enough mass compared to the brick wall, the AgX wall will also fall down. However, the wall will not wobble nor look rubbery. 33

42 34 Chapter 5. Results (a) AgX brick wall frame(b) Original DROP brick wall 1. Ball heading towards theframe 1. Ball heading towards the wall. wall. (c) AgX brick wall frame 10. (d) Original DROP brick wall Right after the ball have col-framlided with the wall. have collided with the 10. Right after the ball wall. (e) AgX brick wall frame 20. (f) Original DROP brick wall The wall stands still. frame 20. The wall begins to wobble. (g) AgX brick wall frame 30.(h) Original DROP brick wall frame 30. The wall falls down. Figure 5.1: Simulation of a lock constrained wall getting hit by a light ball, frames The AgX wall (left column) stands still whereas the original DROP wall (right column) wobbles and falls down.

43 5.1. Brick wall 35 (a) AgX brick wall frame 40.(b) Original DROP brick wall frame 40. The wall falls down. (c) AgX brick wall frame 50. (d) Original DROP brick wall frame 50. The wall falls down. (e) AgX brick wall frame 60. (f) Original DROP brick wall frame 60. The wall slides on the ground. (g) AgX brick wall frame 70.(h) Original DROP brick wall frame 70. The wall lies on the ground. Figure 5.2: Simulation of a lock constrained wall getting hit by a light ball, frames The AgX wall (left column) stands still whereas the original DROP wall (right column) wobbles and falls down.