Facade Stanford Facial Animation System. Instruction Manual:

Transcription

1

2 Facade Stanford Facial Animation System Instruction Manual: A. Introduction 1. What is Facade? 2. What can you do with Facade? 3. What can t you do with Facade? 4. Who can use Facade? B. Overview of Facade features 1. Visualization Tool 2. Facial Tool 3. Animation and Rendering Tool 4. Camera Tool 5. Lip-sync Tool 6. Texturing Tool 7. Lighting Tool C. Resources 1. Smart Library 2. Animation Demos D. Tutorial: A quick guide to modeling and animating in Facade 2

3 A. Introduction 1. What is Facade? Facade is a parameterized facial animation system that uses a fixed polygon face topology. It can be used to create and animate facial types as well as facial expressions. The first version of the software was written by Steve DiPaola and later developed by the class of Art173A The Illusion of Life: Facial Animation (Winter 2001) at Stanford University. For more information on Facade visit 2. What can you do with Facade? You can use Facade to create character types, from human-like realistic characters to exaggerated cartoon characters. You can use existing character types or expressions to create new ones of your own by changing selected facial characteristics. After creating characters, you can create your own animation short. You simply set key frames that Facade can then interpolate between. You can include automatic lip-sync in your animation. You can add your own textures to a Facade face. You can manipulate camera movements in your animation. You can manipulate the lighting of each scene in your animation. 3. What can t you do with Facade? Facade can only be used to do facial animation. If you need to do body animation then you can use Facade to animate just the face of the character. 4. Who can use Facade? Pretty much anybody can use Facade. If you would like to experiment with creating character types, facial expressions and animated shorts using faces, then Facade would be an appropriate tool to use. The Library provided with the software has multiple character types and expressions so if you are a beginner you don t have to start from scratch. You can easily use and modify existing faces. 3

4 B. Overview of Facade features The features of Facade include visualization tools, parameter handling, animation, camera handling, and a command line interface. Some features such as lighting, texturing, and lip-sync are activated using text commands. The rest of the features are activated via the graphical user interface windows (GUI). Here is a list of the different features: Visualization tool (GUI) Facial tool (GUI) Animation and Rendering tool (GUI) Camera tool (GUI) Lip-sync tool (command line) Texturing tool (command line) Lighting tool (command line) 1. Visualization Tool You can use this tool to view the face you are working on. By clicking on the face and dragging your mouse you can view the face from all different angles. Also, if you click the right button of your mouse in the visualization window, a menu will pop up. Menu options: Draw Mode o Wireframe o Polygonal Patches o Smooth surface Wireframe mode Polygonal Patches mode Smooth Surface mode Face reset - Simply resets the face Print help - Prints help instructions in the command window 4

5 Quit 2. Facial Tool The Facade Facial Tool contains all the parameters of the face that you can control along with facilities for filtering out specific face characteristics, loading and saving files, and taking screenshots of your characters. a. File system Using the buttons on the bottom right you can load and save files. To load a file, type its name in the Filename text box and hit the Load button. The file must be present in the Library folder, which is found in the Facade main directory. To save a file, type its new name in the text box and hit Save. The file will be saved in the Library folder. b. Parameters Each parameter controls a specific part of the face. For instance, the Lid parameter controls how open or close the eyelid will be. You can adjust each parameter in real time by clicking on the buttons next to each text box or by actually typing in a number in the text boxes. A complete list of parameters along with a description of what they do and their minimum and maximum values follows: Eyes Lid Open/close eyelids Scale X Scale eyes horizontally

6 Scale Y Scale eyes vertically Move X Move eyes horizontally Pupil Rot X Move pupils horizontally Pupil Rot Y Move pupils vertically Brow Center Change eyebrow center Brow Range Change eyebrow range Brow Move Y Move eyebrows up/down Brow Tip Y Move inner part of eyebrow up/down Brow Up Move eyebrows vertically Mouth Jaw Rot Rotate jaw Upper Lip Move upper lip up/down Scale X Scale mouth horizontally Thickness Change lip thickness Smile Change degree of smile Top Lip o Change top lip to an o shape Bot Lip o Change bottom lip to an o shape Lip Corn X Extend end of mouth horizontally Top Lip Out X Horizontal movements of upper lip Top Lip In X Top Lip Mid X Lip Corn Y Extend end of mouth vertically Top Lip Out Y Vertical movements of upper lip Top Lip In Y Top Lip Mid Y Lip Corn Z Extend end of mouth in/out Top Lip Out Z In/out movements of upper lip Top Lip In Z Top Lip Z Bot Lip Out Y Vertical movements of lower lip Bot Lip In Y Bot Lip Mid Y Bot Lip Z In/out movements of lower lip Face Nose Tip ScX Scale nose tip horizontally Bridge ScX Scale nose bridge horizontally Nose Tip Y Move nose tip vertically Nose Tip Z Move nose tip in/out Jaw ScX Scale jaw horizontally Cheek ScX Scale cheeks horizontally Chin Y Move chin vertically Chin Z Move chin in/out

7 Rot Neck X Rotate neck around the x-axis Rot Neck Y Rotate neck around the y-axis Rot Neck Z Rotate neck around the z-axis Neck Sc Make neck narrow/wide Neck Len Change neck length Chin-Mouth Scale chin from mouth down Chin-Eye Scale face from eyes down Fhead Sc Scale top of head vertically Fhead Cut Change forehead Squash Squash/stretch head On the right of the window you will notice three radio buttons: Both - If you select "Both" then the changes you make to a parameter will affect both sides of the face. Right only - If you select Right Only then the changes you make will only affect the right side of the face. Left Only - If you select Left Only then the changes you make will only affect the left side of the face. Notice that some of the parameters gray out once you select either "Left Only" or "Right Only." c. Parameter Filters At the bottom of each parameter column you will notice a checkbox. There are three checkboxes total: Read Eyes Read Mouth Read Face You can use these checkboxes to choose which parameters of a face you want to load from the Library folder. If all three checkboxes are checked (default) then every time you load a file from the Library, all the parameters related to that face will be loaded and the entire face will appear in the visualization window. However, sometimes you will want to create a new face that has the eyes of a library face and the mouth of a second library face. Thus, you will want to filter out certain parameters and only load the parameters you need. If only the Read Eyes box is checked, only the eye parameters of the library face will be loaded onto the existing face. If only the Read Mouth box is checked then only the mouth parameters will be loaded. If only the Read Face box is checked then only the face parameters will be loaded. Combinations of filters allow only the appropriate parameters to be loaded. 7

8 d. Additional Facilities Screenshots By pressing Screenshot on the bottom right side of the window, an antialiased (5 passes) screenshot of the current visualization is generated and stored in the Shots folder, which is found in the Facade main directory. If a Shots file is not present you must create one. The screenshots are saved as ppm files. 1 Undo The Undo button offers unlimited undo support for the face, eye and mouth parameters present on the parameters window. Also, once you start using the Animation window, you can use the Undo button to undelete deleted keys, "unedit" edited keys (does not work for Key 0), and remove inserted keys (does not work for Key 0). You cannot undo scaled keys, camera keys, lip-syncing, or any parameter changes made using the keyboard interface. Quit Quit the application. If you haven t saved the face you are currently working on, all changes will be lost. 3. Animation and Rendering Tool Use the animation window when you are ready to create your animated short. This window will allow you to set keyframes, add and delete keys, go through keys, play your animation, scale keys and load and save animation sequences. a. File system Use the Load Sequence button to load an animation file. All animation files must be present in the Anims folder, which is found in the Facade main directory. Type the name of an existing file and hit Load Sequence. To save a sequence, type a new name and hit Save Sequence. The file will be saved in the Anims folder. b. Adding frame keys 1 ppm - portable pixelmap format. This format can be converted to other formats (tif, jpg) using standard tools like Paint Shop Pro and Photoshop. 8

9 To add a key at a specific frame, type the desired frame number in the New Frame text field and hit Write Key. The face currently present in the visualization window will be written as a key at that frame. c. Deleting frame keys To delete a key, type the frame number at which it appears in the Frame text field on the left side of the animation window. Then hit Delete Key. d. Viewing each key To view the keys present in your animation sequence use the Next Key and Prev Key buttons at the top left side of the animation window. Each key will appear in the visualization window. e. Playing the animation To select which part of your animation you want to see, use the Play From and Play To fields at the bottom left side. In the Play From field type the frame number from which you want to start viewing your animation and at the Play To field type the last frame. To play your animation hit Play Anim at the top left side of the animation window. The animation will play at 30 frames per second in the visualization window. f. Scaling Facade allows you to scale frames, so if your animation is too fast or too slow you can easily fix it. In the Scale From field type the frame number at which you want to start scaling. At the Scale To field type the frame number at which you want scaling to end. In the Scale Fac field indicate the scaling factor. Hit Scale Anim to scale your animation. g. Choosing the type of interpolation Facade supports three types of interpolation between the key frames. Linear Quadratic Noisy Choose the one you like best by selecting the appropriate radio button. The default is linear interpolation. h. Rendering The Batch render button renders all the frames in ppm format (see screenshot feature above) and saves the sequence of images in the Movies folder. The Movies folder is found in the Facade 9

10 main directory. You can use a program like Quicktime to generate a movie from this image sequence. 4. Camera Tool The camera tool allows you to change the camera position and make animations that involve motion of the head in space, zooming in and zooming out etc. a. Camera enabling Before you can use the camera you have to enable it by checking the Enable camera box on the right side of the camera window. b. Camera movements Currently you can only move the camera using the keyboard. Use the following keys: Up arrow to pan up Down arrow to pan down Left arrow to pan left Right arrow to pan right Home Key to move the camera backward Page Up key to move the camera forward End key to zoom in Page Down key to zoom out c. File system Use the Load from File button to load an animation file. All animation files must be present in the Anims folder. Type the name of an existing file and hit Load from File. After you ve 10

11 changed the camera movements in your animation, you can save the sequence by typing a new name and hitting Save to File. The file will be saved in the Anims folder. d. Adding frame keys To add a key at a specific frame, type the desired frame number in the New Frame text field and hit Write Key. The face currently present in the visualization window will be written as a key at that frame. e. Deleting frame keys To delete a key, type the frame number at which it appears in the New Frame text field on the left side of the camera window. Then hit Delete Key. f. Scaling You can scale frames involving camera movements in the same way you can scale frames using the animation window. In the Scale From field type the frame number at which you want to start scaling. At the Scale To field type the frame number at which you want scaling to end. In the Scale Fac field indicate the scaling factor. Hit Scale Anim to scale your animation. g. Reset camera You can reset the camera to its original place by hitting the Reset Camera button on the right side of the window. 5. Lip-sync Tool Facade allows you to add automatic lip-sync to your animation. Lip-sync can only be enabled via the keyboard for now. a. Quick technical overview of the lip-sync technique used To add automatic lip-sync in your animation, all you need is a.wav file and the program BaldiSync. BaldiSync is part of a larger toolkit for speech research and application development called CSLU speech toolkit. BaldiSync takes a.wav file and does automatic phonetic transcription via a forced-alignment procedure, that is, it uses speech-recognition techniques on the.wav file to discover the phoneme boundaries. BaldiSync outputs a.sob file, which is a binary format that contains both the audio data as well as transcription information. A script called lerp.tcl Tcl/Tk can then be used to extract the alignment information from the.sob file and generate a.lips file which contains mouth parameters for every frame (assuming 30 fps) of your animation and a.wav file. For the purposes of animation, the output you care about will be the.lips file and the.wav file generated by lerp.tcl. The.lips file contains speech-related parameters and their values for every 11

12 frame of the utterance. The.wav file contains the actual sound data, which the program will play at animation-time. Both files need to go into the Anims folder located in the Facade main directory. For more information about the lip-sync implementation see our online lip-sync page. b. How to add automatic lip-sync to your animation 1. Download and install the CSLU speech toolkit, if you haven't done so already. 2. Record a.wav file 3. Generate a.sob file from your.wav file (and the text of whatever is being said in the.wav file) in the BaldiSync program which comes with the toolkit. 4. Run the lerp.tcl Tcl/Tk script to generate a.lips file which contains lip-sync parameters for every frame of the utterance, as well as a.wav file which is named correctly. Put your.lips file and the.wav file into the Anims folder. 5. Click on the visualization window to enable keyboard commands. 6. Enable lip-sync by typing "le mylipsfile.lips" into the Facade application, where mylipsfile.lips is the name of your.lips file generated in step 4. To erase something you wrote use the delete key, not the backspace key. 7. Create an animation which plays enough frames to allow the entire utterance contained in your lip-sync to play. 8. Press Play Anim in the animation window to see your animation. 6. Texturing Tool You can also texture the face you are working on using Facade s texturing feature. Note that texturing can only be enabled via the keyboard for now. In order for a texture to be used in Facade, it must be in bitmap format (BMP format) and it must be found in the main Facade directory. A number of templates for generating textures can be found in the main Facade directory also they are called facadetemplate*.bmp. a. How to create and load textures 1. Locate the texture templates in the main Facade directory. They are called facadetemplate*.bmp. 2. Load one of the templates in a program like Photoshop. 3. Paint on the template or align a face image on top of it 4. Export the result to a.bmp file and save it in the main Facade directory. 5. Click on the visualization window of Facade to enable keyboard commands. 6. Enable texturing by typing "t mytexture.bmp" into the Facade application, where mytexture.bmp is the name of the texture you wish to load. To erase something you wrote use the delete key, not the backspace key. 7. To turn texturing off, type in "t" with no parameters. 12

13 7. Lighting Tool Finally, you can modify the lighting to achieve the effects you want in your animation. Note that lighting can only be enabled via the keyboard for now. a. Available lights There are 6 different lights, each one identified by a number from 0 to back left from the bottom 1 - front right top 2- front right bottom 3 - front left bottom 4 - front left top 5 - back b. How to use lighting 1. Click on the visualization window of Facade to enable keyboard commands. 2. Type in a light number (0 5) and then hit enter. This will enable a specific light. 3. To disable a light, do the same thing again, i.e. type its number and then hit enter. 13

14 C. Resources When you download Facade you will find a number of included resources whose purpose is to help you use the tool effectively. 1. Smart Library A Library folder is included in the main Facade directory to provide a number of ready faces and expressions that you can use to see exactly what Facade is capable for. You can even use them to start off your own face or animation. The Library contains 91 files total. In particular, it contains 42 character files, 32 expression files and 17 lip-sync position files. You can differentiate between the types of files by looking at the file name. c_name Character type files - name is the name of the character. e_expr_name Expression files - expr is the type of expression (happy, sad, pleased etc) and name is the name of the character making that expression. This character should be present in the library as a character type file, e.g. e_surprised_ira is an expression of character c_ira. If the expression does not correspond to any of the characters in the library, name is replaced by a number, e.g. e_happy_1, e_happy_2 etc. as opposed to e_happy_joe, e_happy_ira etc. l_sound Lip-sync positions sound is the letter that that particular mouth position represents, e.g. l_o is the mouth position for making the sound oh. 14

15 a. Character Library 15

16 b. Expressions Library 16

17 c. Lip-sync Library 2. Animation Demos The Anims folder contains a number of animation demos made using Facade. 17

18 D. Tutorial: A quick guide to modeling and animating in Facade Let us begin by taking another look at the user interface for facial parameters in Facade. At first glance this window can be pretty daunting, but it's actually pretty intuitive once you get used to it. Before you start this tutorial, go ahead and make changes to each parameter one by one to get a feel for how the face moves and for how much control you have over the face. Remember that you can click and drag in the facade window to spin the head and look at it in three dimensions. Notice on the right hand side of the window you have the option of selecting either "Both," "Left Only," and "Right Only" for your parameter controls. Also notice that some of the parameters gray out once you select either "Left Only" or "Right Only." After you've played with the controls a bit, and you've made a face that you're happy with, go ahead and type in a name for the face in the blank box next to "Filename." Here I'm going to call my face "skeptical," since that's the expression I was going for. 18

19 Click "Save" and you now have a face added to your library. At any point in time you can type in the name of this face and click "Load" to bring it back. One of the key features of Facade is that it allows you to read just parts of faces in the library. For instance, if I made a new face and wanted skeptical eyes on that face, then I could just check the box next to "Read Eyes," uncheck the boxes next to "Read Mouth" and "Read Face," type in "skeptical" in the Filename box, and click "Load." So, here I've put skeptical eyes on an old man character that I created. 19

20 Now that you're familiar with the modeling aspects of Facade, let's try to make some animation. Download my skeptical old man library file, place the file in your Library directory and load him up in Facade. Now go to the animation window and click on the button called "Write Key." This writes a key frame at frame 0. Now let's change some parameters on the face. Making sure that the "Both" option is selected, change the rotation values for "Neck X," "Neck Y," and "Neck Z." Let's also close his eyes so set the "Lid" value to 0 for both eyes. Now select "Left Only" and raise the left corner of his mouth by slightly bringing up the value of "Smile" and "Lip Corn Y." So, now we have the skeptical old man with some slight changes made, as shown on the screenshot on the right. In the animation panel, find the empty box next to "New Frame," type in 30, and hit the "Write Key" button. This made a new keyframe at frame 30, one second later than the previous keyframe. Now make some more changes to the face. When you're done, type "60" in the New Frame box and click "Write Key." Now click "Play Anim" and watch your animation. Okay, so now we have two seconds of rather poor animation (Facade runs at 30 fps), but you can go back in and add inbetween keyframes at will. Basically, whenever you want to create a new keyframe, just type in the number of the frame at 20

21 which you want the new keyframe to be in the "New Frame" box and click "Write Key." Now let's say you messed up the timing between frames 0 and 30, and you wanted there to be three seconds between those two keyframes rather than just one. Well, go to the animation panel and set the "Scale From" to 0, the "Scale To" to 30, and "Scale Fac" to 3. Click on "Scale Anim," and you'll notice your animation is now 120 frames because you tripled the amount of time between your first and second keyframes. In the animation window, obviously you can step through your keyframes by clicking on "Next Key" and "Prev Key." Another important attribute, however, is the "Frame" box. Here you can type in any frame number of the animation and see what that frame looks like. You can also click on the arrows next to the box to scroll through the animation one frame at a time to detect where any problems in your animation may exist. If you wish to delete a key frame, simply go to that key frame and hit the "Delete Key" button. Well, that's about all you need to know to make great animation in Facade. Remember you can animate by either changing the parameters yourself and setting key frames, or by simply loading up different parts of different faces located in the Library and setting key frames. Good luck and happy animating! 21