3 CHOPS - LIP SYNCHING PART 2

Transcription

1 3 CHOPS - LIP SYNCHING PART 2 In this lesson you will be building a more complex CHOP network to create a more automated lip-synch. This will utilize the voice split CHOP, and the voice sync CHOP. Using this method will allow you to create a network that can be used many different times for different projects. Houdini Training 4.0 Side Effects Software

2 3.1 GETTING STARTED 1. Launch Houdini. 2. From the main File menu, locate and open the following.hip file: /voicechop.hip 3. From the desktop selector button located at the top of the viewport, select the CHOPs_Training desktop. This.hip file contains a grid with several mouth shapes that are being fed into a blend SOP. Each file SOP containing the target shapes is labelled with the appropriate mouth shape. You will be working within CHOPs to ultimately create channels that will be exported to the various channels in the blend SOP. 4. Working in the CHOPs (left side) of the desktop, navigate up one level to rename the CHOP network to be voice. Press the i key to return to the CHOP network editor. 5. From the string of generator CHOPs, place a file CHOP into the tile layout window. Rename it to be phonemes. Click on the Channel File option button to select a new file. Navigate to the file called phonemes.aiff. Click on the green audio flag of this CHOP. From the Dialogs menu located at the top of the interface, select Audio Panel. The hot key for this is Ai. This will bring up the audio panel. Working with the Audio Panel. Toggle on the Test option. This will play the audio back while you are playing back the time slider. Toggle on the Graph flag of the phoneme CHOP. Home the Graph to display it in it s entirety. Hold down the information button of the phoneme CHOP. Press play to playback the audio file. What you are hearing is a voice clearly enunciating several different phonemes. These are a very good base to start. These phonemes cover most of the mouth 2 Side Effects Software 1999 Houdini Training 4.0

3 shapes that are formed when we speak. Depending on the nature of you project, you might want to increase the mouth shapes to refine the lip synching even more. By doing this you are creating what we a phoneme library. Once this is done, you will be grouping the phonemes into visually similar phonemes (visemes). These are phonemes, when pronounced, make the mouth look similar to each other. O and U are two examples. This will be done with the voicesplit CHOP. 6. Append a voicesplit CHOP to the phoneme CHOP. Toggle on the Graph flag of the voicesplit CHOP. You will notice that the default settings of the voicesplit CHOP has modified the audio file. Toggle on the information button of the voicesplit CHOP. Notice that the voicesplit CHOP created fifteen channels out of the original audio file. 7. Toggle on the Graph flag of the phoneme CHOP. Count the number of peaks in the curve. These represent each phoneme being pronounced. There are only 13 peaks Notice how there are only 13 peaks. The voicesplit CHOP is taking the small peaks during a period of silence, and making it a channel. When a channel amplitude is higher than the silence level setting in the voicesplit CHOP, then the voicesplit CHOP will consider it to be a phoneme being uttered. Working, once again, with the voicesplit CHOP. Increase the Silence Level interactively until you see that there are only 13 channels remaining. The value should be around By default the channel names range from chan1 through channel 13. The voicesplit CHOP will allow you to name the channels within the CHOP. This saves you having to use a rename CHOP. You are going to name the channels, and at the same time, you are going to group them based on their common viseme. Houdini Training 4.0 Side Effects Software

4 8. While displaying the parameters of the voicesplit CHOP, playback the audio for the phoneme CHOP. In the Channel Names field. delete the default chan1, and replace it with the pronounced phonemes in the order in which they are uttered. Refer to the illustration of the voicesplit CHOPs parameters to view the channels names. This will allow you to scroll the playback interactively. Grab the blue timebar with the left mouse button and move it to the right along the graph. Notice also that most of the phoneme channel names end with a number. These numbers represent each group of visemes. The next step is to separate the different phonemes. One group will a group of voiced phonemes. These are phonemes that require the use of the vocal chords to make a sounds ( ah, oo, l, m ) to name a few. The second group will be a group of non-voiced phonemes ( f, th, s, p ). The reason for this, is that the voicesync CHOP has separate inputs for both types. 9. Append a delete CHOP out of the voice CHOP. Rename it to be voiced. In the Delete field, remove the t* default, and enter the channel names of the non-voiced channels: B CH4 F SH4 T 10. Ac to copy this CHOP, and Av to paste a duplicate of it. Rename it to be non_voiced. In the Delete field, change it from Scoped channels to Non-Scoped channels. You have now grouped off the channels into voiced and non-voiced groups. You are almost ready to use the voicesync CHOP. But first, you have to load in a different audio file. 11. From the list of generators CHOPs, place another file CHOP to the left of the existing CHOP network. Click on the Channel File parameter button to allow you to navigate, and select a new file. Rename this CHOP to be audio_in. Find and load the file called testing.aiff. 4 Side Effects Software 1999 Houdini Training 4.0

5 Toggle on the audio_in CHOPs audio flag. Ai to open the Audio panel. If it is not set to Test, then do so. Press play to hear the audio. You are now ready to pipe this audio into a voicesync CHOP. 12. Press the T key, and from the Filters menu, select a voicesync CHOP, and place it below both CHOP networks. Audio In Voiced Phonemes Non-Voiced Phonemes 13. Pipe the output of the audio_in CHOP into the left-hand input of the voicesync CHOP. Pipe the output of the voiced CHOP into the middle input of the voicesync CHOP. Finally, pipe the output of the non_voiced CHOP into the right-hand input of the voicesync CHOP. Toggle on the information button of the voicesync CHOP. Notice what the voicesync CHOP has done with the groups of visemes. The voice sync CHOP has in effect merged the similar viseme channels together. This way you will only have to export one channel to a mouth shape that makes the shape of each viseme group. This saves having to use a math CHOP to do the same thing. Note: This is the only CHOP that groups channels this way. Houdini Training 4.0 Side Effects Software

6 Also, the it creates an additional channel for when the audio files amplitude goes below the Silence Level. This channel is called SIL (for silence) and will be used as a mouth rest position when there is no sounds being uttered. Toggle on the Graph flag of the voicesync CHOP. Hit the Home button to home the graph if necessary. You will notice that many of the channels remain static, and therefore, would not provide adequate channels for lip-synching. 14. Toggle on the information button of the phoneme CHOP. Note that the sample rate of the audio channel is Hz. 15. Toggle on the information button of the audio_in CHOP. Note that its sample rate is set to 8000Hz. You have to resample the audio file being read in by the audio_in CHOP. Generally you do not need a file with such a high sample rate. For lip synching anywhere between is sufficient. This greatly reduced the amount of memory used up by Houdini. Similarly you do not need a stereo audio file. A mono file will do the trick, plus it cuts down the file size by half. 16. \ on the output of the phoneme CHOP, and from the Filters menu, select a resample CHOP. Place it between the phoneme CHOP, and the voicesplit CHOP. In the resample CHOPs Sample Rate field, change it to Toggle on the Graph flag of the voicesync CHOP. 6 Side Effects Software 1999 Houdini Training 4.0

7 You should now see that each of the channels as step curves representing the voicesync CHOPs interpretation of the Audio Source channel compared to the voiced and non-voiced phonemes. These step curves are the curves that will be controlling the blend channels. If you exported them in their current state, you would have somewhat choppy mouth movements. 18. Append a filter CHOP after the voicesync CHOP. Toggle on its Graph flag. Filter Width = 1 Notice what the default values do to the curves. They are no longer the hard edged step curves, but they are a little too smooth. Interactively decrease the Filter Width parameter. The final value will be up to your discretion. It depends on the character you are animating. Houdini Training 4.0 Side Effects Software

8 For now, however, enter a value of 0.2. Filter Width = 0.2 You are now ready to export these channels up to the blend SOP. Obviously the channels names at their default cannot be exported anywhere...so: 19. Append a rename CHOP to the filter CHOP. In the To field, enter: Mouth:mouth_shape:blend[1-9] Toggle on the export flag of the rename CHOP. Press play. At the moment the animation is somewhat rough, but you shall leave it as is for now. The next step is to build a second CHOP network which will control the various facial expressions which are built into the geometry. These expressions are narrow, smile, frown and smirk. ADDING FACIAL EXPRESSIONS Narrow Smile Frown Smirk 1. While working in the existing CHOPs network editor, press the u key to navigate up to the root of CHOPs. 2. Add a new CHOP network, and rename it to be expression. 8 Side Effects Software 1999 Houdini Training 4.0

9 3. Enter into the expression CHOP editor. You are going to build a CHOP network that will allow you to control the expression interactively as you are playing back the animation. This will require you to use different input CHOP. The keyboard CHOP as well as the mouse CHOP. 4. From the list of Generator CHOPs, place a keyboard CHOP into the tile layout window. You are going to be controlling the triggering of each expression by pressing the numbers 1-5 on the keyboard. Then you will be using the mouse to control the amplitude the expression. Working with the keyboard CHOPs parameters. In the Name 1 field change it from k1 to normal. Leave the Type field at Momentary. Leave the Key 1 field at 1. In the Name 2 field enter narrow. Leave its parameters as they are (i.e. Type 2 set to Momentary, and Key 2 set to 2) In the Name 3 field enter smile. Leave its default as they are. Open the Keys 2 folder.. In the Name 4 field, enter frown. In the Name 5 field, enter smirk. Toggle on the Graph flag of the keyboard CHOP. Houdini Training 4.0 Side Effects Software

10 At the top of the Graph editor, change the Graphs settings to All Scoped Channels. Press the 1 key Notice how nothing appears to be happening in the graph window. hmmmm Press the Scroll Lock key on the keyboard. You will notice that once you press the Scroll Lock key, the Play Bar area turns Orange. What the Scroll Lock key does, is locks out all regular keyboard command, that might be used to control the regular Houdini interface, and feeds them directly to CHOPs. Depending on what keys you have programmed it will not show you anything until you hit a matching key. 5. Press any of the key from 1-5. Now you will notice that the channel which matches the key number will rise up to a value of 1, and then come back down to 0. You actually want the value to remain at 1 and hold until another key is pressed. 6. Append a logic CHOP after the keyboard CHOP. Toggle on its Graph flag. Change the Channel Pre-op button from Off to Radio Button. Open the Common folder. Toggle on the Time Slice button. Press any of the keys from 1-5. The channels are now operating more like a switch. When the narrow channel is on, it will remain on, until another number is pressed. The modelled face in the rest position is the first input into the Blend SOP. Since that is the default facial position, it is not necessary to override this with a normal channel. In other words, when there is no other expression being triggered by the keyboard CHOP, it will default to the rest facial position. Therefore the normal channel will not be exported, but serves the function of turning off any of the other facial expressions. 7. Append a delete CHOP after the logic CHOP. In the Channel Names field, delete the t* default, and enter normal. 10 Side Effects Software 1999 Houdini Training 4.0

11 Note: If you are having trouble entering things in the keyboard, it is because the Scroll Lock key is still enabled. If necessary, toggle off the Scroll Lock key. Toggle on the Graph flag of the delete CHOP. You now only have the four expression channels. SETTING UP THE MOUSE CHOP 1. From the list of Generator CHOPs, place a mouse CHOP to the right of the existing CHOP network. squeak Change the Position X field from geo1:tx to mag. Delete the Position Y value. The default values for the X parameter go from -1 to 1. You want the magitude to be in the positive range. 2. Append a math CHOP after the mouse CHOP. Open its Range folder. In the From Range Min. and Max. fields, enter -1 and 1 respectively. In the To Range Min. and Max. fields, enter 0 and 0.5 respectively. 3. Append another math CHOP out of the delete CHOP. Pipe the output of the math1 CHOP into the newly placed math CHOP. In the math2 CHOPs OP folder, set the Combine CHOPs parameter to Multiply. Toggle on the Graph flag of the math2 CHOP. Houdini Training 4.0 Side Effects Software

12 Toggle on the Scroll Lock button. Press the 2 key. This makes the narrow channel a value other than 0. What the current value is, however, is determined by the position of the mouse relative to its X position on the screen. Move the mouse around, and you will notice that the narrow channel is increasing and decreasing depending on your mouse movement. Toggle off the Scroll Lock key. At the moment each action of changing expressions is going to happen instantly. This might be the desired look. However, if you want to make the movement a little more smooth and fluid, then a lag CHOP will take care of this. 4. Append a lag CHOP after the math2 CHOP. Leave it at its defaults for now. Toggle on its Graph flag. Depending on what kind of look you want for your character. You can decrease the Lag Min. and Lag Max values to reduce the amount of lag, or you can increase it. That is all the setup that you need to do in this CHOP network, the rest will be done in the voice CHOP network. From there, you are going to fetch in the expression channels from the lag CHOP and merge everything together. MERGING EVERYTHING TOGETHER 1. Switch from the expression CHOP network to the voice CHOP network. 2. Place a fetch CHOP to the right of the filter CHOP. Rename this CHOP to be get_expr_channels. In the OP field change it from obj to ch. In the Path field. select expression. 12 Side Effects Software 1999 Houdini Training 4.0

13 In the second Path field select lag1. Leave the Channels field at *. In the Fetch field, change it from OP Control Channels to CHOP Data Channels. Toggle on its Graph flag. You should see the four expression channels in the graph window. The channels should reflect the last setting that they were at when you left them. 3. \ on the output of the filter CHOP, and insert a merge CHOP into the chain. 4. Pipe the output of the fetch CHOP into the newly placed merge CHOP. Now you have an additional four channels being piped into the rename CHOP, so you have to make one minor change in the rename CHOP to allow the expression channels to be exported also. 5. Select the rename CHOP to make it the active CHOP. Rename it (the CHOP) to to_sops Change the rename to field to Mouth:mouth_shape:blend[1-13] Houdini Training 4.0 Side Effects Software

14 For the convenience of the lesson, the final four inputs of the blend SOP match the order of the expression channels. (So you do not have to worry) If the order of the channels did not match, the easiest solution would be to reorder the inputs into the blend SOP. This is much easier than changing the order going of channels in CHOP. 6. Press the Scroll Lock button. 7. Press play to view the animation. While it is playing back, move the mouse around, and press any of the numbers 1-5 on the keyboard. The mouth is now speaking and you are interactively controlling the expressions. RECORDING AN ANIMATION SEQUENCE Now that you have created an interactive method of animating a character. You are basically controlling it like a puppet, but at the moment you cannot render out your animation, because it is being generated each time you press play. What you are going to do, is record your animation sequence. 1. Append a record CHOP after the to_sops CHOP. 2. Ensure that the method of playback is set to Stop at end. Toggle on Real Time. 3. Toggle on the Graph flag of the record CHOP. Ensure that the Scroll Lock key is enabled. Before you press play, remember that while you are going through the animation, you are recording your expressions with the keyboard and mouse CHOPs. 14 Side Effects Software 1999 Houdini Training 4.0

15 4. Press play...and animate the facial expressions. When it reaches the end of the animation, the counter will stop. Important...do not reset your frame counter, and press play to view what you have done. You are still in recording mode, and you will over=write your animation if you do this. 5. Lock off the record CHOP. Toggle on the Export flag of the record CHOP. Toggle off the Scroll Lock button. Press play. You are now viewing the animation which you recorded a moment ago. Rename this CHOP to be Anim1. Disconnect this CHOP from the CHOP network. 6. Repeat the last five steps, and rename the new record CHOP Anim2. Because these CHOPs are locked off, disconnecting them from the Network will not alter the animation channels contained within this CHOP. Using this method of recording the animation passes will allow you to easily experiment with different expressions, record them, and then play them back with the ability to quickly compare them. When you decide on the right one, then you leave its Export flag on, and render away. You can also label these depending what emotion they are conveying (i.e. happy Angry Sarcastic ) NOT HAPPY WITH THE LIPSYNCHING? When you play back the animation, you will notice some problems with the way that the voicesync CHOP is interpreting the audio data. For example, the curve which represents the F phoneme is static, even though the word Four is uttered. One solution to this problem is to have the same person providing the audio for lipsynching, also provide the phonemes. In this lesson so far, the voices for the phonemes, and lipsynching data are two different people. Houdini Training 4.0 Side Effects Software

16 If you do not have the ability to get the same person, then you should try and get someone of the same gender, age and similar sounding voice. THe closer it is, the better. It is also important when recording the phonemes to have as little background noise as possible. Any unwanted spikes in the channels can have a detrimental effect on the interpretation by the voicesync CHOP, which will result in poor animation. 1. Select the file1 CHOP to make it the active CHOP. In the Channel File field, click on the parameter butt, and load up the following audio file:../mark.aiff This audio file is the same content as the other audio file, except that it is spoken by the same person in the audio_in CHOP. 2. Toggle on the Graph flag of the voicesync CHOP. Notice the difference in the way the voicesync CHOP is interpreting the audio data with respect to the voiced and non-voiced phonemes. 16 Side Effects Software 1999 Houdini Training 4.0