Studio Tour 2D Compositor

Transcription

1 Introduction Hello! My name's Jack. I'm the 2D Compositor at Lightbox Animation Studio. It's my job to put the whole animation together. I synchronise (often just called sync) the soundtrack with the digital images and merge the individual layers of animation with the backgrounds (and sometimes with live action) to make each completed frame. It's quite a complicated job and I have to carefully and accurately read the exposure sheets (X-sheets). I'm excited about the changes that computer technology has brought to my role. Animation has grown from just the name given to describe the creation of moving hand drawn cartoon characters, to a totally different world encompassing many facets of digital storytelling. Here's where I fit in - my job comes at the very end of the animation production process. 1 of 25

2 What is a Composite? When you place a layer over another layer using transparency, the result forms a composite. The background will be visible wherever the first layer is transparent, making the first layer appear to be part of the background. The art in compositing is to create an integrated whole that looks as if all the elements in the picture were photographed by a single camera that witnessed the event. Look at this example of a composite frame from an animated cartoon. The bus, the smoke, the red explosion, the grey bricks and the green background are all separate layers but are integrated in this composited 2D TVC animation. Traditional Vs Digital Compositing Composited 2D TVC animation Before the advent of computers, compositing animated elements required the compositor to work with up to six levels of acetate cel which were registered with a peg bar. The images were photographed under a rostrum camera frame by frame. Chris, the Head of Photography, can tell you more about the rostrum camera. Mistakes were costly as the film needed to be processed in order for the final effect to be seen and approved by the director. Mistakes meant the film needed to be re-shot. Computer compositing allows changes to be made quickly with images rendered and corrections made almost instantly on playback. The computer has also provided the compositor with a host of special effects. Digital compositing allows us to isolate and manipulate visual information in parts of an image giving us control over each image. Individual parts or layers of an image can be manipulated with effects such as shadows, lighting 2 of 25

3 effects, blur and colour correction without affecting other parts of the composition. Composited images are usually derived from one of three sources: Hand drawn Computer generated Film or Video Look at the composited scene created in Adobe After Effects. An example of a composited scene using Adobe After Effects software Here are the three different layers that have been combined to create the final result. Individual layers of composited image 3 of 25

4 Composited graphic elements can be composed entirely of text, box and circle shapes created in Adobe Illustrator or Adobe Photoshop. The animation below was created by "breaking apart" the final image to its essential geometric shapes of circle, square and text. By adding a light sweep effect and a lens flare, the logo is recreated using composited techniques. Compositing Audio Composited logo Audio is an integral part of the compositing process. Animated elements must all synchronize to the music, sound effects and dialogue. The compositor can apply special audio effects to audio layers, and adjust the waveform values. Audio can also influence visual elements in the timeline directly. For example, using special coding, the audio can squeeze and stretch images to create the illusion of various items "dancing to the beat of the music." The computer automates such effects making the process a breeze for the animator. Common audio formats are: AIFF, AU, MP3 and WAV. Waveform pattern in a composited sequence Scope of Digital Compositing Digital Compositing can allow the animator to control: Mattes Colour Correction 4 of 25

5 Keying Live action composites Lighting Effects such as Fire, Electric Arcs, Explosions etc Filters such as Blur, Sharpen, Drop Shadows, Glows Paint Effects such as Watercolour, Oils, Fresco etc Perspective, Skew, 3d rotation Morphs and Warping Particle Effects Retiming, slow motion effects Audio Effects Motion Tracking Image Stabilization Wire removal Rotosplines I will now explain each of these aspects of digital compositing in detail. Mattes A matte is an image designed to control the transparency and opacity of another image. Mattes are used during compositing when we only wish a portion of one image to be included in the output image. Typically, black or white areas of the matte are used to specify 100% transparency or opacity, while intermediate greys determine partial transparency/opacity. Defining the level of Transparency from white to black You may also hear the term mask used when referring to mattes because generally the two terms are interchangeable. 'Mask' is the more common term when specifying an image used to control or limit a color-correction on another image. 5 of 25

6 Creating a composite image using a matte starts with a subject that has been filmed in front of an evenly lit background, generally green or blue. Later in the compositing process, whether photographic or electronic, all the background colour in the picture is replaced with another image, known as the background plate. This known as a keyable source. This is illustrated below. Digital compositing requires a keyable source that can be turned into matte shape that can be keyed over a background plate Mattes can be animated as in this example below. A turning 3D cube has a blue shape attached to one of its sides. As the cube moves towards us, the blue shape gets bigger. This effect can be used as a "wipe" to introduce new visuals that appear in the blue window. This is an example of a simple composited effect. Colour Correction An animated matte used as a transition effect If Trina, the Art Director, needs me to change the colour of a model's dress from red to purple, I am able to isolate the necessary image detail and alter its appearance easily using color luminance information gathered from the digital image. 6 of 25

7 Colour alteration is achievable in compositing Luminance Key The luminance key keys out all the regions of a layer with a specified luminance or brightness and this keyed out region can be replaced by either another image, or a color from a color generator. This is commonly used in the creation of titles. A title card with white on black letters is placed in front of a camera. The camera signal is fed into the keyer's foreground input. In television terms, the Keyer is a box attached to the vision mixers desk that can alter the level of luminance being fed from the cameras. The background video signal is fed into the keyer. The level control knob on the keyer is adjusted to cause all the black on the title card to be replaced by the background video. The white letters now appear over the background image. In digital compositing specialised computer software has replaced the manual process described above. Luminance keying works well with titles, but suffers problems when introduced into live action composites. When we want to key people over a background image, problems arise because of the types of clothing worn, skin tones and other subtle details like hair and shoes etc. Those areas might key with the background causing parts of one's anatomy to disappear. For example if the background is green and a news presenter is wearing a green tie, the viewer will see the keyable source or new background appear where the tie should be, ruining the effect. 7 of 25

8 The key effect is selected here using a software program and the Colour Tolerance or degree of selection is adjusted by sliding a selection bar back and forth In the example shown below, a 2D animated butcher had to be animated so that he pulls a knife out of his holster and then carves up a 3D flying dollar sign. Both the butcher and the gold dollar logo had to be composited on separate layers. If you look at the knife blade you will notice a glinting starburst that appears as the knife shatters the gold dollar symbol. Notice how the glinting starburst fades gradually away. How do we key something like this without showing the blue coming through? The black area is totally transparent and the white area is totally opaque but we can also have an area in the middle, a grey area that allows some degree of transparency. This is useful in keying only partial visibility into the scene, like semi-transparent shadows and glassy surfaces where we need to see through an object. Hence the use of a luminance matte, shown below, which allows things like semi-transparent star bursts to fade away gently into your composited background. 8 of 25

9 2D animation and matte Chroma Key The Chroma Key keys out all image pixels the same as or similar to a specified key colour. Using the Chroma Key we can cut out a hole in the image allowing another image to fill through the hole creating a "key" shape. Chroma key blue is an industry standard colour. It is intense and bright enough in its chroma range to be easily removed when composited in the Film or TV studio. The compositor selects the effects needed using compositing software. Broadcast cameras such as a rostrum camera are used in traditional animation to record the composited animation. A rostrum camera uses three independent sensors, one for each colour, Red, Green and Blue. Most cameras can output these RGB signals separately from the Composite video signal. An original chroma key is created by sending the blue channel of a camera into a keyer. This works reasonably well but manufacturers have created specialised chroma-keyers that accept all three colours, plus the background composite signal and the foreground composite signal. This has made it possible to select any colour for the key. As keyers became more sophisticated, with finer control of the transition between background and foreground, the effect became less obvious and jarring. Today's high-end keyers can make a soft key that is almost invisible. 9 of 25

10 Filters A filter is an effect that can change the underlying appearance of an image. Examples of filters are: blurs sharpen distort ripple perspective glows. There are also a range of artistic filters which can be used to give a particular style to a production, for example: Coloured Pencils Dry Brush Rough Pastels as well as a range of filters for specific effects such as: Emboss Extrude Glowing edges They are commonly found in image manipulation software such as Adobe Photoshop. In the image below a Blur filter has been applied to a mask on the background layer, (shown in yellow box) to focus the eye on the central characters, highlighting and enhancing the couple. Filters such as this can be animated over time creating a dynamic experience for the viewer. 10 of 25

11 Composited image with Blur filter applied to selected layers We can also create sophisticated special effects like a perspective drop shadow to anchor our character into the background. In the previous image, the digital shadow is actually a distortion of the couple's physical shape. We have complete control over its colour, opacity and fuzziness. Paint Effects Most good compositing programs allow you to paint directly over your sequence of images. Paint effects are more than just applying colour or drawing over your video. Most Paint effects allow the compositor to paint a series of strokes that can be used to mask off sections of the background that need to be removed. Sometimes unwanted detail, such as wire rigging or smudge marks in the original animation, need to be taken out. Wire-removal is within the duties of compositors when actors are required to perform with animated characters. Where actors are needed to fly in the air and interact with animated characters, motion rigs that transport the actor through the air need to have all traces of support lines removed to achieve the appearance of effortless flight. Wire removal can be either hand-painted out with paint effects, which is slow and laborious or semi automated with rotosplines. (I will discuss these in detail later on.) Paint effects can also be used to flood the screen with texture. An example of this would be to populate a flat background with a forest setting. Each stroke of the pen will create a variety of pre-made trees and shrubs that the compositor can add with ease. Perspective, Skew, and 3D Rotation 11 of 25

12 Perspective, Skew and 3D Rotation allow complete control over animated layers and enable the compositor to independently alter the compositional depth of the final presentation. Animated layers can be repositioned and some compositional details can be enhanced, by rearranging the different levels. Animated layers can be flipped so that characters are facing different directions, eliminating the need to redraw characters at different angles. Recycling various elements can save thousands of dollars. Morphing Images rotated in perspective Morphing allows the compositor to turn one shape gradually into another. For example, a human face can morph into a dog's face, by mapping a series of points or grids over various human facial characteristics such as eyes, nose and chin and matching them to the animal's similar facial characteristics. These two sets of facial characteristics can then be blended over time as the software remaps and distorts the image. 12 of 25

13 A nose is being warped by the distorted grid shape Warping There are many different methods to warp a character. The old system involved a grid shape that was applied to the subject. When points were moved from A to B this resulted in a simple Warp effect. Computer software uses mathematical points (commonly called Spline or Bezier points) to place a set of co-ordinates onto the images. These points are used to distort the underlying image over time. Modern methods have evolved where splines (a series of points connected by a line or curve) or simple Bezier curves are drawn on to the subject in the compositor window and warped as easily as redrawing the new curve in a different location. The software does the rest. In the following image, the original curve was drawn on the upper eyelid. A new curve was drawn above the eye and the software warps the image and pushes the pixels to their new location. Curves Warping with Bezier 13 of 25

14 Other Effects Particle Effects are mathematical points that the user can select to create simple effects such as explosions or starbursts. They replace the tedious hand animation of minute details, such as the animation of 100 fairy lights, smoke trails or fire effects. Particles can create the illusion of chaos under controlled circumstances. The points can be substituted by proxy to show 100 dancing butterflies or 100 dollar notes falling to the ground. Images can be attached to each particle and the computer simulates the gravity and force that may play a part in the particle world using real forces interactively. Altering speed and timing is a digital effect that compositors can produce. Just as the compositor has control over vision in this way, audio can be adjusted as well. The waveform can be reversed and played backwards. The pitch can be sped up or slowed down. Audio effects such as reverb, delay, flange and chorus can be added. Tim, the Sound Designer, can tell you more about the audio aspects of animation. Motion Tracking Most 2D compositing software offers some form of Motion Tracking. The software tracks motion by matching pixels from a selected area in the first frame to pixels in each succeeding frame, thereby following that motion. For example, suppose you have footage of a bareheaded person walking, and you'd like to add a hat to this person's head. Motion tracker will look at each successive frame mapping the bare head and apply the hat in the correct location. The following image shows a car mirror being used to lock the 3D model of the aircraft chasing the vehicle. Motion Tracking was employed to keep the aircraft locked to the mirror. Because the video was handheld, any camera movement had to be tracked by the compositing software to create the illusion of realism required. Motion tracking 14 of 25

15 Image stabilisation is an option offered by 2D compositing software to smooth out bumpy hand-held motion that needs correction. The Stabilizer controls can track the specified range of bumpy motion and then adjust the layer's anchor point or rotation. When played back, the motion appears smooth because the layer itself moves incrementally to offset the unwanted motion. Rotosplines What do you do when you need to remove a subject from its background but you forgot to film the subject in front of a keyable source? If you've tried to cut out a shape in Photoshop you realize how time consuming and difficult the procedure is. Imagine if you need to do it at 25 frames per second. Quite a challenge isn't it? Well, yes and no. This is where rotosplines become useful. Essentially a rotospline is an animated mask that is placed around the subject's body shape. The mask has various points that can be keyframed along the timeline creating a moving mask. The compositor can also soften the edges where motion is blurred or difficult to see. Look at the image below. There are many effects happening in this shot. I have inserted the subject into a dramatic sky that has been colour corrected and given a blue filter. The green cane field is taken from another scene and blended into this sequence. A yellow spotlight has been created and composited into this scene to create the illusion of light falling down from the heavens. 15 of 25

16 A Rotospline (animated mask) allows the compositor to replace the background in difficult-to-key footage. Rotoscoping Rotoscoping is a process in which an artist traces an image from a film frame so that the movements on the film can be used in other ways. Over the years, rotoscoping has become an important facet of both traditional animation and live-action visual effects. Rotoscoping software works using splines, which are a series of points connected by a line or curve. These splines are adjusted from frame to frame, so that they continue to conform to whatever shape the artist is tracing. Because rotoscoping software includes the tools to paint an image, rotoscope artists now find themselves doing a lot of paintwork as well. Rotoscoping allows the compositor the scope to transform moving video into animation. The trick is in knowing what to leave out. Following a moving object such as this dog with its blurred feet, in the image below, requires the artist to simplify the visual motion. 16 of 25

17 Transforming video into animation using rotoscoping 17 of 25

18 Animated Masks Let's take an example of creating the illusion of chalk writing text on a blackboard. This isn't as difficult as it may first seem. The first requirement is to create a 3D animation of a solid piece of chalk animated as a random pattern of strokes and jerky motion to create the illusion that it is creating the writing. A compositing example showing 3D piece of chalk writing on a blackboard In fact all the chalk is required to do, is "dance on the spot." The real work is handled by an animated mask, which is required to reveal the words gradually. The mask, shown below, is inversed and told to play back at a certain speed. The 3D chalk-writing movie is then keyframed to match the stroke of the words forming. The red mask is animated over time to reveal the text underneath The timeline below shows the key frames of chalk matching the revealed section of the animated text matte. The wriggly line below the key frames measures the speed of the motion. The sharper the line spike, the faster the chalk whizzes past "writing out its message." 18 of 25

19 Timeline of the compositor showing the key frames of the chalk movie carefully matched to playback the animated text matte The 3d animated movie of the chalk randomly writing is composited into the blackboard Animated Television Commercials (TVC's) Its not unusual for some animated TVC's to have many layers. Let's look at one I have recently completed. The commercial below has 24 layers, comprised of: background text small animated movies starbursts explosions glow effects. Small 2D animated movies were made over a green keyable background and the entire composition was composited in Adobe After Effects, a professional compositing program used to create motion graphics. 19 of 25

20 Screen shot of Adobe After Effects Considerable amount of time was taken to create the individual components, so that when integrated into the final composition, the effect appears smooth and effortless. 20 of 25

21 Special Effects Each element in this TVC occupies a layer in the compositing package. The animations appear separately on a green keyable background. Many different graphic components are integrated to create this animation. Its hard to watch a feature film these days without seeing composited special effects used somewhere in the film. Whether it's just a film processing technique of colour correcting the footage or tinting the sky, or spectacular explosions, special effects are here to stay. Compositors are in increased demand in the special effects industy as modern computer software gives power to individuals to blur the gap between fantasy and reality. The explosions featured in the movie image below were taken from stock footage special effects CD's created to service the film industry. The actor is asked to dodge and react to invisible staffing fire from an aircraft that will be added in later in post-production. 21 of 25

22 3D model aircraft mixed with 2D explosions were featured in this special effects film Effects such as this were once only available on million dollar budget special effects films, however scenes like this can now be created on a home PC with a Digital Video (DV) Firewire card for little cost. A DV Firewire card allows a computer to capture footage from a DV Camera. Once in the computer it can be edited and then sent back to the camera without any loss or degradation of image. To assist in the creation of composited animated effects, third party manufacturers have created a number of plugins that offer a range of powerful effects. The animated pixie dust in the following image is a digital effect created by a third party plugin from Digieffects that offers a tremendous amount of power to the compositor. 22 of 25

23 The pixie dust is a digital effect created by a plugin Often created to assist special effects artists working on big budget films, the range of special effects include: balls of light lens flares electric arcs old film-look fire fog warping effects. 23 of 25

24 Motion Graphics Some of the different special effect plugins available to the digital compositor Composited effects are not just limited to animated characters. Motion graphics is a new area that is visible on TV station idents, logos, advertisements and film credit rolls. This is a specialised field in its own right and consists essentially of animated graphics, grids, glows, textures and lights effects to create a sophisticated visual look to moving text and graphics. Motion graphics, such as the following example for a travel agent, consists mainly of 2D graphics created in Illustrator and Photoshop. Its strength comes from the volumetric effect of flickering light that gives this TVC its powerful energy. 24 of 25

25 Motion graphics used in a TVC The last thing I have to tell you about compositing is perhaps the most important. The secret of good compositing is to create a believable unified reality, where the audience is unaware that the film they are viewing has many separate layers and the final product is skilfully woven into a compelling experience. If you want to learn more about compositing there are some Useful Web Sites you might like to visit. 25 of 25