Matchmoving Technique. MatchMoving. By Zuber Zeeb. Movement Position Tilt Roll Height Focal Length Focus. Zeeb Zuber

Transcription

1 MatchMoving By Zuber Zeeb Movement Position Tilt Roll Height Focal Length Focus

2 This documentation gives an analysis of one of the most remarkable and predominant special effects technique, its history, and how it is being expended on numerous movie productions to persevere with the ever evolving filming industry. What is Matchmoving, how does it work? Matchmoving is a technique that is used to input computer generated imagery into a live footage. The result of this skill is a realistic illusion; ensuring viewers believe the CG element is an actual component of the live footage. Typically, a matchmover s purpose is to match the live footage camera movement into a virtual camera so that the perspective view of the CGI is equivalent to the live footage when merged together. 1 View an example: Fig.0.1 Fig.0.2 Fig.0.3 This is a CG table. A virtual This is a live photographic image The final result of the matchmoving camera has been positioned in taken for the CG table to be technique and compositing. The CG the CG environment, containing inserted into. element has been inserted into the the same attributes as the live live image with the correct photographic image camera, to shadows, lighting. achieve the same perspective view for the CG element. Matchmoving or Motion-Tracking has two varieties; the first being 2Dimensional. This is where you would track the camera movement of 2D footage to input special effects like motion blur or adding other 2D features e.g. changing the image of a billboard on the 2D footage. 2 This Matchmoving is sufficient when the special effect and the footage do not alter much in perspective. Software such as After Effects contains tools that allow users to select a feature of the footage to track and follow to create a path, which is the camera movement. The special effect is inserted and follows this path. An example can be viewed at: 1 Wright, S.(Paul Temme, Dennis Mcgonagle, Chris Simpson.).(2008). Compositing Visual Effects: Essentials For The Aspiring Artist. Oxford: Elsevier. 2 Wikipedia.(2010). Match Moving.[Online]. June Available from: [Accessed: 15 th December 2010]

3 Corner selected as the specific feature The path of the corner is identified 2D Image inserted Fig.1.1 Specific feature is some location on the footage in which the software can easily identify throughout the footage. This is tracked frame by frame and then a path of this feature is created for the footage. Fig.1.2 The W.K.D advertisement is inserted into the footage and is attached to follow the path throughout the footage. The 3dimensional class of the technique is where you would recreate the 3dimensional space of the live footage and then track the camera movement of the footage using the footage and the 3dimensional space to insert the CG element, with the same perspective, into the live footage. Below is an example of the typical 3dimensional Matchmoving process, giving an idea of how CGI is inserted: Gathering data: where the matchmover gathers information that will enable them to recreate certain elements of the live footage in 3D. 3 Sketch of scene Measurements and camera characteristics A sketch of the scene including measurements and camera movement. 3 Hornung, E.(2010). The Art And Technique Of Matchmoving: Solutions for the VFX Artist. Pages 1-3. Oxford: Elsevier.

4 Set layout: where the real location of the footage needs to be recreated into 3dimensional CGI so Matchmoving can occur. Depending on what needs to be matchmoved, only specific areas of the set may need recreation. 4 Fig1.3 The two buildings have been created in CGI and positioned in the same location as the live footage buildings are. The position of the CG horse for the footage can easily be defined. The CG buildings are removed when inserting the CG horse. Fig1.4 This is a live scene. Matchmover required an input of a horse walking through the two buildings. This would only require the recreation of the buildings to help position the horse and form the correct perspective view. Defining the Camera: This is where the matchmover recreates the live footage camera and its characteristics, into a virtual camera using the information gathered so that the CG elements match in perspective with the live footage 5. There are two approaches to defining the camera motion: 4 Dobbert,T.(Willam Knibbe, Pete Gaughan, Elizabeth Cambell, Tim Turner.).(2005). Matchmoving:The Invisible Art Of Camera Tracking. Pages 7-9. CA:Dan Brodnitz. 5 Dobbert,T.(Willam Knibbe, Pete Gaughan, Elizabeth Cambell, Tim Turner.).(2005). Matchmoving:The Invisible Art Of Camera Tracking. Pages 7. CA:Dan Brodnitz.

5 Manual: In the past hand tracking was used whereby the matchmover would manually position the virtual camera by looking through the virtual cameras image plane and lining the virtual set layout accurately with the perspective view of the plate (movie screen), frame by frame. 6 Looking through the virtual camera you can see the set layout (Black lines) lined correctly with the plate. This was a time consuming process as it required you to keep moving the camera until the position was correct for each frame. Automatic: where the matchmover uses Matchmoving software that determines and utilizes the geometric properties of a given photographic image sequence to reconstruct the live footage camera movement into an animated virtual camera. 7 The following steps are used for this approach: The matchmover would set 2D markers on the live footage, either by selecting important features of the footage, or by creating markers on the location before the video is shot, and then identifying them in the software. 8 The matchmover would identify those same locations (2D markers) of the live footage on the virtual set by setting 3D locators at these points. The software then calculates straight lines from the virtual camera to the 3D locators and then to the 2D markers on the footage. When the straight lines are correctly lined up, there is only one location in which the virtual camera would be for that frame. This process is repeated for every frame of the footage. 9 View figure below. 6 Dobbert,T.(Willam Knibbe, Pete Gaughan, Elizabeth Cambell, Tim Turner.).(2005). Matchmoving:The Invisible Art Of Camera Tracking. Pages 7. CA:Dan Brodnitz. 7 Dobbert,T.(Willam Knibbe, Pete Gaughan, Elizabeth Cambell, Tim Turner.).(2005). Matchmoving:The Invisible Art Of Camera Tracking. Pages 7. CA:Dan Brodnitz. 8 Hornung, E.(2010). The Art And Technique Of Matchmoving: Solutions for the VFX Artist. Page 6. Oxford: Elsevier. 9 Hornung, E.(2010). The Art And Technique Of Matchmoving: Solutions for the VFX Artist. Pages 7-9. Oxford: Elsevier.

6 Movie screen (Image plate) CG element Inserted Straight line 2D track points 3D locators Live footage Fig.1.5 Imagine this as your final footage where the camera rotates around the phone located on the table. The phone is a CG element that has been merged successfully into the live footage using Matchmoving technique to achieve the same perspective as the live footage. Virtual camera image plane Virtual set with CGI phone. Fig.1.6 To achieve the same perspective view of the phone to the live footage, the software calculates straight lines from the image plane to the 3D locators (yellow dots) and then to the 2D track points (red dots) of the image plate. There is only one location in which the camera can be when lined up correctly. This method is known as Triangulation and is part of Photogrammetry. Matchmoving is about creating a relationship between the camera and set layout. 10 Where did Matchmoving come from, who invented it, where was it first used? The US first developed the idea of using tracking as part of their missile guidance system. In 1979 the military experimented with head mounted displays that combined magnetic head trackers to understand where the pilot was looking. 11 The Matchmoving technique was then introduced into VFX by Tom Brigham and J.P. Lewis in 1985 who implemented an FFT tracker in the NYIT graphics lab for a TV commercial that showed a coin travelling in arc as if it was a rising sun. 12 ILM developed the first 2D tracking software called MM2 which was used in the film Hook (1991) and Death Becomes Her (1992). 13 MM2 was the start of the 2D tracking softwares that allowed users to manually insert keyframe to position changes by hand. Later this process was used as the base to implement 3d tracking systems, where you would adjust the virtual camera frame by frame to get the CG element to 10 Dobbert,T.(Willam Knibbe, Pete Gaughan, Elizabeth Cambell, Tim Turner.).(2005). Matchmoving:The Invisible Art Of Camera Tracking. Page 7. CA:Dan Brodnitz (1999). Multimedia: History, Theory and Practice. [Online]. March Available from: [Accessed: 3rd December 2011] 12 Seymour, M. (2004). Art of Tracking Part 1: History of Tracking. [Online]. August Available from: [Accessed: 1 st January 2011] 13 Movie Magic.(ND). ILM Development of CGI 1980s s Part 1 of 2. Part 1. [Online Video]. Available from: [Accessed: 5 th December 2010]

7 align with the live footage. This was used in early movies such as Terminator 2 and Jurassic Park. 14 What is matchmoving based upon? Matchmoving is influenced by Rotoscoping and Photogrammetry. Rotoscoping is the process of drawing images onto live footage frame by frame to achieve an animation. Matchmoving uses a similar approach by merging CGI into live footage, frame by frame, making sure the perspective of the CGI is the same as the footage. 15 View an example at: Live footage is copied over frame by frame to produce an animation. This is the original footage. You could draw an angel on her shoulder frame by frame. Photogrammetry is used to recalculate the 3dimensional world of a live 2D image taken by a digital device e.g. camera. Photogrammetry uses a process called Triangulation. Triangulation calculates 3dimensional measurements by using 2D images, the recreation of that 2D image in 3dimension, mapping similar points on these sources and calculating straight lines from the camera to these points. 16 Shown in Fig1.6. Who has used Matchmoving to a particular good effect? Erica Hornung has ten years experience as a VFX Artist, who has worked on rich films such as Spiderman and Lord Of The Rings. 17 Hornung has published a book on Matchmoving called The Art and Technique of Matchmoving. 14 Seymour, M. (2004). Art of Tracking Part 1: History of Tracking. [Online]. August Available from: [Accessed: 1 st January 2011] 15 Rotoscope Animation Tutorial, Moore,C. (ND). Rotoscope Tutorial. [Online Video]. Available from: [Accessed: 1 st January 2011] 16 Geodetic Systems.(2006). The Basics Of Photogrammetry.[Online]. October Available from: [Accessed: 5th December 2010] 17 IMDB. (ND). Erica Hornung.[Online]. Available from: 15 th December 2010]

8 Erica works on character Matchmoving in movies, lighting, Rotoscoping and Compositing to improve the scenes visage. Hornung also works on graphic designing, creating illustrations and designing websites for clients. 18 One project Hornung worked on in 2004 was Polar Express, an Imax 3D experience. Hornung considered the integration of Stereoscopic within the movie. Hornung created stereo cameras, determined proper placement and interoccular distance and animated interoccular distance in a full CG environment. 19 Subsequently she executed further stereoscopic projects such as Ice Age 3. The general roles she has been functioning are Senior Matchmover, Digital Artist, Stereoscopic Technical Director and Technical Director. She was also involved in a game called MGS4 Guns Of Patriots where she was a designer and animator. What feature films have used Matchmoving to particularly good effect when relating to Erica Hornung? Spiderman 3: Spiderman was a leap into VFX as it required, like Avatar the movie, to insert live footage into the CG environment. 20 About 95% of the shots were CG as the filmmakers wanted to include some real elements to give the movie an enhanced realistic appearance rather a mere animation. Every vehicle within the movie was practically replaced with digital models. 21 Lord of The Rings: The Two Towers: The movie contained various scenes of CG elements and live footage seamlessly together. There was a battle scene featuring horses and riders where the director, Peter Jackson, required further development of the scene by introducing digital horses and characters so that the scene can show horses taking terrible falls. Matchmoving CGI was used to reduce costs and create an impressive scene Hornung,E.(ND). Graphics.[Online]. Available from: [Accessed: 23rd December 2010] 19 Hornung,E.(ND). Sterioscopic Reel.[Online]. Available from: [Accessed: 23rd December 2010] 20 Steinmetz, A.(ND). Matchmoving a Stereo Project using 3DEqualizer and Maya.[Online]. Available from: [Accessed: 14 th December 2010] 21 Worley,R.(2002). Spider-Man. [Online]. March Available from: [Accessed: 2 nd January 2011] 22 Digital Horses.(ND). Making of Lord of The Rings - Digital Horses. [Online Video]. Available from: [Accessed: 27 th December 2010]

9 Stuart little 2: This film uses character matchmove, which Hornung calls Rotomation. 23 The Artist must match the CGI elements according to the live character. Images from Hornung, E. The Art And Technique Of Matchmoving: Solutions for the VFX Ghost Rider: This movie required matchmoving to apply effects, such as CGI flame and driving the Hellcycle up a building. 24 After Hellcycle was built, interacting lighting or green tape was used as markers for matchmoving the CGI fire to the specific areas of the Hellcycle. 25 Images from: (2010). What is the relevance of Matchmoving to production today? Is it still much used today? Matchmoving is critical to current productions for powerful special effects that involve CGI inserted into live footage. 26 The Insertion of CGI through matchmoving is used to produce scenes that cannot be produced in reality or to save the costs for such scenes. 23 Hornung, E.(2010). The Art And Technique Of Matchmoving: Solutions for the VFX Artist. Page Oxford: Elsevier. 24 Ghost Rider, Caines, D. (ND). Ghost Rider movie blog special effects. [Online Video]. Available from: [Accessed: 2 nd January 2011] 25 Ghost Rider, Caines, D. (ND). Ghost Rider movie blog Hellcycle. [Online Video]. Available from: [ Accessed: 2 nd January 2011] 26 Luc, R.(2003). Where is matchmoving going?.[online]. September Available from: [Accessed: 20th December 2010]

10 Matchmoving has enhanced dramatically. Beforehand, softwares required Survey Based Tracking where the matchmover would identify the scene element position and measurements to help calculate the camera position and movement through tracking points. Now, matchmoving software identify 2D points and creates 3D environments with 3D points so that a virtual camera can be created. 27 However, new technologies for Matchmoving do not relinquish their preceding, but rather make the matchmoving process more efficient. If automatic Matchmoving fails, then matchmovers may require previous technologies to aid the Matchmoving process such as going back to using manual tracking until they are satisfied. 28 Matchmoving is easier to implement due to better technology and therefore nonspecialists can combine CGI with live footage. In future, matchmoving would be the process of dragging and dropping the CG element into the live action Luc, R.(2003). Where is matchmoving going?.[online]. Page 2. September Available from: [Accessed: 20th December 2010] 28 Luc, R.(2003). Where is matchmoving going?.[online]. Page 2. September Available from: [Accessed: 20th December 2010] 29 Luc, R.(2003). Where is matchmoving going?.[online]. Page 3. September Available from: [Accessed: 20th December 2010]

11 Bibliography Hornung, E.(2010). The Art And Technique Of Matchmoving: Solutions for the VFX Artist. First edition. Oxford: Elsevier. Dobbert,T.(Willam Knibbe, Pete Gaughan, Elizabeth Cambell, Tim Turner.).(2005). Matchmoving:The Invisible Art Of Camera Tracking. First edition. CA:Dan Brodnitz. Wright, S.(Paul Temme, Dennis Mcgonagle, Chris Simpson.).(2008). Compositing Visual Effects: Essentials For The Aspiring Artist. Oxford: Elsevier. Wikipedia.(2010). Match Moving.[Online]. June Available from: [Accessed: 15 th December 2010] IMDB. (ND). Erica Hornung.[Online]. Available from: 15 th December 2010] IMDB. (ND). Hook.[Online]. Available from: 15 th December 2010] IMDB. (ND). Death Becomes Her.[Online]. Available from: 15 th December 2010] Seymour, M. (2004). Art of Tracking Part 1: History of Tracking. [Online]. August Available from: [Accessed: 1 st January 2011] Luc, R.(2003). Where is matchmoving going?.[online]. September Available from: [Accessed: 20th December 2010] Montgomery,J.(2004). Art of Tracking Part 2: Tips & Apps Overview.[Online]. September Available from: [Accessed: 1 st January 2011] (1999). Multimedia: History, Theory and Practice. [Online]. March Available from: [Accessed: 3rd December 2011] Carlson,W.(2003). CGI Historical Timeline.[Online]. November Available from: [Accessed: 3rd December 2011] Geodetic Systems.(2006). The Basics Of Photogrammetry.[Online]. October Available from: [Accessed: 5th December 2010] Steinmetz, A.(ND). Matchmoving a Stereo Project using 3DEqualizer and Maya.[Online]. Available from: [Accessed: 14 th December 2010] Baburam.(2007). The Making Of Spiderman 3.[Online]. July Available from: [Accessed: 1 st January 2011]

12 Worley,R.(2002). Spider-Man. [Online]. March Available from: [Accessed: 2 nd January 2011] Weintraub,S.(2010). Exclusive GHOST RIDER 2 Update from Producer Mike De Luca; Sequel Title May Be GHOST RIDER: SPIRIT OF VENGEANCE.[Online]. January Available from: [Accessed: 25 th December 2010] Hornung,E.(ND). Sterioscopic Reel.[Online]. Available from: [Accessed: 23rd December 2010] Ghost Rider, Caines, D. (ND). Ghost Rider movie blog Hellcycle. [Online Video]. Available from: [ Accessed: 2 nd January 2011] Ghost Rider, Caines, D. (ND). Ghost Rider movie blog special effects. [Online Video]. Available from: [Accessed: 2 nd January 2011] Robkizzy.(ND). Horizon How to film the impossible 1 of 5. [Online Video]. Available from: [Accessed: 5 th December 2010] Movie Magic.(ND). ILM Development of CGI 1980s s Part 1 of 2. Part 1. [Online Video]. Available from: [Accessed: 5 th December 2010] Rotoscope Animation Tutorial, Moore,C. (ND). Rotoscope Tutorial. [Online Video]. Available from: [Accessed: 1 st January 2011] Digital Horses.(ND). Making of Lord of The Rings - Digital Horses. [Online Video]. Available from: &index=38. [Accessed: 27 th December 2010]