Motion Capture, or Mo-Cap for short, is a way of recording the movement of people and objects, which can be used in filmmaking. In computer animation and visual effects studios the technique is used to combine actions in the real world with digitally-created computer objects and actions. The more advanced form of Mo-Cap, which is used to recreate facial expressions, is called 'Performance Capture'. Perhaps the technique's biggest success to date is the character of Gollum, who was played by Andy Serkis in The Lord of the Rings: The Two Towers (2002). James Cameron's Avatar (2009) is another milestone. The technique has also been used to make animated films, particularly those by Robert Zemeckis' studio ImageMovers.

Mo-Cap

For Mo-Cap, performers wear special Mo-Cap suits with markers, which traditionally look like blue  ping-pong balls, attached to key parts of their body. As a performer moves around, cameras record the markers' motion and the information is fed into a computer. The computer is able to replicate the movement of the markers on a computer-generated character, moving the character in an identical fashion to the original performer. So if a performer waves their arm, for example, the computer sees these markers moving then each marker is linked to the relevant part of a computer-created character and that movement is copied by the character.

Where To Film

Mo-Cap can be utilised within location filming to include genuine backgrounds or, for the greatest level of control, within special soundstages called a Volume.

Volume

The Volume is a three-dimensional cube-shaped stage area. Instead of ordinary cameras, the Volume is surrounded by numerous Receptors on each of the four walls and possibly even the ceiling. Spotlights flood the Volume with infrared light. The Receptors then detect any movement made within the Volume.

All the actors and props inside the Volume have markers on their bodies and faces - typically 250 markers on each actor. The props are wire-framed hollow models that allow the infrared light to pass through and not block the Receptors while allowing the actors to interact with them realistically. Each prop has a barcode to allow the computer to realise which prop is in use. Large-scale props and rigs are used to make everything as realistic as possible. So for example, door props are used to capture the movement of opening a door. For the flying scenes in A Christmas Carol (2009), Jim Carrey was hoisted up in the air. In Beowulf (2007) to recreate the motion of a ship, a boat-shaped rig was built that actually swayed side-to-side. For the horse-riding scenes in the same film, real horses covered in 95 markers were brought into the Volume to capture a horse's movements, and were even ridden up a replica spiral staircase.

Computer software combines all the Receptors' data, triangulating the information to make a virtual reconstruction viewable from any conceivable angle.

On Location

Mo-Cap can also be used on location and is often combined with green-screen technology. For example, The Lion, the Witch and the Wardrobe (2005) is a typical fantasy film that utilises special effects to bring a range of unearthly characters and creatures to life. These include creatures who are part-human and part-animal, such as goat-like fauns, centaurs who are half-human and half-horse, minotaurs and so forth. The human actors wear costumes that include sections made of a tight green fabric covered with orange tracking marker dots in the areas to be edited out. In the case of a faun, which resembles a human with goat-like legs, this would mean the actor wearing green tights with orange dots. The dots help keep a sense of perspective, distance and depth of field to ensure the characters remain in focus. They also ensure that when the actors move during their scenes, the computer records their movement and weight distribution information. The actor's legs can then be easily edited out of the scene and realistically replaced by their computer-generated goat equivalent.

T-Pose

A T-Pose is nothing to do with standing around waiting for the kettle to boil so you can make a nice cuppa. Before actors begin the Mo-Cap process, they stand in what is called a T-Pose, with legs together and arms out to the sides like a letter T. This exposes all the markers on the body and face allowing the computers to pick up their exact location. Actors may also have to make a series of movements called ROM (Range of Motion) movements to ensure their every action is being picked up.

Dead Eyes and Uncanny Valley

In animation, Mo-Cap has been criticised for leading to eerily realistic-looking creepy characters who appear to have strangely plastic skin and dead eyes, in a phenomenon known as 'uncanny valley'. This postulates that the more humanlike something appears, the more positively humans respond to it up until a point; following this point people feel increasing revulsion and creepiness when confronted by artificial replicas. The more the replica appears almost, but not quite, entirely like a real person, the greater the revulsion.

Performance Capture

When Mo-Cap is used to replicate far more subtle movement, particularly facial expressions, it is called 'Performance Capture'. For example, as ImageMovers' The Polar Express was heavily criticised for the characters' dead eyes, for Beowulf they used Electrooculograph electrodes to record eye-muscle movement to convert what the actor's eyes were doing into movements on the animated character, creating more realistic eyes. Later, high definition video camera helmets were developed to film every movement of an actor's face and allow its digital replication.

Technique Advantages and Disadvantages

Advantages

• Mo-Cap and Performance Capture can create digital characters with stunning degrees of realism, who move naturally in exactly the same way people do in real life.

• As Mo-Cap records performances in a 3D environment, it excels at creating realistic 3D films that are more impressive than films recorded in 2D and converted into 3D. For example, all ImageMovers' films have been released in 3D.

• An actor is a completely blank slate and can become anyone or anything. An actor in a Mo-Cap suit can convincingly play a giant gorilla, a young Cleopatra, an elderly Gandhi, Nelson Mandela or anyone or anything all in the same day. Their age and appearance is irrelevant, only their body and facial muscle movements matter.

• It can be quick to film. A film traditionally needs a plan of the camera angles from which a scene is to be captured, and cameras need to be moved into position, possibly requiring the scene to be re-enacted several times to ensure it can be captured in long shot and close-up from all the required perspectives. With Mo-Cap the scene only needs to be enacted once and the computer combines the data from all the Receptors so that the digital characters can be seen from any conceivable angle.

Disadvantages

• Uncanny Valley.

• Although filming itself is quick, processing and turning the filmed actions into realistic images is extensively time consuming.

• Cost. Not only of computer equipment and all the Receptors, but also the process of filming, which essentially combines the costs of a live-action film with the costs of an animated film.

Role in Animation

In 2006 two of the three films nominated for the Best Animated Feature Oscar had been created using Mo-Cap: ImageMovers' Monster House and winner Happy Feet¹, with Pixar's Cars the non-mo-capped nominee. This led to animators claiming Mo-Cap was puppetry rather than true animation - the end credits to Pixar's 2007 film Ratatouille bore the message '100% Pure Animation – No Motion Capture or any other performance shortcuts were used in the production of this film'. Despite being a proponent of Mo-Cap and Performance Capture, Zemeckis agreed that the two are very different techniques, stating:

To call Performance Capture 'animation' is a disservice to the great animators.

The debate was settled in 2010 when the rules for qualifying for the Best Animated Feature Oscar were amended. Under this change it was stated that:

An animated film is defined as a motion picture in which movement and characters’ performances are created using a frame-by-frame technique... Motion capture and real-time puppetry are not by themselves animation techniques.

Studios

ImageMovers

Since 2000 ImageMovers, an independent film company that was created in 1997 by acclaimed Oscar-winning filmmaker Robert Zemeckis², has made a number of live-action films as well as pioneering Performance Capture animation techniques in five animated films. Despite some early success, its last animated film, Mars Needs Moms (2011), was a huge box office disaster.

ImageMovers' animation style utilised not only Mo-Cap but also the effective use of 3D. Their first three films were co-produced by various different studios, while the final two were made by ImageMovers Digital, a joint venture with the Walt Disney Company.

• ImageMovers:
1. The Polar Express (2004)
2. Monster House (2006)
3. Beowulf (2007)
• ImageMovers Digital:
1. A Christmas Carol (2009)
2. Mars Needs Moms (2011)

At the time of release The Polar Express was incredibly successful despite being the most expensive animated film made to date. However, the film was criticised for the 'plastic' appearance of its characters as well as the expressionless eyes. ImageMovers would work hard to improve this in later films. Monster House was nominated for a Best Animated Feature at the 79th Academy Awards. Like Who Framed Roger Rabbit it was executively produced by Robert Zemeckis and Steven Spielberg. Beowulf was the most complained about film of 2007 on both sides of the Atlantic, attracting criticism for the amount of horror, violence, nudity and sex in a film rated 12.

ImageMovers Digital had announced plans to make six animated films. However, four of the planned films were cancelled. These included a Mo-Cap remake of Yellow Submarine³, as approval to use 16 Beatles songs had been secured. Yellow Submarine had been scheduled for release during the 2012 London Olympics, as that was felt to be the ideal time to promote a British-themed film. It would have been followed by a Mo-Cap remake or sequel of Who Framed Roger Rabbit. Following the disastrous performance of Mars Needs Moms, Disney immediately severed all ties with ImageMovers and these projects were abandoned. Since 2011 ImageMovers still exists as Zemeckis' own production company, but it has only made live-action films.

Other Studios

Although ImageMovers is the animation studio that to date has most strongly embraced Mo-Cap, it is not the only one to do so, although on the whole results have been variable. India's low-budget Sinbad: Beyond the Veil of Mists (2000) dipped its toe in the water to be the first feature-length Mo-Cap film. Final Fantasy: The Spirits Within (2001) was the first photorealistic Mo-Cap film and the first to enjoy a widespread release; however, it was a box office bomb, recouping only half its budget. Although Happy Feet (2006), animated by Animal Logic, was a success, Happy Feet Two (2011), this time animated by Dr D Studios, flopped and lost over $40 million. The Adventures of Tintin: The Secret of the Unicorn (2011) was a successful film directed by Steven Spielberg.

Andy Serkis has so embraced the technology that he has founded a Performance Capture studio based in London called The Imaginarium. He uses Performance Capture techniques to create special effects characters, rather than to make animated films. Films that have benefited from his experience include the Planet of the Apes trilogy (2011-17), Avengers: Age of Ultron (2015) and Star Wars: Episode VII – The Force Awakens (2015). In 2012 he began an adaptation of The Jungle Book, only to be forced to postpone the release when Disney made a live action remake in 2016. In December 2017 Serkis announced his film would be renamed Mowgli to avoid confusion and it would be released in October 2018.