One of Adobe After Effects' most important functions in film and video production is compositing. This is the combining of separate visual elements to create single integrated imagery, often for the purpose of placing actors in environments that are too difficult or expensive to film, or that simply don't exist in the real world, as in the case of countless science-fiction and fantasy films. After Effects provides a full suite of masking, matting, keying, and rotoscoping tools for digital compositing - but have you ever wondered where those odd terms come from?
The terms we're using so casually here for our digital tools - key, matte, bluescreen - all have their origins in the analog history of film and video production. A matte originally referred to a matte painting, a scene painted on a relatively small canvas or piece of glass by an artist to represent (usually) the background in an effects shot. It was called a matte painting because it was painted with non-reflective, or matte, paint - this non-glossy paint wouldn't reflect the studio lights and give away the effect. Fritz Lang's 1927 classic Metropolis, for example, contains many famous matte paintings and effects shots - here's an example:
(You can see many more of these shots in the re-release trailer, available here)
So how would an effects artist in the pre-digital era make a 10-inch-square painting look like an entire full-sized city behind a character in the foreground?
One of the only ways to create a special effects shot in the old days was by double-exposing film. (Rear projection was another option, but had its own set of problems and limitations.) This particular type of shot, where a character is keyed in front of a matte painting, was usually done like this: first, the matte painting, a cityscape let's say, would be shot close up, but with a slight defocusing to suggest distance and size. Next, shooting again on the already-exposed film, the character would be shot on a sound stage through a keyhole cut in a matte box. The matte box was simply a small black box that fit around the camera lens; the keyhole, a small hole cut through the bottom of the box roughly in the shape of the character, hence the keyhole shape. In the second exposure, only the light coming in through the keyhole would be captured on film - in this case, our character, surrounded by a soft edge because of the proximity of the matte box to the lens. The rest of the film wouldn't get any light, preserving the first exposure, and so in the final print, if all the proportions, framing, focus, and exposures were just right, the character would appear to be in front of the cityscape.
As you can imagine, this required a lot of trial and error, not to mention having to wait for the film to be developed just to see if the shot worked or not. However, in the right hands, it could be highly effective, as in one of the earliest examples of multiple-exposure special effects work, Georges Méliès' L'Homme orchestre, from 1900, in which Méliès creates a seven-fold-exposure special effects shot:
Figure 2 - Méliès
(You can see the entire film here)
Another famous example of this type of multiple-exposure shot (and probably inspired by Méliès) is Buster Keaton's The Playhouse, from 1921, in which Keaton plays all the members of the orchestra:
Figure 3 - Keaton
(See the complete film here)
Over time, keyhole was shortened to key, and matte painting to simply matte, and there you have it. Nowadays, "keying" generally refers to any kind of technique for removing backgrounds from foregrounds, short of rotoscoping, which we'll discuss below. "Matte" has come to mean almost any kind of image used with, or generated by, keying or rotoscoping techniques, whether it's a background plate, a high-con, or an alpha channel.
However, you can, no doubt, see an immediate limitation in the keyhole/matte box technique - the camera, foreground, and background all have to be static. Careful planning might allow for simple panning, but anything more complex was simply too difficult. The need to work with moving elements and traveling, or moving, mattes, soon led to the use of rotoscoping, or tracing mattes directly on film. Rotoscoping is, generally speaking, the process of painting over or tracing live action images frame by frame on film to create realistic animation or traveling mattes.
The rotoscope was invented by Max Fleischer around 1914 and used to create some of the world's first full-motion cartoons, as in Fleischer's Cartoon Factory from 1924:
Figure 4 - Koko
Max Fleischer and his brother Dave went on to create the original Popeye, Betty Boop, and Superman cartoons, and the second fully-animated feature film, Gulliver's Travels. Rotoscoping is considered to be the first instance of motion capture used for animation. Nowadays, digital rotoscoping has replaced film roto, and is almost exclusively used to create traveling mattes for live-action characters.
However, rotoscoping, while effective, is incredibly labor-intensive. The need for faster and more efficient matting techniques led to the development of the optical color-keying technique commonly called chromakeying or bluescreening. With optical bluescreen keying, the foreground element (our hero) was shot against a seamless blue backdrop, although other colors were used occasionally. Then the background element was shot on a separate piece of film. After that, the hero footage was optically reprinted to another piece of film through a red filter that removed all the blue hue in the scene and replaced it with black. Usually, this filtered image was printed to high-contrast black and white film, resulting in a matte consisting of a clear outline in the shape of the hero on a black background. Finally, all three pieces of film were optically printed in layers to a new piece of film, with the background first, the black and white matte next, and the hero footage last. The hero footage was printed through the clear outline onto the background, while the black matte area prevented the blue from printing onto the final composite. This technique allowed for motion composites, although too much motion would result in a lot of motion blur in the foreground image and consequently a difficult key.
(As an aside, one particular bluescreen term that you may have heard of and been puzzled by is garbage matte. This rather peculiar name originated on the early bluescreen stages, where there would often be a lot of extraneous stuff - cables, light stands, lawyers - at the edges of the bluescreen set that wasn't painted blue and so couldn't be automatically removed, or keyed, from the shot during post-production. However, since this stuff, or garbage, was always at the edges of the frame and well away from the principal action, a simple non-moving matte - basically, a piece of blue cardboard with a hole cut into it - could be optically printed onto the film to mask the garbage at the edges of the frame with blue and allow the bluescreen compositing process to work correctly. Nowadays, garbage mattes are created digitally with masking tools in compositing software, but the old term lives on.)
Among the first feature films to use the color key technique was Michael Powell's 1940 The Thief of Bagdad where it was used for many of that film's classic special effects shots:
(The complete film is viewable here)
RKO studio technician Larry Butler is credited with developing the color key technique in the 1930s, and subsequently enhanced the technology to work with the Technicolor film process. Butler won an Academy Award for his visual effects work on The Thief of Bagdad.
Many other artists, engineers and technicians contributed to the development of our current keying technologies, but among the premiere names is Petro Vlahos. Vlahos refined the color keying process in the 1960s by developing the color difference matte technique, for which he won an Oscar in 1964. He went on to pioneer analog electronic keying with the Ultimatte process in the 1970s and later founded the Ultimatte company, one of the best-known producers of digital keying hardware and software.
These days, color keying is ubiquitous in films, television, and commercials, along with all the other digital compositing techniques whose history we've discussed here. I hope you've found it both interesting and educational. For those who'd like to learn more about practical compositing techniques in After Effects, take a look at my Core After Effects series, particularly Chapter 2, in which I discuss color and luma keying, rotoscoping and masking, blending modes, and track mattes. Thanks!