Motion stabilizing and tracking is one of the more technical processes in Adobe After Effects, and can be confusing for those who haven't worked with it before. There are a lot of options in the Motion Tracker panel, and even though it's largely automated once you get it set up, getting it set up can take some tweaking.
But the results can be stunning - you can stabilize an otherwise unwatchable bit of "shaky-cam" video into a rock-solid piece of usable footage, blur out perpetrators' faces in surveillance video or pixelate objectionable logos or body parts in reality programming, or create undetectable special effects.
My new series for MacProVideo, "Adobe After Effects 201 - Motion Stabilization and Tracking" shows you how to create just these kinds of effects in After Effects using AE's built-in motion tracking tools. What I'd like to do here is introduce you to some of the concepts and techniques I discuss in greater detail in the series to get you started with your own motion tracking projects.
The grandfather of motion tracking may well be Étienne-Jules Marey (1830-1904), a doctor and scientist who was among the world's first "chronophotographers" or photographers of time. Among many other devices, he invented a camera that could capture multiple images over time on a single photographic plate, and used it to study human motion. On this page, you can see an image of one of his motion studies, in which he dressed one of his assistants in a black suit with white lines, then took multiple shots of the assistant walking in front of his chronophotograph:
In particular, note the image with the caption "Correct Exposure...", in which you can clearly see just the assistant's motion abstracted from his body. If you were to connect the dots in that image, the resulting lines would look a lot like a successful motion track.
Here's a good resource for more about Marey
Later developments in motion tracking include "rotoscoping", the process of painting over or tracing live action images frame by frame onto film to create realistic animation. The rotoscope was invented by Max Fleischer around 1914 and used to create some of the world's first full-motion cartoons. Max Fleischer and his brother Dave went on to create the original "Popeye", "Betty Boop", and "Superman" cartoons, and the second fully-animated feature film, "Gulliver's Travels". This kind of rotoscoping was also used in special effects shots to add objects to existing filmed scenes, so that, for example, a character could pick up and carry a prop that wasn't available in the original scene - an early form of motion tracking. Rotoscoping is still used today in film and video production, primarily to remove actors and objects from their original backgrounds and place them in new environments.
These were all originally analog techniques, of course - the advent of digital image capture and processing led to an explosion of techniques for motion analysis and tracking, first for the military and law enforcement (intrusion detection, facial tracking and identification), and later for the entertainment industries, where motion tracking is used for special effects shots that place real actors in 3D-generated scenes. Most recently, increased computing power has led to new developments in real-time motion tracking for gaming, like the just-released Microsoft Kinect system.
All right, let's look at some of the details of motion tracking and stabilizing in After Effects, with some specific examples from my series.
First, motion stabilization works by tracking the location, in terms of X and Y pixel data, of an object in the image frame that does not move relative to the rest of the objects in the frame. What this means is that you can't expect to stabilize a video clip by tracking an object that is itself moving in the frame - a person's head, or a cloud, for example. You need to track an object that doesn't move - a chair or a fence, as in my examples. Once this unmoving object has been tracked, After Effects then exactly copies the X/Y data to the Anchor Point of the layer being stabilized, not its Position. Copying the tracked data to the layer's Anchor Point has the effect of moving the layer in the opposite direction from the relative motion of the tracked object, which essentially cancels out the motion of the tracked object - for every pixel the tracked object moves to the right, relative to the frame, the layer moves a pixel to the left, so the tracked object stays in the same location, again relative to the frame.
So, for example, here's some original footage shot on my deck last summer of a young ruby-throated hummingbird. I shot this with an HDV camera, and had to zoom way in to get a good shot of the bird perched on our anti-deer fence. Since I was hand-holding the camera, that combined with the zoom made for some pretty shaky footage:
I brought that into After Effects, set up a Stabilize Tracker on the footage, and tracked a section of the top of the fence near the hummingbird that didn't move - being a solid fence, it didn't move relative to the rest of the world in the video frame. After tracking and applying the tracking data to the video clip, this is what I got:
Now, you'll notice that this shot has been zoomed in a bit from the original, and is slightly softer than the source footage - that's because the motion of the video clip relative to the frame meant that I saw the edges of the video itself in the After Effects composition, resulting in moving black bars around the stabilized footage. One of the ways to deal with that is to scale or enlarge the stabilized footage, as well as re-positioning it, so that you no longer see those edges. This is an inevitable result of this kind of motion stabilizing, but there are some ways to improve that - for example, you can use higher-resolution footage to begin with, so that you don't need to scale as much when you compensate for the stabilization, as I did in this version of the shot:
You'll notice, though, that both of the stabilized shots still have a little rotation in them, where I twisted the camera slightly while holding it. Rotation stabilizing is a little different from position stabilizing - with position stabilizing, you just need to compensate for motion in the X and Y dimensions, that is, horizontal and vertical. Compensating for rotation means that you have to add a third dimension of stabilizing to remove the twisting in the footage. Fortunately, After Effects can do that too.
However, when you choose to stabilize the rotation of a layer, After Effects operates a little differently from straight position stabilizing. The Motion Tracker calculates the relative rotation of the tracked objects in the frame in degrees - objects, in this case, because stabilizing rotation requires tracking two features in the frame. When you Apply the tracked rotation to the layer, After Effects inverts the rotation values and applies them to the layer's Rotation property - that is, if the tracked object rotates +30 degrees, After Effects applies a value of -30 degrees to the Rotation property. If the object rotates -30 degrees, a value of +30 degrees is applied to Rotation. The effect is the same: the tracked object's rotation is canceled out, and the image in the frame appears to be motionless, as in this example:
Another common use for motion tracking is something you see constantly in reality TV shows - blurring out the faces of "alleged" perpetrators in cop shows, concealing nudity, and obscuring the logos of products the producers don't want to license. We can do that in After Effects too, by tracking the head of our hummingbird using a similar technique to the Stabilize tracking. In this case, though, we don't apply the tracking data to the position of the layer to stabilize it - instead, we take that data and use it to track a pixel mosaic effect onto the face of our perpetrator hummingbird:
Or, by simply swapping out the Mosaic effect for a Gaussian Blur, we can blur his face instead:
Once you know how, you can easily adapt these techniques to obscure anything you need to in complex motion footage.
Another common special effects application of motion tracking is sign replacement. A typical example would be a shot in which you have a bus moving through your scene, and you want to replace the sign on the side of the bus with a sign advertising one of your own sponsors instead. Or you might want to change the text in a road sign for comic effect in your low-budget indie feature. This kind of motion tracking is a little more complex, but easily doable in After Effects. It depends on tracking three or four points in the image, then using that tracking data to distort another image you've added to the shot, a process called multi-point tracking and corner pinning.
In my example, in order to increase the realism of the shot, I first show you how to stabilize the image to make the four-point tracking easier, then how to set up the four-point track and corner pin effect itself. Once you've done that, I show you how to de-stabilize the image again to make it seem much more realistic - with a little work, you can create a seamless special effects shot, going from this:
Use this as a cutaway shot in your indie film, and you'll end up with a completely undetectable special effects shot with just a few minutes work.
So, I hope this introduction to my Motion Stabilization and Tracking series has piqued your interest - please check out the full tutorial here, and use what you learn there for your own special effects adventures.