Moving Shot Stabilization
A little while ago I wrote about Video Stabilization. All examples there had one thing in common - there were no panning in the clips. This was for a very simple reason - since the technique was based on identifying reference points and then fixing those points on a specified pixel location, there was no way to handle large panning movements. If we, for example, fix the feature in the center of the first frame, then any panning will be limited to having that feature in the frame. Worse, since the stabilized output should only contain the parts of the video that all input frames have in common (otherwise you get black borders as the frames are translated and rotated to match up), even with relatively small pans we end up using only a small region around the center of the frames.
For my next project I wanted to do some long takes and speed them up. First I thought I could just speed up the video, but that resulted in little more than a couple of seconds of impressionist blur. The problem was then how to choose features for stabilization. After a few false starts I ended up with the following:
Pick the feature in the center of the first frame.
Select a maximum distance, maxDistance, from that point. I chose 64 pixels. Draw an imaginary box around the center of the image with side length = 2 x the maximum distance.
If the feature we're tracking moves outside of the box, select the point in the box that is nearest. Pretend that the feature was found there, and use whatever is there as the next feature to track.
The result is that any shake smaller than maxDistance pixels end up compensated for. Movements larger than that are not compensated for, which means that we can do pans as we see fit.
Here's a sample time lapse video (sped up by a factor of 12) of me walking from my door to the north point of Centralbron. The left half is the unstabilized video, and the right half the stabilized. For a good view of the shakes involved, just pause the video when a streetlight is in the picture.