Dev:Ref/Release Notes/2.64/Motion Tracker

Blender 2.64: Motion Tracker

Dopesheet View

Dopesheet View in movie clip editor

A dopesheet view was added for motion tracking data, implemented as separate view of the movie clip editor just like the curves view. (r46123)

To support this in a nice way, the behaviour of view switching in the clip editor was changed, so that toggling between modes opens specified a view in the whole area of the movie clip editor. Hence, to display a curve or dopesheet view, the editor must be split into two, with one switched to the curve or dopesheet view.

To make this setup more comfortable to work with, a new screen layout "Motion Tracking" is also available in the default configuration. It contains a standard movie clip editor, curves and dopesheet view already open and ready to use.

Usage

Currently the dopesheet view is for visualization and does not have any tools to actually edit data. It displays channels for selected tracks and each channel visualizes tracked segments of tracks as dark bars and keyframed positions of tracks as small diamonds.

By default, this view sorts tracks in alphabetical order, but here's list of all possible sort orders with the Sort Order option:

• Name: sort selected tracks in alphabetical order based on their names.
• Longest: sort tracks by longest tracked segment length.
• Total: sort tracks by overall amount of frames.
• Average Error: sort tracks by their average reprojection error after solving camera or object motion.

Channels Sort Order

There's also an option called Invert to change the sort order from ascending to descending, next to the sort order method option.

Planar Tracking

The biggest change in motion tracking in this release is the new planar tracker. The planar tracker is a complete rewrite of the old tracking system that leverages Ceres Solver for the subpixel planar refinement. The new tracker supports new motion models and so can track more challenging scenes. Planar tracking takes advantage of the fact that there are often planar surfaces in footage, by attaching markers to points on these flat planes. This provides more information about the possible deformation of the marker in following frames, and such markers can be tracked even if partially occluded. The new tracker also supports illumination-compensation to deal with markers that are shadowed partway through the footage.

Marker Shape

Corners slide zones of marker

Defining a pattern's shape and size now works differently. Now each marker corner can deform individually to define the shapes needed for planar tracking. Corner positions can be edited by dragging them with a mouse. Dragging with LMB will change the position of an individual corner. If during a drag, Ctrl is held down, the whole pattern will be scaled instead of just one corner.

The same scaling effect can be achieved by pressing S twice; this will switch scaling to scaling the pattern area only (single S scales the whole marker as before). The Pattern can also be rotated using the R button which, depending on the used pivot point, will either rotate patterns around their own centers or rotate the whole markers around the median point.

To match the perspective transformation of a marker on a plane, the individual corners must be edited manually.

Note that deforming a pattern is not only useful for planar / affine tracking. Since only pixels within the pattern will be considered this can help to specify a better pattern to track even for simple position tracking.

The Search area can not be rotated; this is intentional. It doesn't make sense to deform the search area.

Tracking Settings

Corner drag zones of a marker

The Tracking settings panel was almost completely changed. The old tracking-algorithm related options were removed and new options were added.

First, the old option called Tracker was eliminated. The old Hybrid tracker was doing exactly the same as the SAD tracker, with the exception that it could do more accurate subpixel precision tracking by following the brute-force search with a subpixel refinement. The SAD tracker was also limited in that it only worked on 8-bit images and is unsuitable for 32-bit float images like those used in Mango.

Secondly, the KLT tracker with its pyramid tracking helped tracking blurry footage, but could be quite useless when the background texture was moving in the opposite direction of the feature point. In such cases the Hybrid tracker was much better than KLT, and for this reason the KLT tracker was made obsolete. Options for tracking of blurry footage might be added to the Hybrid tracker later.

The new tracker is a unified algorithm with more options, and specifically some options to make it work exactly the same as the old non-planar Hybrid tracker. The options are:

• Motion Model defines which possible motions tracking feature has. Currently supported models are: Location only, Location+Rotation, Location+Scale, Location+Rotation+Scale, Affine and Perspective. This option should be set depending on which motion a particular feature has and it'll make tracking most accurate for such a motion. Perspective is usually used to track a planar feature, but often Affine is a good enough approximation and may have more stable tracks.
• Prepass enables a two pass tracking, where the first pass is a brute force tracking of location only, and the second pass will use tracking of the full motion model refining the first pass. To get behavior that is almost identical to the Hybrid tracker in previous Blender versions, enable Prepass and set the motion model to Location only.
• Normalize means patterns will be normalized by their average intensity while tracking, to make them invariant to illumination changes. An example where this is useful is a scene where a marker moves in the shadow of an actress. Previously it would not be possible to track such a situation, scale adaption would shrink the area to compensate for the change in illumination, losing the track. With "Normalize" turned on, the patch is correctly tracked and scale is maintained.
• Correlation is now a single value for all tracking settings and defines the minimal correlation between a matched pattern and a reference to be considered a successful tracking. If the tracker is giving up too easily, decrease this value, or if the tracker is slipping too much when it should give up sooner, increase this value.

Here is a video that shows the new planar tracking in action. Note that the final version has a better preview window and runs faster than what's in the video. The footage used in this video is from the excellent collection of Hollywood VFX plates.

Smaller features

Grease Pencil

Grease pencil can now be attached to a track datablock, so each individual track can have its own grease pencil datablock. This was added to support features like masking while tracking, so some areas of a pattern could be masked away. Such masking is really helpful when tracking some difficult features.

Currently it's possible to display grease pencil for either the clip or the associated active track, not both at the same time.

Grease Pencil options

A toggle between clip and track was added in clip editor. As soon as it's switched to "Track" all further strokes will be added to the track datablock and they'll be relative to this track. This means if track the is being moved/tracked around the frame, grease pencil strokes will follow this track. Different tracks can also share the same grease pencil datablock, which is extremely useful for masking, a mask can be drawn once and used for all tracks that are tracking features with the same shape.

Start Frame and Frame Offset

Start Frame option

A Start Frame property was added to all movie clips datablocks, which can be accessed in the Footage Settings panel in the movie clip editor. It indicates at which scene frame number the clip starts to play.

This is a global setting which means it affects all clip users such as the clip editor itself, motion tracking constraints and compositor nodes.

A Frame Offset option affects the footage itself and defines an extra offset which is added to frame number when converting a scene frame to the frame number in the file name. This option does not affect tracking data or any other associated data.

This option was added to support features such as processing a clip in Blender with the compositor, rendering it and replacing the clip datablock with a new rendered clip which is not necessarily rendered for the same frame range as the original. But it could be used in other ways such as cases when one scene contains several different motions and several clips should be used for solving/background.

Tripod Motion Solving

Tripod motion solver option

There's a new solver option called Tripod Motion. It can be used for footage where the camera does not move and only rotates. Such footage can't be tracked with a generic solver approach, and it's impossible to determine the actual feature points in space due to a lack of information. So this solver will solve only the relative camera rotation and then reproject the feature points into a sphere, with the same distance between feature and camera for all feature points.

Other Features

• Add file image and movie file drag-n-drop from external applications to movie clip editor (r46198)
• Add pivot point for transformation tools in movie clip editor. Supported modes: Median Point, Individual Centers and Boundbox Center.
• Add Lasso select for movie clip markers. (r46622)
• Display default grid image with size of movie clip if the current frame is missing (r46030)
• Use texture buffers to display clip frame, for faster performance. (r46122)
• Bug fixes for tracking and proxy generation of float images. (r46453), (r46944))