利用者:Sftrabbit/GSoC 2013/Documentation/Modal Reconstruction

Modal Reconstruction

Modal reconstruction assumes that the camera does not undergo any translation. This occurs if, for example, the footage was shot on a tripod. Because of these restricted requirements, modal solving can be performed with a specialized algorithm that is faster than a normal reconstruction. This algorithm provides an accurate reconstruction for the orientation of the camera, but, due to the way it works, does not give accurate bundles (the 3D positions corresponding to tracked features in the image). In fact, the resulting bundles are projected on to a sphere around the camera.

The process of modal reconstruction can be summarised as follows:

1. Reconstruct for all frames using normalized tracks (i.e. camera distortion removed)
   1. Begin with a 0 rotation R
   2. For each image (frame of footage):
      1. If we have reconstructed any points yet:
         1. Project markers in image on to a sphere to get one set of points
         2. Rotate points reconstructed from the previous image with R to get second set of points
         3. Compute estimate of Δ rotation to get from previous set of points to new set of points
         4. Add Δ to current R
         5. Refine R with Ceres (minimise projection error for all markers in current image)
      2. Add camera with rotation R to the reconstruction
      3. For any markers in this image without reconstructed points:
         1. Project those markers on to a sphere and add them to the reconstruction
2. Perform modal bundle adjustment
   1. Adjust camera orientation with Ceres (minimise projection error for all markers in all images)
3. Refine camera intrinsics
   1. Refine all camera pose data and camera intrinsics (if selected for refinement) using Ceres (minimise projection error for all markers in all images)

How this is incorporated into multiview reconstruction depends on the algorithm chosen. Ideally, the user should be able to provide multiple clips of the same scene, specify camera data for each one (including whether the camera was on a tripod or not) and then perform a reconstruction. The above algorithm may be used in whole or in part for this. For example, the multiview reconstruction algorithm may begin by reconstructing for each view independently.