Yamyam: 1版をインポートしました

2018-06-28T21:04:26Z

1版をインポートしました

wiki>Sftrabbit: /* Tracks storage */

2013-09-23T20:20:49Z

‎Tracks storage

新規ページ

=Technical Overview=

==Focal length constraints==

===DNA===

The relevant DNA types are found in <code>source/blender/makesdna/DNA_tracking_types.h</code>.

The <code>MovieTrackingSettings</code> DNA struct contains the following relevant properties:
* <code>refine_intrinsics</code> - bit flags identifying intrinsics to refine. Some combination of <code>REFINE_FOCAL_LENGTH</code>, <code>REFINE_PRINCIPAL_POINT</code>, <code>REFINE_RADIAL_DISTORTION_K1</code>, and <code>REFINE_RADIAL_DISTORTION_K2</code>. Only some combinations are currently supported.
* <code>constrain_intrinsics</code> - bit flags identifying which intrinsics to refine. Currently, the only valid flag is <code>CONSTRAIN_FOCAL_LENGTH</code>. This could potentially expand in the future.
* <code>focal_length_{min/max}</code> - these are the values that define the constraint range. They are stored in pixel units.

===RNA===

The relevant RNA structs are found in <code>source/blender/makesrna/intern/rna_tracking.c</code>.

The following properties have been modified or added:
* <code>refine_intrinsics</code> - corresponds directly to the <code>refine_intrinsics</code> in DNA
* <code>use_focal_length_constraint</code> - a boolean property that sets the <code>CONSTRAIN_FOCAL_LENGTH</code> flag in DNA.
* <code>focal_length_{min/max}_pixels</code> - corresponds to <code>focal_length_{min/max}</code> in DNA. The minimum must always be less than the maximum (see <code>rna_trackingSettings_focal_length_{min/max}_pixels_set</code>).
* <code>focal_length_{min/max}</code> - used for setting the focal length in millimeters. The conversion to and from pixels is performed by <code>rna_trackingSettings_focal_length_{min/max}_set</code> and <code>rna_trackingSettings_focal_length_{min/max}_get</code>.

===User Interface===

The user interface is implemented in <code>release/scripts/startup/bl_ui/space_clip.py</code>.

The constraints fields of the {{Literal|Solve}} panel are enabled when any option from the {{Literal|Refine}} select box containing "Focal Length" is chosen. This is determined by checking if <code>"FOCAL_LENGTH"</code> is a substring of the value of <code>settings.refine_intrinsics</code>. The minimum and maximum fields are then only enabled when <code>settings.use_focal_length_constraint</code> is <code>true</code>.

If <code>tracking.camera.units</code> is set to <code>'MILLIMETERS'</code> (in the {{Literal|Camera Data}} panel), the minimum and maximum fields correspond to the <code>focal_length_{min/max}</code> RNA properties. Otherwise, they correspond to the <code>focal_length_{min/max}_pixels</code> RNA properties.

[[File:focal-length-constraint.png|none|frame|Applying a focal length constraint.]]

===Kernel===

A number of changes have been made to <code>source/blender/blenkernel/intern/tracking.c</code>. First of all, the function <code>BKE_tracking_camera_focal_length_set</code> has been added to ensure that the constraint values update to match the given focal length when the constraint is disabled

The <code>reconstruct_refine_intrinsics_get_flags</code> function converts from the DNA properties to <code>MovieReconstructContext</code> properties, which are similar their DNA counterparts but take values from the libmv API (see below).

When the reconstruction job begins, <code>BKE_tracking_reconstruction_solve</code> calls both <code>cameraIntrinsicsOptionsFromContext</code> and <code>reconstructionOptionsFromContext</code> to convert these context properties to the appropriate libmv options structs. The former applies the <code>CLAMP</code> macro to the focal length to ensure it is not outside the constraint boundaries.

===Libmv API===

Constraint data is passed to libmv in a <code>libmv_ReconstructionOptions</code> object. This struct is defined in <code>extern/libmv/libmv-capi.h</code>. It now has the following properties that currently mimic the DNA properties (but there is no need for them to be identical):

* <code>refine_intrinsics</code> - some combination of <code>LIBMV_REFINE_FOCAL_LENGTH</code>, <code>LIBMV_REFINE_PRINCIPAL_POINT</code>, <code>LIBMV_REFINE_RADIAL_DISTORTION_K1</code>, and <code>LIBMV_REFINE_RADIAL_DISTORTION_K2</code>.
* <code>constrain_intrinsics</code> - currently the only accepted value is <code>LIBMV_CONSTRAIN_FOCAL_LENGTH</code>
* <code>focal_length_{min/max}</code> - constraint values.

===Libmv internals===

The constraint properties primarily affect bundle adjustment. Previously, only a single option was passed to the bundle adjustment algorithm as an argument. Now that this has grown, a <code>BundleOptions</code> class type has been introduced in <code>extern/libmv/libmv/simple_pipeline/bundle.h</code>. These options do not all take the same form as the properties passed into libmv, however:

* <code>bundle_intrinsics</code> - flags specifying which intrinsics to include in bundle adjustment. See the <code>BundleIntrinsics</code> enum for a list of flags.
* <code>constraints</code> - flags specifying abstract constraints to apply during bundle adjustment. Unlike previous properties, these constraints may not refer to a particular intrinsic. Currently, the accepted values are <code>BUNDLE_NO_CONSTRAINTS</code>, <code>BUNDLE_NO_TRANSLATION</code>, <code>BUNDLE_CONSTRAIN_FOCAL_LENGTH</code>.
* <code>focal_length_{min,max}</code> - as before, these are the constraint values to apply to the focal length under adjustment.

Libmv uses Ceres Solver for bundle adjustment by minimizing the reprojection error. Ceres is an unconstrained minimizer, however, so it will adjust the function parameters without bounds. To apply a bound to the focal length, the <code>ConstrainFocalLength</code> function applies a transformation to the focal length parameter (if the appropriate options have been set). This transformation is simply <code>arcsin</code> scaled and translated such that the upper and lower limit of its result are the constraint bounds. The transformed variable will be unconstrained when minimised, but will always give a focal length within the bounds when transformed back using <code>UnconstrainFocalLength</code>.

constrain(f, min, max) = arcsin(-1 + ((f - min) / (max - min)) * 2)
unconstrain(f, min, max) = min + (max - min) * (1 + sin(f)) / 2

----

==Multicamera reconstruction==

===Libmv API===

The libmv API for multicamera reconstruction is almost identical to as it was before. One small change is that the camera intrinsics argument to <code>libmv_solve</code> is an array, where each set of intrinsics applies to a single camera. However, since array type arguments in C are transformed to pointer types, this is merely a semantic difference. Instead of passing a pointer to a single <code>libmv_CameraIntrinsicsOptions</code> object, the client passes a pointer to the first element of an array of <code>libmv_CameraIntrinsicsOptions</code> objects. The elements each correspond to a different camera.

===Tracks storage===

The <code>Tracks</code> class stores <code>Markers</code> which correspond to points in the images that libmv uses to reconstruct the scene. <code>Markers</code>, prior to my changes, had two important members: <code>track</code>, which identifies the track that the marker belongs to; and <code>image</code>, which identifies the image that the marker corresponds to. To accommodate multicamera reconstruction, a marker also needs to be associated with a particular camera.

My first approach was to have a <code>camera</code> member to identify a camera, and then the <code>image</code> member would identify which frame of the camera that marker is associated with. So two markers could have the same <code>image</code> but could actually be associated with different frames from different <code>camera</code>s. This, however, meant many changes to the way the libmv algorithms worked. It also interprets the <code>image</code> identifier as "frame", although libmv doesn't actually concern itself with the order of images in a video.

Instead, I took a different approach, in which the <code>image</code> identifier is unique for all frames from all cameras. That is, the first frame of one camera and the first frame of another would have different <code>image</code> identifiers. A new <code>camera</code identifier would then be used to annotate the markers to associate them with a particular camera. All markers with a specific <code>image</code> identifier should also have the same <code>camera</code> identifier.

<source lang="cpp">
struct Marker {
int image;
int track;
double x, y;
int camera;
}
</source>

This works well because libmv only needs to keep track of which cameras the markers belong to. This information isn't actually used in reconstruction. The algorithms only need to receive a bunch of markers from images - they don't even need to be from frames of a video. When reconstruction is complete, libmv can use these camera identifiers to pass the reconstructed camera orientations back to the client.

The <code>Tracks</code> class has a number of member functions that provide access to <code>Marker</code>s based on their associated <code>camera</code>. For example, <code>MarkersForCamera</code> will return all the markers associated with a particular camera.

===Reconstruction storage===

Similarly, the reconstructed camera positions/orientations have associated physical cameras. That is, the <code>EuclideanView</code> reconstructed from markers with a specific <code>camera</code> identifier will have the same <code>camera</code> identifier. This allows the client the understand which of the reconstructed views correspond to which cameras and frames. <code>AllViewsForCamera</code> can be used to get all views reconstructed for a particular camera.

===Libmv internals===

All of the internal algorithms needed to be updated to be multicamera-aware. For those algorithms that rely on having images from a single camera (such as the two frame reconstruction initialization), this simply meant extracting tracks for the first camera and using only those.

For bundle adjustment, the multiple sets of camera intrinsics passes into libmv need to be used for calculating the reprojection error for each marker. This works by adding the appropriate intrinsics to the parameters of each marker's cost function according to its <code>camera</code> identifier:

<source lang="cpp">
problem.AddResidualBlock(new ceres::AutoDiffCostFunction<
OpenCVReprojectionError, 2, 8, 6, 3>(cost_function),
NULL,
&ceres_intrinsics[camera](0),
current_view_R_t,
&point->X(0));
</source>

The <code>OpenCVReprojectionError</code> cost functor then uses these intrinsics to compute the reprojection error for each marker.

Whenever an algorithm produces a reconstructed camera view, it adds it to the <code>EuclideanReconstruction</code> object and associates it with the appropriate camera.

===Terminology===

The following terminology was decided on to avoid confusing clashes of names:

* View - a specific instance of a camera at some orientation and point in space (giving an image of the scene)
* Multiview reconstruction - reconstruction of a scene from multiple views
* Camera - a physical video camera
* Multicamera reconstruction - a form of multiview reconstruction, where the views are associated with some number of video cameras

Previously, views were known as cameras, which meant that a <code>EuclideanReconstruction</code> contained <code>EuclideanCamera</code>s which were each associated with a particular physical video camera. Now, a <code>EuclideanReconstruction</code> contains <code>EuclideanView</code>s instead.

----

==Code clean-up==

===Unified libmv API===

While studying the libmv source, I noticed that <code>libmv_solveReconstruction</code> and <code>libmv_solveModal</code> had a lot of code in common. They were each part of the libmv C API and would be called based on whether the client code wanted to perform a modal solve or not. I thought a better approach would be to have a single libmv_solve function that will use the modal solver when a certain option is passed.

To this end, <code>libmv_ReconstructionOptions</code> now has a <code>motion_flag</code> option that may have the <code>LIBMV_TRACKING_MOTION_MODAL</code> flag set. If it is set, the modal solver will be used. Otherwise, normal reconstruction will take place.

This cleans up the libmv interface by moving the decision of whether to perform a modal solve or not into libmv and also makes clear the steps that are common between both solvers.

===C/C++ interface consistency===

I also noticed that these was a lot of inconsistency between naming conventions within the libmv C API. This is probably caused by the fact that it's an interface between C and C++, which each have their own code styles, and there's no real guideline for this situation. To fix this, the following rules were made:

* All identifiers exposed by the API begin with the library name followed by an underscore. In this case, <code>libmv_</code>.
* Types are in camel-case and begin with a capital letter. For example, <code>libmv_CameraIntrinsicsOptions</code>.
* Functions are in camel-case and begin with a lower-case letter. For example, <code>libmv_cameraIntrinsicsInvert</code>.
* Types that are exposed by the API should be typedefed '''unless''' only a declaration is provided in the API. That is, if the definition of the type is hidden within the library, the type should not be typedefed.

← 古い版	2018年6月28日 (木) 21:04時点における版
(相違点なし)

利用者:Sftrabbit/GSoC 2013/Documentation/Technical Overview - 版の履歴

Yamyam: 1版 をインポートしました

wiki>Sftrabbit: /* Tracks storage */

Yamyam: 1版をインポートしました