What I did this week:
- Complete the multi-view reconstruction pipeline and pass the results from libmv to Blender. Now we are able to see the visualization results of multiple camera motion in Blender 3D view. Specifically, this is done by multiview_reconstruct_retrieve_libmv_tracks function.
- Fixing the correspondence saving/loading error. I found that correspondences couldn’t get loaded successfully and discussed this with my mentor (Sergey), who helped me solve the pointer error. Thanks go to him again! It appeared that ID pointers are to be restored using lib pointer map.
- Change the multi-view pipeline a bit to cope with single-view solver. Since we may remove single-view solver in the future, the multi-view solve button will revert to the single-view case if there are not enough correspondences between cameras.
- Fixing the multiview reconstruction bug, which previously got a huge re-projection error. This is a substantial progress and we can now get robust results of multi-view camera reconstruction. The re-projection error can be as low as sub-pixel level. It appeared that the previous bug is caused by using normalized track for intrinsics refinement.
- A bit of code formatting. move two functions with BKE_ prefix to blender kernel and revise the code to conform to blender coding style.
What I will do next:
- Now that a basic multi-view solver is finished, I would like to solicit everyone who is interested in this project to test the pipeline and report bugs and suggestions. The user interface (see the pic below) is crude for now and I will improve it during the next couple of days. But the basic functions should be good to go. Specifically, the pipeline works as follows:
1. Open two video clips (one primary camera and one witness camera),
specify some markers and get them tracked.
2. Link correspondences using the Correspondence panel in the PRIMARY video clip.
Correspondences should be more than 8 to make the multi-view solver work smoothly.
3. Click ‘Solve Multiview Camera’ button to reconstruct multiple cameras simultaneously.
- There are some ‘TODO’s left in the code here and there during my first implementation. I would like to finish the left components and improve the pipeline next week.
- Half of the project involves a decent user interface to work with multiple clips conveniently. As Sergey has pointed out, this is also the part that is undecided and open-ended. I will work on the design of the user interface in the next few weeks. At the mean time, I will continue to improve the pipeline and refactor the code.
