Google Summer of Code 2008: Freestyle integration
MIDTERM REPORT
Maxime Curioni
To get an idea of the work I have accomplished so far, I’ll go over the original project proposal and describe what I did and did not implement, and what I need to do for the remaining of the summer. I’ll also present some of the challenges that will have to be met to complete the project.
Completed Work
Legend:
| FINISHED
|
1) Freestyle as an external renderer
|
Schedule: May 26 - Jun 8
|
| Freestyle execution directly within Blender
|
| -
|
results produced by the original program and by Blender are identical.
|
| -
|
the scene is converted to 3ds using a tweaked version of Blender’s internal 3ds export script. lib3ds is used as an external library to import that temporary 3ds file within Freestyle.
|
| -
|
SWIG is used to compile the wrapper library used to execute Python scripts. This library is unfortunately added manually following instructions in Freestyle’ SConscript file (the instructions are platform-dependent).
|
| Qt’s dependencies removal
|
| -
|
we can’t consider the source totally Qt-free, considering that some of liqQGLWidget’s classes are used to provide a camera functionality (essential for later OpenGL commands).
|
| Drawing components isolation
|
| -
|
rendering is achieved programmatically without any UI interaction.
|
| -
|
the process to produce the rendering is: initialize the controller and view; set up the camera; export the scene to the temporary folder and load it into the controller; add a style module; calculate the view map; build up the strokes strips; raster the final result in the backbuffer (see source/freestyle/intern/app_blender/api.cpp).
|
| -
|
the result is copied back to the render window by copying (with glReadPixels) the back buffer into a RenderResult, eventually given for display to Blender.
|
| -
|
the result can be saved to disk as usual (using the F3 keystroke for example), independently of Freestyle’s original saving functions.
|
| -
|
rendering only works for the ‘Render window’ target (bogus for ‘Image editor’).
|
| -
|
instead of using an external interpreter, I directly used the one within Blender
|
| Support for Blender’s native geometric data
|
| -
|
I only use the first proposed solution (using the internal 3ds exporter)
|
| -
|
I skip the conversion from Freestyle’s model format to use the native Blender data format (render database). This is dealt with in the following phase
|
| -
|
the rendering style cannot be modified yet (for the mean time, hard-coded as the ‘contour’ style). The style module will be changeable from the UI by specifying its path.
|
| -
|
objects need to be manually selected before using the renderer. It could crash if the scene is empty or if “non-viewable” objects (like the camera or the lamp) are selected. This drawback is due to the way the export script works.
|
| -
|
the camera settings used for rendering are different from the ones from those from Blender: the camera uses by default perspective projection with a 45° horizontal field of view.
|
| CURRENT
|
2) Render layer for NPR line drawing
|
Schedule: Jun 9 - Jun 29
|
| Integration with Blender’s internal rendering engine
|
| -
|
a ‘Freestyle’ UI button is added to enable/disable the execution (similar to Edge)
|
| -
|
a new Freestyle layer is added in the layers tab
|
| Complete Python integration
|
| -
|
In previous phase, the Python interpreting functions had been ported to Blender’s internal Python interpreter
|
| -
|
SWIG’s dependency is currently being removed from the project. I am trying to do the migration as efficiently as possible. I am redefining Freestyle’s Python API to better support Blender’s native Python API.
|
| Rendering within a render layer
|
| -
|
I haven’t been able to render Freestyle to a render layer with offscreen rendering. I tried achieved it with frame bufffer objects. Despite using traditional code to do the job (drawing to an attached buffer and reading from it), the result is totally garbage, somehow copied from the back buffer.
|
| -
|
I will not spend more time on it because this offscreen rendering is a temporary solution anyway, eventually replaced by OpenGL-free stroke rasterizing (see below).
|
| OpenGL-free stroke rendering
|
| -
|
I will work on it as soon as SWIG is removed.
|
| -
|
I should mention that getting rid of OpenGL was not really explicited in the original proposal per se. I only mentioned using Blender’s native graphical commands to render strokes. It is now imperative that it is delivered by the end of the summer to consider the project as finished. Replacing OpenGL rendering will be a challenging task and it is difficult to estimate how much time will be spent on it (see below).
|
| -
|
the Camera settings are corrected to reflect those in Blender. Unfortunately, the code only supports perspective projection. The orthographic case is unfortunately not taken into account by libQGLWidget's Camera class.
|
| -
|
lib3ds will be removed as soon as strokes can be rendered without OpenGL. It is still unclear how it will be achieved. Nonetheless, one thing for sure is that the geometric data will be pulled directly from the render database.
|
| -
|
a style still cannot be selected from the UI.
|
Work to be done
Here is a summary of the remaining work planned in the proposal:
| TODO
|
3) Decomposition into reusable functional modules
|
Schedule: Jun 30 - Jul 20
|
| View map as a stand-alone render pass
|
| Available render passes' utilization to speed up the calculation of the view map
|
| Style definition through compositor nodes
|
| TODO
|
4) Graphical tools to control view map calculation
|
Schedule: Jul 21 - Aug 9
|
| Graphical edge selection for view map limitation
|
| Edge group support
|
| TODO
|
5) Extra documentation and debugging
|
Schedule: Aug 19 - Sep 1
|
| API crystallization (based on artists’ feedback)
|
| Original style modules migration to new API
|
| Tutorial: how to define new styles
|
| Tutorial: how to implement a new C++ based Python object attached to Freestyle’s API
|
I would want to be more specific about each phase but it’s too early to predict what issues I’ll face and where the bottlenecks will be. I consider the removal of OpenGL the most critical part overall.
Upcoming challenges
Timing
The first challenge that I am faced with is keeping up with the project’s schedule. In my original proposal, I had planned to have the render layer functionality working by the end of June, expecting to make some progress while finishing up school. Unfortunately, due to a very demanding end of semester, I was not able to work on the project at all. This explains while I am now two weeks late according to plan. Thankfully, I had originally added a lot of time on the view map manipulation phase to handle a change in the project’s schedule.
The current situation means that I do not have any more room left to adapt to changes in the implementation. I still consider that I can finish up the project for September 1. I just need to reorganize the project schedule to keep the essential features. I’ll decide on a new schedule with Jean-Luc in the next few days and will add it to the report when determined.
Removing OpenGL bindings
To consider Freestyle fully integrated into Blender, it needs to be inserted appropriately in Blender’s rendering pipeline. The greatest challenge is replacing the OpenGL commands currently used for rasterizing. The internal engine is tile-based and multithreaded, something that can’t be achieved by Freestyle currently. Even though Freestyle will eventually be made fully compatible with these requirements, a non-tile-based rendering will be achieved for the scope of the Summer of Code project. That means the stroke strips will have to be independently rasterized after the threaded rendering function is finished. Some work will have to be done to make sure that antialiasing and stroke textures are supported.
After the program, a tile-based approach will be implemented. There are different approaches to resolve the problem. One of them is pre-generating the stroke triangles and only raster them in each tile after being properly culled. Another approach would be first determining in which tiles each triangle ends up in, then letting each tile render its overlapping triangles independently. Another and probably better approach would be to convert the stroke strips into triangles in the scene, placed in the image plane. Blender’s internal engine would therefore be in charge (via ray-tracing) of rendering the strokes, taking care of all texture and antialiasing issues. This seems like a great solution even attainable during the summer. The only real unknown is the performance hit for converting the strokes to triangles.
Fortunately, OpenGL is not needed for Freestyle’s internal calculations. Getting rid of the last remnants of libQGLWidget should be straightforward.
Style specification
In Freestyle’s original program, a style module is used for a whole scene. Even though the different style modules can be combined to produce a finalresult, they are at still applied to all of the objects of the scene. It would be nice to allow greater freedom , allowing artists to keep a fine-grained control over where and when a style module is used. Should it be defined at a layer level ? Per object ? Deciding any of these two choices would surely have an impact on Freestyle’s rendering pipeline, which would have to be rethought for this feature.
Results
Test with 'japanese_bigbrush' style module.
Test with 'cartoon' style module.
