Dev:Source/UI/Window Manager Compositing
Window Manager Compositing
OpenGL buffer swapping, drawing overlapping menus or brushes, partial redraw should get a clean implementation in Blender 2.5. Especially front buffer drawing should be eliminated. In 2.5, the window manager will be in control of what gets drawn when, and this way we can better support multiple techniques to deal with varying graphics cards and OpenGL drivers.
Goals
- Correctness: no flickering or other artifacts.
- Speed: drawing should be as fast as possible.
- Minimal memory usage: graphics cards have limited texture memory.
OpenGL
Double Buffering
- We assume to have double buffering available. That means we have a front buffer containing what is actually shown on the screen, and an offscreen back buffer.
- Front buffer drawing is slower than back buffer drawing, with some cards and drivers to the point of being unusable. Additionally, it means partially drawn results may be show on screen.
Swap Methods
- Swap copy: the back buffer is copied to the front buffer.
- Swap exchange: the pointers to the back buffer and front buffer are switched.
- Are there cards that do something else still? For example clearing the back buffer each time (irrespective of copy or exchange). We assume not at this moment.
- We cannot enforce either of those two methods to be used, although we can give hints for which one we would like.
- There appears to be no way to query OpenGL about which method it is using. We could however test this empirically, by drawing a unique color or pattern in the back buffer, swapping, and testing if it is still in the backbuffer.
- These methods can change at runtime, for example nvidia switches from swap copy to swap exchange when going fullscreen in X11.
Partial Swap
- All cards can swap the whole window.
- Some cards support swapping a list of subregions rather than the whole window.
XOR Drawing
- When drawing with XOR blending, no information is lost. We can do the exact same draw operation again to restore the buffer in its original state.
- This gives quite restricted drawing possibilities so is most likely only useful for brush or gestures, and perhaps even sometimes for those cases is too restrictive.
Other Buffers
- Rather than relying only the front and back buffer, we can also use additional buffers, with the advantage that swapping will not interfere with them.
- This buffer can be stored in main CPU memory by reading pixels back from the graphics card. The disadvantage of such a method is generally performance due to slow transfer rate between CPU and GPU, and additionaly it causes a pipeline stall blocking all other operations. This makes the method mainly usable for smaller regions and not the full screen.
- The buffer can also be stored in main GPU memory. This typically takes the form of a texture or accumulation buffer.
- Accumulation buffers can be drawn to directly, and are useful for selection, but cannot be assumed to be available everywhere.
- Textures are available everywhere. Only using extensions (FBO) can we draw to them directly, however all OpenGL 1.1 implementions in principle permit copying from the backbuffer (and frontbuffer) to textures. Such a copy is supposed to be quite fast since it happens on the GPU. We however have to be careful here to not use too much GPU memory, as this is limited especially on older cards.
Asynchronous OpenGL
- Modern OpenGL implementations do not execute drawing operations immediately and then wait until they are done. Rather a large buffer is kept storing all the state changes and data which is then submitted to the GPU.
- While the CPU is doing those draw calls, the GPU may still be rendering the previous frame. Typically synchronization for the current frame is forced in the swap buffers call of the next frame, i.e. with a 1 frame delay.
- Calls like glFinish and glReadPixels force the CPU to wait until the GPU has finished work breaking this asynchronicity, and hence are ideally avoided.
Setup
- There is a screen subdivided into non-overlapping regions (ignoring area and screen decorations for now).
- Inside those regions or overlapping multiple regions we have a stack of other things to draw, i.e. panels, menus, brushes and gestures.
- For selection we need to temporarily draw into a buffer and read back from it. This buffer has the size of a region or smaller and is not required to be the backbuffer, although that is an option.
- For some operations like sculpting or painting we like to partially redraw regions.
Solutions
Non-overlapping Regions
First we deal with the problem of non overlapping regions. In case of swap copy this is easy to solve. We merely have to redraw those regions in the back buffer each time, and can then safely swap the whole window.
For swap exchange this is more complicated; currently there is a method that will track if each region is valid in the back and front buffer. The result of this is that some regions are drawn twice when they could be drawn once. As long as the region is being repeatedly redrawn no second redraw is needed (i.e. during transform, view navigation), once another redraw is not required anymore, it will do a second redraw.
If partial swapping is available in a swap exchange system, this may be useful in avoiding those second redraws. It would seem to me however that using partial swapping then will effectively make the system effectively switch to swap copy, so perhaps for this use case hinting for swap copy may be just as effective. Regardless, we can use such partial swap to only mark the just-drawn regions for swapping.
If we really want to avoid that second redraw, even though it may in practice not be that common, we can still try two things. First, we can ensure that the back buffer (or an interesting part of it) is also stored in an texture, before it is swapped to the front buffer. Then on the next redraw after the swap we can write the contents of that into the back buffer. The main disadvantage of that is memory usage. Alternatively it is possible to copy parts of the front buffer to the back buffer, this may be slow on some OpenGL implementations however.
Overlapping Regions
Menus and brushes should be quick to draw and so we could redraw them each frame, there is little point in trying to avoid drawing them. Panels could be redrawn along with each region, as happens in 2.4x, though it would be good to make redrawing them possible without redrawing the region they live in.
One possible solution is to use one texture the size of the back buffer and draw or copy all non-overlapping regions into that. We can then copy the complete texture or parts of it to the back buffer before drawing, and draw panels, menus and brushes over that. By copying only parts from this buffer we can avoid redrawing panels too often.
Also possible is backing up the area under the overlapping region before it is drawn, rather than the whole back buffer. This is effectively what the menu system does in 2.4x, and panels and brushes could work similarly. Before drawing they would need to backup this area (if it is not already done). Then on subsequent redraws or when the menu dissappears, it can be restored. This backup can be to a texture rather than to the CPU as was done for menus in 2.4x.
This works better for swap copy than it does for swap exchange. With swap copy we can copy to the texture from the back buffer, however with swap exchange we may need to copy from the front buffer. This may be sufficiently fast, or alternatively an extra redraw could be done.
Selection
The easiest solution here is to just draw to the region where the selection happens. Most likely it will need to be redrawn anyway. An offscreen buffer could also be used, though the advantage of that is not entirely clear to me. For sculpting with partial redraw, which needs the full z-buffer of the region, perhaps one redraw could be saved that way.
Partial Redraw
Partial redraw is not really that different from drawing any of the non-overlapping regions. With swap exchange we don't really have to worry about redrawing it twice, though perhaps last partial redraw area could be cached so that we can draw it in the other buffer the next time. Alternativaly the we could redraw both partial redraw regions combined.
