Dev:Source/UI/CopyPixelsToTexture

2D Image Drawing Engine

Blenderwiki.MatthewPlough - 29 Jul 2005

1 Introduction

I am working on a rewrite of how 2D image drawing is handled.

Andrea Weikert (Elubie) is working on a very similar project; see http://www.elubie.de/blender/iconpatch/ for information and a patch.

2 Current features

On-screen images are currently drawn using the glCopyPixels family of functions. Conceptually, it is easy to understand, and functions' effects are kept localized. However, this implementation has more than its fair share of problems.

3 Problems with the current implementation

As outlined below, the current implementation is visually unattractive and is often slow. Today's video cards' <noautolink>OpenGL</noautolink> implementations are not optimized for drawing 2D using commands that resemble traditional blitting operations.

3.1 Preview problems

• In material, texture, lamp, and world previews, the preview image is not displayed if the left edge of the preview is not on the screen.
  
  

• When the preview is displayed, glCopyPixels performs nearest-neighbor interpolation, which looks blocky. This can be easily improved.
  
  

• If the preview is zoomed out, strange horizontal lines appear, which look absolutely terrible.
  
  

3.3 Other problems

• Martin Poirer pointed out that the little purple object centers are drawn using a 2D image. This slows down drawing considerably when lots of objects appear on the screen.

• While header icons cannot be zoomed, another problem discussed above also pertains to them. As illustrated below, they also are not drawn when their left edge is not on the screen.
  
  

• The same is true for other icons.
  
  

4 General requirements for a new implementation

4.1 User Requirements

On the user side, a new implementation must be faster than the old one and offer higher image quality. A faster implementation will allow Blender to handle larger scenes, moving the bottleneck to a video card's geometry performance, rather than its ability to copy pixels quickly.

4.2 Developer Requirements

On the developer side, a new implementation must be easy to program. The programming interface must be consistent, without functions that each have innumerable arguments. Functions should be orthogonal, with as little coupling as possible. Multipurpose functions should not be written; rather, each function should do one thing and do it well. (This should be recognizable as a Unix-style API)

5 Formal requirements for a new implementation

• NS_ indicates "not satisfied".
• S (section)_ indicates that section section) of the implementation analysis describes how the implementation satisfies the requirement.

5.1 Memory

5.1.1 Memory overhead shall be kept to an absolute minimum. NS_

5.1.2 The implementation shall cause no memory leaks in Blender-controlled memory. NS_

5.1.3 The state of the 2D texture engine shall not be saved in .blend files. NS_

5.2 Error checking

5.2.1 API functions shall check their arguments for validity. NS_

5.2.2 If it is unreasonable for an API function to check an argument for its validity, the function's documentation shall state that no check is performed and that unchecked runtime errors may occur. NS_

5.3 Image Quality

All image quality requirements are designed for sane video drivers. If a buggy video driver prevents one of the following requirement from being met, attempts shall be made to correct the bug in the video driver before writing messy workarounds.

5.3.1 If any part of the drawn image is clipped or occluded, the visible parts of the image shall be drawn. S 7.3.1_

5.3.2 If the image is drawn at a larger or smaller than the native texture size, interpolation shall be performed so that all visible texture area is drawn, completely filling without missing pixel artifacts the screen area where the texture is to be drawn. S 7.3.2_

5.3.3 The option shall be given to draw the image with interpolation better than nearest-neighbor. S 7.3.3_

5.4 Performance

5.4.1 Drawing of images shall be substantially faster in the new implementation than in the current glCopyPixels-based implementation. S? 7.4.1_

5.4.2 Preview images are updated after each scanline is rendered. Thus, update performance must also be fast. If copies of images to be drawn are passed to <noautolink>OpenGL</noautolink> for hardware accelerated drawing, the API interface shall provide a function to incrementally update <noautolink>OpenGL's</noautolink> image copy. S 7.4.2_

6 Implementation of requirements

6.1 API functions

GLuint tex_alloc(int xy_size)

The tex_alloc function allocates an RGBA texture of size xy_size x xy_size In order to comply with OpenGL 1.1, xy_size must be a power of 2 greater than or equal to 64. However, tex_alloc does not check the validity of xy_size at this point. The function returns >0 on success, 0 on error.

<strong><pre>int tex_free(int texid)

This function deletes the texture referenced by =texid it returns 0 on error.

int tex_use(int texid)</pre>

This function binds to the texture referenced by texid It 0 on error; otherwise it returns texid Note that errors in this function will generate viewing artifacts, but will not cause crashes. Thus, the return value may be ignored.

int tex_update(int xofs, int yofs, int w, int h, const void *pixels)

The tex_update= function updates the current texture. At this point, it is an unchecked runtime error to have no texture bound before calling this function. It returns 0 on error.

This function is very similar in usage to the =glTexSubImage2D= function, but its arguments are much easier to remember. It updates a rectangular piece of the bound texture; xofs and yofs indicate the offset in pixels from the lower left corner of the texture, and w and h indicate the width and height of the piece to update. The pixels argument is a pointer to the first element of an array of bytes; four bytes specify each pixel. The byte order is (red, green, blue, alpha). *This needs to be tested on a big endian system to make sure it works correctly!*

int tex_draw(int x0, int y0, int x1, int y1, int alpha)

This function draws a 2D axis aligned square in the x-y plane with bounds (=x0 y0, (x1 y1. The currently bound texture is drawn on the square; the lower left corner of the square is mapped to the texture origin with UV coordinates (0, 0), while the upper right corner of the square is mapped to UV coordinates (1, 1).

7 Implementation analysis

7.1 Memory

Memory requirements have not yet been satisfied; I have not worked on them yet. While my implementation uses only a GLuint= as a handle to a texture, more than that will be needed. Andrea Weikert's code provides a more sensible data structure; I will analyze it once CVS is fixed.

There is a known texture memory leak in my code; I provide a function to free a texture ID (and hence the texture memory) and have tested it, but do not use it.

7.2 Error checking

I have not worked on these requirements either. It may not be possible to error-check everything.

7.3 Image Quality

7.3.1 As shown in the example below, requirement 5.3.1 is satisfied. Furthermore, all sane video drivers cull and clip textures correctly. My code calls <noautolink>OpenGL</noautolink> texture functions correctly, hence culling and clipping happen.

7.3.2 The examples below illustrate that requirement 5.3.2 is satisfied. All sane video drivers scale textures correctly. Since my code calls <noautolink>OpenGL</noautolink> texture functions correctly, correct scaling occurs.

7.3.3 Requirement 5.3.3 is satisfied. The following lines of code cause the image to be drawn with bilinear interpolation:

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

7.4 Performance

7.4.1 Requirement 5.4.1 is believed to be satisfied. On modern graphics cards, drawing textures with hardware scaling and interpolation is much faster than software scaling and interpolation attempts. Benchmarks that draw the image with glCopyPixels and with textures will be needed to verify that the requirement is in fact satisfied.

7.4.2 The tex_update= function fulfills requirement 5.4.2. Incremental updates of one pixel, up to the entire texture are possible with this function.

8 Testing

8.1 3Dlabs results

Testing on a 3Dlabs Wildcat Realizm 200 revealed no repeatable issues.

8.2 ATI results

Elubie tested my texture preview rendering method on her Windows machine with an ATI Radeon 9000. Two problems quickly emerged.

She reported that the update operation often failed to complete, as shown in the following image. However, she says that she has not been able to reproduce this since she initially tested it.

Furthermore, materials sometimes failed to update, as shown here:

I was unable to reproduce these problems on a machine with an ATI Radeon 9600 graphics card. This is not to say the problems don't exist -- it will just be a whole lot more of a hassle to correct them.

8.3 Nvidia results

I have not yet received reports of testing on Nvidia hardware; I will test on an Nvidia Geforce 2 MX 400, but would like to test on both higher- and lower-end hardware.

8.4 Macintosh Results

Matt Ebb reports that my implementation works correctly on Mac OS X 10.4 on an ATI Radeon Mobility 9700, as shown in this screenshot:

8.x Other results

I am eager to hear results from other operating system and video card combinations. I am posting a Mac OS X 10.4 build and a Windows build in the "Testing Builds" forum. Also, I am posting a link to Elubie's build for comparison.

9 Others' thoughts

• Daniel Dunbar: What about texture compression? Can compression default to enabled even if Blender doesn't ask for compressed textures? How would this affect image quality?
• Daniel Dunbar: Would it be possible to wrap this so it uses either glDrawPixels or texture drawing on the back end?
• Daniel Dunbar: The glaDrawPixelsTex= in glutil.c already does something similar. atthew Plough: However, I'm not sure that it can meet performance requirements or the incremental update requirement. It's a useful reference, though.
• Andrea Weikert (Elubie): I have already done something very similar: http://www.elubie.de/blender/. A patch is coming very soon, so you can take a look at my code. One disadvantage of using textures is that it might not work well with older graphic cards and settings. (I had to increase quality to at least medium on my ATI R9000 to make mipmaps work (for nice quality on zooming, high quality freaks out blender). So having glaDrawPixels as a fallback (UserPref?) might not be a bad idea.