Dev:2.5/Source/Imaging/Colour Management

Linear Workflow

Linear workflow has been discussed in so many places around the net now, that it's silly for me to repeat it all here. For an overview about what linear workflow/gamma correction is, check the links below and this neat summary:

Quote MasterZap

If you render on a "standard" computer monitor (i.e. an sRGB monitor which has a practical gamma of 2.2 on average) with no regard to gamma anywhere in your workflow (i.e. pretending the monitor has gamma=1, like unfortunately most software defaults to), then when you think you are making something twice as bright, you are actually making it almost 5 times as bright.

This makes even the most trivial math turn out completely wrong. Basically, your renders come out as if 2 + 2 = 10

This is why highlights blow out unrealistically, why reflections look wrong, why you can't seem to be able to use physically correct lights with a quadratic falloff, and why you have to save everything in comp with a bunch of horrendous dirty tricks like "screening" your speculars back on (Yuk!).

In a nutshell, users work in a gamma corrected colour space as seen on their monitors (sRGB or similar), but correct rendering/compositing calculations need to be done in a linear colour space.

The solution to this is to:

• linearize (inverse gamma correct) all user inputs used for colouring that have been designed to look correct on users' (sRGB) monitors, such as image textures, material colours, image comp nodes before they are used for rendering calculations
• perform all rendering/compositing in a linear colour space
• at the end, gamma correct the result back to sRGB for viewing or saving out 8 bit RGB images.

References (Rendering):

• http://renderwonk.com/blog/index.php/archive/adventures-with-gamma-correct-rendering/
• http://http.developer.nvidia.com/GPUGems3/gpugems3_ch24.html
• http://www.djx.com.au/blog/2008/09/13/linear-workflow-and-gamma/
• http://forums.cgsociety.org/showthread.php?f=2&t=610790

References (Compositing):

• http://mysite.verizon.net/spitzak/conversion/index.html

Implementation

Latest patch: http://mke3.net/blender/devel/rendering/b25_colour_management08.txt

This patch implements linear workflow in Blender under the umbrella term 'colour management'. Currently the patch supports conversion between linear RGB and sRGB, but in the future this (and additional) functionality could be extended to compensate for colour profiles embedded in image files, soft proofing, on display for the compositor and so on.

Examples in use:

• http://mke3.net/blender/devel/rendering/b25_colormanagement_test01.jpg
• http://mke3.net/blender/devel/rendering/b25_colour_management_comp02.jpg
• http://mke3.net/blender/devel/rendering/b25_colour_management_comp.png

(test image from http://www.4p8.com/eric.brasseur/gamma.html )

When colour management is on, inverse correction is applied to inputs into the render pipeline (to make sRGB data linear) and gamma correction is applied to outputs of the render pipeline (for correct perceptual display).

Inputs which get corrected include

• Textures used to influence colour
• Colour swatches in materials/world
• 8 bit per channel image nodes in the compositor

Outputs which get corrected include:

• Image editor render result view (in progress and after render)
• 8 bit per channel image files saved during animation render

Consistent with convention (and the OpenEXR spec), with color management enabled, Blender's floating point image buffers are defined to be in linear colour space. Correction only occurs when converting down to 8 bit per channel colour such as image files or for display on sRGB monitors.

Notes/todo

• previewrender.c: ED_preview_draw() - I commented out BLI_lock_malloc_thread() in order to make glaDrawPixelsSafe_to32() work correctly. I'm guessing this may not be safe, but I'm not sure what implications it has. Just making a note for reviewers to check!
• Todo: If necessary, replace pow()s with a LUT, as mentioned here: http://mysite.verizon.net/spitzak/conversion/sketches_0265.pdf . Recent tests have only seen very minor slowdowns due to colour management being on, but it could still help.
• Todo: screen_ops.c: Clean up the 8bit conversion section of image_rect_update() (where all the FTOCHARs are), to use the consistent floatbuf_to_srgb_byte functions. Should be ok, may just need to feed it the right coords.
• Todo: Change the one flag to separate options for:

* Color Management (global on/off)

* Materials

* File Output

* Display

so people can use their own custom workflows if they choose.

WIP scratchpad (a bit out of date)

Two tasks, can be done together perhaps:

• Linear workflow for rendering
• Colour profile management for output devices

Requirements

• All Blender's rendering and compositing should be considered to take place in linear colour space

• Colour management settings should be tied to the Blender Scene/Blender Image blocks, and giving consistent output results no matter what computer its rendered on (safe for render farms).

Definition

So there are two conversions:

• Images get converted to linear on the way in to the render/compositing pipeline
• Final results get gamma corrected for viewing on-screen and saving as 8 bit imagery
• These conversions can be a simple gamma(/inverse) correction, or they can be more complicated, using colour management profiles.

• Users work in a gamma corrected colour space, linear is for computer calculations only
• images, preview renders, colour swatches, render output, viewer nodes should all be considered/converted to colour managed/gamma corrected (sRGB etc) colour space

Proposal

• All 8 bit rendered image files should be defined to be saved in sRGB colour space.
• All internal float buffers (ibuf->rect_float) are defined to be linear RGB. (consistent with EXR spec)
• Other internal 8bit image buffers can be linear or non-linear colour space, possibly tagged with profiles, which can then be used (or ignored) for inverse correction in the rendering process.
• All other colour inputs to rendering (textures, colour swatches) are defined in sRGB space, and therefore need inverse correction while rendering.

Output

Can consider Blender's internal float buffers as a kind of 'master', from which corrected 8bit imagery can be made.

i.e.

• linear float buffer -> 8 bit sRGB PNG
• linear float buffer -> 8 bit sRGB Image Viewer
• linear float buffer -> 8 bit projector profile corrected Image Viewer

Implementation

Idea: tag imbufs with profile data on load. If no profile data exists in the file, give a default sRGB profile with gamma value from scene (default 2.2). Then as needed, use the data stored in the profiles to inverse correct. Float buffers can be linear or sRGB?

+ can choose at access time whether to correct or not

+ can properly account for images with non-sRGB embedded profiles

NOTE! It's not just images that we need to linearise before rendering - it's any texture mapped to colour (procedurals, ramps, etc) and individual colour swatches (eg. diffuse colour) too.

However, implementing this for images would be a very good first step, just keeping in mind the requirements for other inputs too.

• (ZanQdo) why?, aren't procedurals and ramps already linear? Their display on the UI might need gamma correction but I don't see why they would need inverse correction before being fed to the renderer.

* (Matt) Linear vs gamma is really a matter of interpretation - i.e. its how you interpret those pixel values to what they mean. When you edit a ramp, or a colour texture, you're editing it so that it looks correct in sRGB space, just like you paint a texture in photoshop to look good in sRGB space. To get the results out of the render that you actually see on screen when previewing in sRGB space, they need to be converted to linear, and then gamma corrected back to sRGB at the end of the render. The gamma correction is going to happen at the end of the render no matter what, so if you don't inverse correct these first, they won't end up the same as you saw it in the first place.

This also has some other implementation details since in blender, the texture preview *is* actually a render, but that's a small detail that can be worked out properly when the previews are brought back into 2.5 :)

While both ideas sound workable, Zanqdo approach is better because it makes the system be as simple as "Blender workflow uses linear floats", eliminating any doubts or special cases, pushing the problems to load and save files, and screen pixels (ie, the 8 bpc "zones"), while the core is fully linear float. Example: 0.5 means half... half grey and half intensity, always. Or going from 0, 0.8, 0.3 to 0, 0.4, 0.15 keeps the relative weights. Users and coders do not have to care if the floats handled are for one or another task, the numbers are exactly what is typed or read on screen or stored in memory, and the display is the one that gets corrected as well as files to comply with the "encoding" they use. In the implementation it is just moving "Colour swatches in materials/world" and "Textures used to influence colour" from input to output (they became "brothers" of render result view) in the above classification. Gsrb3d 19:20, 17 July 2009 (UTC)

Implementation Details

Loading

• Upon loading an image file into an imbuf, examine embedded ICC profile information stored in the file. If an embedded profile exists, tag it in the imbuf profile tag

Rendering

• When textures (mapped to color) are accessed in do_material_tex() etc, inverse correct before returning back to the calling function.
• When texture type shading nodes are accessed through the 'col' output, inverse correct before sending through
• When image textures are accessed via 'TexFace', inverse correct

Compositing

• As compositing happens directly on float buffers, all compositing buffers should be assumed to be in linear RGB space.
• Image/texture nodes need to be linearised on input.

Displaying / File Saving sRGB

• If an sRGB 8bit buffer is required, tag it as sRGB before conversion to 8bit from float
• in IMB_rect_from_float(), if the imbuf is tagged with a profile, gamma correct from linear RGB, to whatever is tagged in the imbuf profile tag.

OR

• alternatively gamma correct the float buffer on the GPU and display 8bit result after that

Displaying other profiles

• to display (soft proof) for another colour space (such as for a projector), tag the view node/render result imbuf with the specified profile.
• in IMB_rect_from_float(), if the imbuf is tagged with a profile, gamma/colour correct from linear RGB, to whatever is tagged in the imbuf profile tag

OR

• perform an additional colour correction/translation on the existing 8bit image buffer (in the case of viewing an 8bit image directly). NOTE: we don't want to overwrite the original buffer! store in crect.

http://mke3.net/blender/devel/rendering/b25_colormanagement_test01.jpg

Linear colour space during compositing comparison: ( test image from here: http://www.4p8.com/eric.brasseur/gamma.html ) http://mke3.net/blender/devel/rendering/b25_colour_management_comp.png

patch cleanup

- make function floatbuf_to_srgbbyte() and floatbuf_to_byte?

use in:

• glutil.c glaDrawPixelsSafe_to32()
• screen_ops.c image_rect_update()
• imbuf/divers.c IMB_rect_from_float()

Code Layout

Texture sampling

render/shadeoutput.c: shade_lamp_loop() / shade_color()

→ render/texture.c: do_material_tex() (and friends)

- This is the last place that we have access to MTex, showing whether the texture is mapped to colour or not

→ render/texture.c: multitex()

→ render/imagetexture.c: imagewraposa()

- Here, the imbuf is generated from Image block, imbuf is sampled, colour returned in TexResult

Render output display

(blender 2.5) - this is the same code path for images, viewer nodes, render results:

editors/space_image/image_draw.c: draw_image_main()

- has access to Image, Ibuf

→ editors/space_image/image_draw.c: draw_image_buffer()

- has access to Ibuf only, Image shouldn't be too bad to add?

→ editors/space_image/image_draw.c: image_verify_buffer_float()

→ blenkernel/intern/colortools.c: curvemapping_do_ibuf()

- applies image editor curvemapping, stores in 8bit ibuf->rect

• or imbuf/intern/divers.c: IMB_rect_from_float()

- simple conversion from float ibuf->rect_float to 8bit ibuf->rect

editors/space_image/image_draw.c: glaDrawPixelsSafe()

- Draws the 8 bit ibuf->rect

editors/screen/screen_ops.c: image_rect_update()

- also converts float to 8bit inline, for display of render progress. probably should be unified. Has access to RenderResult and RenderJob, and via RenderJob can access Scene, Image, Render, ...

Saving rendered images:

render/pipeline.c: do_write_image_or_movie()

- creates/allocates an imbuf from the RenderResult.

- watch out: this is only for still images, movies does its own float->8bit conversion. Perhaps this should use rect_from_float().

→ blenkernel/intern/image.c: BKE_write_ibuf()

→ imbuf/intern/writeimage.c: IFF_saveiff()

→ imbuf/intern/divers.c: IMB_rect_from_float()

- (if no 8bit ibuf exists)

Material colours:

render/pipeline.c: shade_lamp_loop() / shade_color()

- copied from mat->r to ShadeInput

render/rayshade.c: shade_ray()

- copied from mat->r to ShadeInput

render/rendercore.c: shade_sample_sss() / RE_shade_external() / bake_shade()

- copied from mat->r to ShadeInput

render/strand.c: shade_strand_shade_point()

- copied from mat->r to ShadeInput

render/sss.c: sss_create_tree_mat()

- copied from mat->sss_col to local col var

Maybe we can just make an init_shadeinput() function here to inverse correct sRGB colours in Material struct to linear RGB colour space in ShadeInput struct?

All those pow()s may be expensive though. Problem is, Blender uses the exact same material structures as in the UI - no conversion to a 'render material' as for lamps.

Though material diffuse colours etc are usually in the range 0.0-1.0 - perhaps we can use a precomputed gamma table for this, and use pow() for colours > 1.0? is this even worth it at all? Can ignore this for the time being and just concentrate on textures.