Can we trust the preview?

Introduction

November 2013.

Current way of dealing with RGB and Alpha values in Blender has solid grounds.

However the way of displaying RGB colors in semi-transparent areas bases on pretty hard to understand assumptions.

UV Image Editor doesn't allow us to see the real RGB colors of an external image.

If we want to see real RGB we have to import the image to compositor, convert it's alpha mode which changes the colors, so we anyway don't see what we should, but we see how original would look if we didn't convert it. It means that we have to perform some operations on colors only for preview and remember to revert this operation when we want to perform some real adjustments.

The other way of previewing real colors is to change one setting in the "Viewer" node, but this also requires passing an image through compositor just to look at it.

In this article I’d like to analyze how UV Image Editor works when viewing external images and the results of compositing.

The purpose of this article is to prove that RGB previews work not correctly and that there are some inconsistencies in RGBA previews (composites over transparency grid).

Previews of External Images

Let’s analyze the scenario of saving exactly the same image to two files: EXR and PNG.

The image is a solid color that is filling all of the pixels and we have the Alpha channel that we will "apply" to this color.
According to standards EXR should be saved with "Premultiplied" Alpha Mode and PNG with "Straight" Alpha Mode.

For creating EXR file – RGB colors get multiplied by Alpha channel. New RGB values are different than the original RGB values.

This is a proper way of storing colors, as RGB should represent both "emission" and "occlusion".

"Semi-transparent" areas are the ones where "emission" of colors will be partly "occluded" due to "transparency".

That's why "semi-transparent" areas become darker.

For creating PNG file – RGB values remain unchanged.

Technically we have access to original colors this way, but it's not a proper way of storing colors with alpha.

PNG is one of a very few formats (if not the only one) that's specifications require storing "Straight" colors.

For simplicity sake I ignore color spaces and bit depth

„Premultiplied“ Mode of Preview

When we want to see the image „with transparency“ we want to look at properly prepared display of this image.

The most common way of displaying „transparency“ is to take the image with Alpha channel and composite it over the „transparency grid“.

This simple compositing operation is performed automatically.

It is important to know what „Alpha Mode“ of the input image is required for proper display over „transparency grid“.

This depends on the method of compositing one image over another that is used by the preview.

When we composite “Straight“ colors of the foreground (FG) over fully opague background (BG) the operation performed on colors of those two images should be as follows:

(FG_rgb * FG_alpha) + (BG_rgb * (1 – FG_alpha))

When we composite “Premultiplied“ colors of the foreground (FG) over fully opague background (BG) the operation performed on colors should be as follows:

(FG_rgb) + (BG_rgb * (1 – FG_alpha))

In this case RGB values of the foreground (FG) stay untouched (they don't get multiplied by Alpha as when we deal with "straight' colors), because it is assumed that they are multiplied by Alpha „in advance“.

In Blender’s UV Image Editor the second way of compositing the image over „transparency grid“ is used. It is assumed that images are „Premultiplied“

It is the way that should be used and is used in most of the good compositing applications.

Previews are created such that they composite nicely over transparency grid.

When viewing "RGB colors" only - we are looking at "Straight" colors (which is wrong, because we are not looking at the colors that we have, but at the colors that don't exist).

Operations performed on RGB and A values of input image just for creating preview images are as below.

Please note that those operations are performed ONLY for drawing previews. Those operations don't really change the color values, so they shouldn't happen. The created previews represent different images than the ones we want to preview.

EXR:

PNG:

Such operations create:

correct RGBA previews of "Premultiplied" images.
not correct RGB previews of "Premultiplied" images.
not correct RGBA previews of "Straight" images.
correct RGB previews of "Straight" images.

What should be displayed

Previews should work in "Premultiplied" mode.

They should composite "Premultiplied" images nicely over transparency grid (Display RGBA) and show "Premultiplied" colors when viewing RGB only.

This means that when looking at RGB only of "Premultiplied" images they should nicely "mix" with black color.

Images that have Straight Alpha should not look nice over transparency grid (RGBA display) because they miss multiplication by Alpha

When looking at RGB colors only - we should see jagged edges.

EXR preview

When we open an image saved as EXR in Blender's UV Image Editor - the image is automatically interpreted as „Premultiplied“.

Image on the right shows interpretation settings and three previews of the image:

composited over transparency grid (correct display),
RGB colors (not correct display)
Alpha channel.

PNG preview

Blender automatically interprets PNGs as having „Straight“ Alpha mode. Image on the right shows interpretation settings and three previews of the image:

composited over transparency grid (not correct display, although looks ok)
RGB colors (correct display)
Alpha channel.

RGBA display

If we are dealing with EXR file interpreted as “Premultiplied“ - „real“ RGB colors of it will be passed to preview and composited over transparency grid.

If we are dealing with PNG file interpreted as “Straight“ - „real“ RGB colors of it are multiplied by Alpha to give us nice (but not proper) composite over transparency grid.

This is what happens in UV Image Editor just for creating preview. Values of colors as such stay untouched, so we are not looking at the actual composite.

This way we get correct display of "Premultiplied" image and not correct display of "Straight" image.

RGB display

In this mode we preview RGB colors of our image, but not necesserily the „real“ RGB colors of it.

It is assumed that when viewing RGB we want to see "how premultiplied color would look like if it was not premultiplied."

This is pretty strange. It's hard to imagine a single scenario where it would be useful.

If we are dealing with PNG file interpreted as “Straight“ - „real“ RGB colors of it will be shown, because „real“ RGB values represent “Straight“ colors.

If we are dealing with EXR file interpreted as “Premultiplied“ - its RGB colors will be divided by Alpha channel („un-multiplied“), such that we are looking at the “Straight“ colors that in fact don't exist. We see how something would look like if we did something with it.

A lot of thinking is required to figure out what we are looking at.

Images get converted in one way or another, but just for creating previews. Those operations are not performed "for real" so we see different colors, than the ones we will be working on.

Alpha display

In this mode we preview black and white representation of values of the Alpha channel.

Compositing Previews

We can preview every output of every node in our compositing node tree.

We can link it to the input of “Composite“ node or to “Viewer“ node.

It is assumed that “Premultiplied“ images are previewed.

When image with “Straight“ colors is linked to Viewer or to Composite node - strange result will be shown.

The RGBA preview is strange, but this is exactly what we should expect.

"Straight" colors don't composite nicely over transparency grid, so we shouldn't expect nice results.

RGB display on the other hand looks even more strange, because "Straight" values get divided by Alpha and we see weird bright fringe.

Again: The values get divided, but only for preview. This operation doesn't happen "for real". Actual RGB values stay unchanged.

EXR in Compositor

EXR files linked directly to “Composite“ or to “Viewer“ nodes will be displayed the same as if we didn’t use Compositor.

The left part of the image on the right shows RGB and RGBA previews of EXR file that is not passed through Compositor.

The right part of the image shows RGB and RGBA previews of the same EXR file that in Compositor is linked to “Viewer“ node.

We are dealing with exactly the same issues as when viewing "Premultiplied" images directly in UV Image Editor:

RGB display is not correct
RGBA display is correct

PNG in Compositor

What goes to preview is expected to be “Premultiplied“. In case of PNG files in Compositor this interpretation fails.

PNG files linked directly to “Composite“ or to “Viewer“ nodes will be displayed strangely.

The left part of the image on the right shows RGB and RGBA previews of PNG file that is not passed through Compositor.

The right part of the image shows RGB and RGBA previews of the same PNG file that in Compositor is linked to “Viewer“ node.

In case of "Straight" colors in compositor we have:

"ugly" and not correct display of RGB
"ugly" but correct display of RGBA

In order to get exactly the same preview of the PNG image that is loaded into compositor as we had when previewing it directly in UV Image Editor – we have to manually convert its „Alpha Mode“.

This operation gives us the same display as when directly viewing the image in UV Image Editor.

Exactly the same, but that's just because we really converted the alpha mode. From now on we are dealing with different image, so why do we see the same?

„Alpha Over“

Let’s play with two EXR images that we will call „Green“ and „Yellow“.

Let’s assume that those images were created and saved according to standards.

"Yellow" is a solid color filling the entire image.

"Green" was created by taking solid color and applying Alpha channel to it.

RGB values have been "pre-multiplied" by Alpha and EXR file has been saved.

Here's how those images look like in Blender's UV Image Editor when using different drawing modes:

„Green“:

green_rgba.exr:

(Please note not correct display of RGB colors only)

„Yellow“:

yellow_rgb.exr:

If our goal is to simply place „Green“ over „Yellow“ we can use „Alpha Over“ node with its default settings.

Result:

Everything works perfectly, images have been nicely merged.

“Alpha Over“ node expects “Premultiplied“ image as its second input. If we feed it with “Straight“ colors – it gives us possibility to convert it to “Premultiplied“ by turning one little option in the node itself.

We are able to correctly preview our source images "with transparency".

When viewing RGB only we see how "straight colors" that don't exist look like, but can't see real "Premultiplied" colors.

If compositing an image „over“ another image was the only compositing task – we could say that RGB previews are not that important.

Colors may easily be converted from “Straight“ to “Premultiplied“ and vice-versa, but we can't see them easily.

The problems start when we begin to do some real compositing, not only placing one image over another.

Show me the „real“ colors

There are several compositing tasks that we’d like to perform.

In most cases we have to separate working on RGB colors and on Alpha.

The image may come to Compositor with the Alpha channel and sometimes it’s not necessary to change it at all, but we want to alter just the colors.

In such cases we must not work on “Premultiplied“ colors.

If we are dealing with “Premultiplied“ image and want to alter its colors we have to convert its colors to “Straight“, make the adjustments and only then convert it back to “Premultiplied“. Converting the result back to “Premultiplied“ is sometimes even not necessary.

In Blender’s compositor it is assumed that „everything“ is done using “Premultiplied“ colors, where in fact only one node expects “Premultiplied“ colors:

It is “Alpha Over“ node. All other nodes need to be fed with “Straight“ colors.

When we want to merge two images using “Mix“ node, where we can use several „Blend Types“ – it works well only when we feed it with “Straight“ colors.

In case of working on colors separately we’d want to see the „real“ colors. We’d want to see the colors that we are really working with.

Is it possible?

Yes it is, but there are several unnecessery steps that we have to take.

One option to see the „real“ colors is to disable “Use Alpha“ in the “Viewer“ node:

When we do so our previews in RGBA and RGB modes will look exactly the same and will show the „real“ colors.:

Sometimes we want to see the „real“ colors and then simply „glance“ at the image over transparency grid.

To do it we have to disable „Use Alpha“ in Viewer node just to see the real colors. Then just to see RGBA we need to change this setting in the node itself and as the second step we need to make sure that in the preview – proper drawing mode is selected.

RGB values vs. preview colors

Let’s take a look at the values of some fully opaque pixel of EXR file that is interpreted as “Premultiplied“.

When we preview images in UV Image Editor - we can check the RGBA values by left clicking on any pixel.

When the drawing mode is set to (this is exactly what the tooltip reads):

“Channels of the image to draw:

Color: Draw image with RGB colors“

and we check RGBA values of some fully opaque pixel – we will be given the values of this pixel and the previewed color will be exactly the same.

Now let’s take a look at the values of some semi-transparent pixel of exactly the same image (we are in RGB preview mode):

We are shown the same „color“ no matter if the area is opaque or semi-transparent although we know exactly that those colors are different. Colors in semi-transparent areas are. darker because of premultiplication by alpha.

The values however are different. They are correct.

There is a mismatch between what is shown and the pixel values.

Suggestions

The way the RGB display of semi-transparent pixels should definitely be changed.

Maybe there are situations where the artist would like to see „How the straight colors of premultiplied image would look like“, but I think those situations are very rare.

In most cases when displaying RGB colors we’d rather see what those colors really are.

A lot of confusion may be caused by the current way of previewing. The artist may draw false conclusion that some images are converted before anything happens to them, but it is not true.

In my opinion there are two ways of solving this problem:

RGB draw mode should show „real“ colors. If colors are premultiplied - we should see premultiplied. If the colors are "straight" we should see "straight"
There could be a checkbox in UV Image Editor that could change the way colors are displayed. Default setting should be as above, but the artist could chose to see "unmult" version of the image by checking some checkbox. Even if it existed, I don't think anybody would want to use it anyway.

利用者:Bartekskorupa/Display Issues

目次