Tonemapper

For the last year and a half, I've been learning about color science through a long term project of creating a tonemapper/display transform. A tonemapper takes the raw image data and creates an image out of it.

I created this basic scene to showcase the characteristics of what a tonemapper does.

Picture Philosophy

The human brain is very special. It takes in all the information streaming from your eyes and interprets its meaning in real-time. One special process is the interpretation of a picture. Humans can see a collection of colors printed on a paper or displayed on a screen and think about it in a different context from reality. A photo printed on the wall doesn’t fool anyone into interpreting it as a magic portal. Therefore it is the aim of picture makers to create the best picture, not reproduce reality.

How does it work?

Brightness

Your screen backlight has a maximum brightness which each pixel darkens individually. A printed photo also has the brightest color of the white paper which ink can only darken. However, real life has no maximum brightness, evident when looking at the sun. How do you represent bright lights on such a picture?

The first answer is to put the directly captured colors onto the pixels. However, colors brighter than the screen can represent will be clipped. 

The second answer is to lower the exposure to fit the bright values in. However, everything else will become extremely dark. 

The answer is to apply a curve which preserves the darker values and compresses the brighter values. Changing the exposure of the image now controls how the values push into the curve.

Color

While b/w seems simple, color makes everything much more complicated. Color data is usually made of Red Green and Blue channels. The first thought might be to apply the same curve to each of those channels.

Oops...

Most of the image looks ok, but the blue cloth is overwhelmingly saturated and the highlight on the golden ball looks fake and brassy. How did this happen?

If white is when all channels are equal, other colors are made by darkening some channels more than others. In this graph for this light blue, the green and red channels are darkened.

However, for this pure blue from the cloth, the red and green channels are at 0 and only blue exists.

Inset

When reading images, humans expect bright colors to converge to white. There is no concrete answer of why, especially since human vision doesn't do this. However, it could be an expectation from seeing white specular reflections, the thinning of dark liquids, or the white glow of hotter fires. This desaturation of bright colors has always existed in paintings and film cameras. Most importantly, it exists in the graph for the light blue as the lines get closer to eachother.

Less saturated colors have this convergence, so why don't more saturated colors? As seen in the pure blue graph, the red and green channels are stuck at 0. Pure colors have channels which are stuck at 0 and never approach the other channels to create white. 

Therefore the answer is to desaturate the colors, apply the curve, per-channel curve, then re-saturate the colors. 

This technique was used in film emulsions by using more chroma laden dyes and is used in modern tonemappers like Troy Sobotka's AgX. Instead of desaturating all colors equally, a matrix multiplication transforms the colors into a smaller gamut which is inset from the display gamut. Then the inverse matrix multiplication is used to perfectly re-saturate the colors. Tweaking the position of each primary of the gamut can curve the hue in pleasant ways and compensate for luminance/chrominance power balance.

Only Curve

Desaturate > Curve

Desaturate > Curve > Resaturate

How does it compare?

AgX

The new default tonemapper in Blender 4.0, created by Troy and Eary Chow.

Blender Filmic

The default tonemapper in Blender, created in 2016 by Troy Sobotka to adjust the Blender community to the look of tonemapping.

ACES 1.2

The current industry standard, implemented in After Effects, Redshift, DaVinci Resolve, and more.

Future Goals

I believe that spreading knowledge about tonemapping will create a better future where artists can create better images. By simplifying and demystifying tonemapping, filmmakers can make more intelligent grades and luts, 3d artists can get better images out of the box, and 2d artists can have a more analytical understanding of picking bright colors. To spread this knowledge, I plan to do the following in the future: