July 2011 Archives
The art of resampling
Years ago when I was a fledgling still learning to code, one of the first things I tried creating was an image resizer. I preferred (and given time, still do) to just have a go at things without research so while I succeeded in making a resizer, the results were (predictably) poor. I soon got sidetracked and left the code to rot, but the idea remained.
It’s taken me over 10 years to find a reason to revisit the subject, but I finally have: gamma compression.
In HTML, the color #777 will have roughly half the intensity of #FFF. This matches our perception and makes working with color fairly easy, but the way light really works is much different. Our eyes perceive light on a logarithmic scale—twice the photons won’t appear twice as bright to us. Transforming the actual linear intensity into our familiar representation is called gamma compression.
When we blend two gamma-compressed colors, the result is not the same as if we blended two linear colors. To correctly blend colors, we must first uncompress them into their linear values. Take the gamma-compressed values 0.1 and 0.9. If we just add 0.1 to 0.9, we’ll of course get 1.0: an 11% change of value. Doing it the correct way, we first decompress them into the linear values 0.01 and 0.79. Add 0.01 to 0.79, re-compress, and the result will be 0.905: an 0.5% change of value. Gamma-ignorant processing gave us a way off result!
For an example, lets take a look at NASA’s “Earth’s City Lights”:
Downsizing this 4800×2400 image presents a worst-case scenario for a gamma-ignorant resizer. The sharp contrast between the lights and surrounding darkness makes the blending error very prominent, and the massive downsizing gives it a chance to do a lot of blending.
At the top we see the result of gamma-correct resizing. This is how it's supposed to look—you can still see the lights along the African and Australian coasts. Western USA is clearly still very awake, as well as Europe and parts of Asia.
On the bottom we see the result of gamma-ignorant resizing. The fainter lights have been completely drowned out. Australia and Africa now barely register, and all the other continents look far darker overall. Big difference! The unfortunate thing is that a majority of resizers will look like the image on the bottom. The incorrect result is often good enough that they either don’t notice or don’t care.
One of these incorrect resizers is in Avisynth, a scripted video processor used for everything from simple DVD deinterlacing all the way to heavy restoration of old 8mm film. A while ago I was using it to resize a Quake video and discovered that similar to the lights of the image above, all the starry teleporters lost their stars: a clear sign of gamma-ignorant processing.
I decided to make a high-quality resizing plugin for Avisynth that would use a fully linear, high bit depth pipeline. No shortcuts. Working on it has been a lot of fun and challenge, so I’ll be writing about it here over the next few days.
