The art of resampling

Years ago when I was a fledg­ling still learn­ing to code, one of the first things I tried cre­at­ing was an image re­sizer. I pre­ferred (and given time, still do) to just have a go at things with­out re­search so while I suc­ceeded in mak­ing a re­sizer, the re­sults were (pre­dictably) poor. I soon got side­tracked and left the code to rot, but the idea re­mained.

It’s taken me over 10 years to find a rea­son to re­visit the sub­ject, but I fi­nally have: gamma com­pres­sion.

In HTML, the color #777 will have roughly half the in­ten­sity of #FFF. This matches our per­cep­tion and makes work­ing with color fairly easy, but the way light re­ally works is much dif­fer­ent. Our eyes per­ceive light on a log­a­rith­mic scale—twice the pho­tons won’t ap­pear twice as bright to us. Trans­form­ing the ac­tual lin­ear in­ten­sity into our fa­mil­iar rep­re­sen­ta­tion is called gamma com­pres­sion.

When we blend two gamma-​com­pressed col­ors, the re­sult is not the same as if we blended two lin­ear col­ors. To cor­rectly blend col­ors, we must first un­com­press them into their lin­ear val­ues. Take the gamma-​com­pressed val­ues 0.1 and 0.9. If we just add 0.1 to 0.9, we’ll of course get 1.0: an 11% change of value. Doing it the cor­rect way, we first de­com­press them into the lin­ear val­ues 0.​01 and 0.​79. Add 0.​01 to 0.​79, re-​com­press, and the re­sult will be 0.​905:​ an 0.5% change of value. Gamma-​ig­no­rant pro­cess­ing gave us a way off re­sult!

For an ex­am­ple, lets take a look at NASA’s “Earth’s City Lights”:

NASA’s “Earth’s City Lights”

Down­siz­ing this 4800×2400 image pre­sents a worst-case sce­nario for a gamma-ig­no­rant re­sizer. The sharp con­trast be­tween the lights and sur­round­ing dark­ness makes the blend­ing error very promi­nent, and the mas­sive down­siz­ing gives it a chance to do a lot of blend­ing.

Gamma-corrected “Earth’s City Lights”

At the top we see the re­sult of gamma-cor­rect re­siz­ing. This is how it's sup­posed to look—you can still see the lights along the African and Aus­tralian coasts. West­ern USA is clearly still very awake, as well as Eu­rope and parts of Asia.

Gamma-ignorant “Earth’s City Lights”

On the bot­tom we see the re­sult of gamma-ig­no­rant re­siz­ing. The fainter lights have been com­pletely drowned out. Aus­tralia and Africa now barely reg­is­ter, and all the other con­ti­nents look far darker over­all. Big dif­fer­ence! The un­for­tu­nate thing is that a ma­jor­ity of re­siz­ers will look like the image on the bot­tom. The in­cor­rect re­sult is often good enough that they ei­ther don’t no­tice or don’t care.

One of these in­cor­rect re­siz­ers is in Avisynth, a scripted video proces­sor used for every­thing from sim­ple DVD dein­ter­lac­ing all the way to heavy restora­tion of old 8mm film. A while ago I was using it to re­size a Quake video and dis­cov­ered that sim­i­lar to the lights of the image above, all the starry tele­porters lost their stars: a clear sign of gamma-​ig­no­rant pro­cess­ing.

I de­cided to make a high-​qual­ity re­siz­ing plu­gin for Avisynth that would use a fully lin­ear, high bit depth pipeline. No short­cuts. Work­ing on it has been a lot of fun and chal­lenge, so I’ll be writ­ing about it here over the next few days.

Posted on July 23, 2011 in Feature article, Resampling

Related Posts