Andart

Nervous About Resizing

Communication Nation posts about a New technology for smart image resizing, developed by Shai Avidan and Ariel Shamir (Seam carving for content-aware image resizing, ACM Transactions on Graphics Volume 26 , Issue 3 (July 2007)) (here is a free pdf). The idea is to not just squeeze or crop the image as it is resized to fit (say) a browser window, but to remove less informative parts so that important objects and people remain.

It is a very cool technology that is surprisingly simple once explained (horisontal and vertical "seams" with minimal gradient information are removed. The seams are precomputed by running a dynamic programming method). Expanding the image can also be done using the same method, making image elements flow nicely. The movie is very convincing. It all seems to fit nicely with our cognitive system.

It made me nervous.

That images can be forged, edited and manipulated in numerous ways is nothing new. But this is realtime retouching, potentially making choices of what is important in an image. Imagine a photo in an online document using this technology. When fully expanded we will indeed see person Y between X and Z - no manipulation has been done to the original image. It is just that normally the browser window compresses the image and the page designer has made Y a preferred seam. We will tend to see X and Z, and likely never think of Y. This is a form of manipulation more subtle than just removing Y, since the designer can now truthfully say that the photo is authentic - the anti-Y activity is hidden in the seam data, which is just a display choice.

The expansion of images possible with this algorithm is also very impressive. But that added another worry: in the movies it was hard to see when we passed the size of the "real" picture and started to see generated data or when data was removed. The transition between real and unreal became seamless.

Maybe this is just me being oldfashioned and stuck in a modernist conception of objective reality. But an useful and elegant technology like this, apparently intended for inclusion in browsing, seems to make reality a bit too fluid. Or am I missing the point? Subjectively speaking we do this all the time in our brains anyway. We tend to overlook the boring stuff between the main objects in our visual field, we fill out most of the field with internally generated imagery and we often resize objects depending on saliency. Maybe we are simply breaking free of the old constraint that the canvas cannot grow or shrink. We are used to pictures remaining fixed in size and internal relations just because they couldn't change. Seam-carving makes images just as flexible as the world and our perception.

But I still think we are going to overlook Y on many websites.

Posted by Anders3 at September 1, 2007 11:56 AM