Andart

A Heap of Computer Scientists

Continuing my academic genealogies, here is a PDF map of computer scientists, based on the TCS genealogy by David S. Johnson and Ian Parberry.

Several of the Usual Suspects show up - Alonzo Church, Alfred Tarski and the tree of mathematicians around David Hilbert. So this graph links to the philosophy and math graphs. The node degree distributions look similar too - a power law, of course.

I'm starting to develop a theory for this kind of graph, it might turn into a paper yet. Unlike other social networks the topology is nearly strictly a tree, and the growth process might not be well modelled by preferential attachment (but I'm still experimenting) since each philosopher is active within only a particular lifetime. This time aspect is interesting since it is normally lacking or hidden in social network graphs, but here the tree structure makes it clearer.

Why the Prime Directive is Secondary

OK, maybe I'm beating a dead horse here, but it really annoys me. The prime directive is such a popular concept that it needs to be taken down a few notches, especially outside tge Star Trek universe where it at least was used as an occasional plot device. When I encounter people thinking it is actually a good ethical principle I get irritated.

The prime directive places the independent cultural development of a species higher than the welfare of its individual members (shades of "the needs of the many"?). While this sounds nice on paper, it has many nasty consequences. It means that the Federation should just look on when preventable disasters happen, even when inaction would be regarded as a moral outrage when occuring in respect to a non-protected culture.

Cultures may also be local. If the directive applies to the inhabitants of Zorgon XIII it ought to apply to indigenous peoples, not just preventing missionaries but also not giving them food, shelter and medicine if they need it.

If the directive is regarded as an ethical principle then it is presumably universal, and should also apply to the Federation. So the cultural development of the Federation is more valuable than the lives of any number of inhabitants. This way one gets a nice foundation for authoritarianism. It is easy to make the comparision with totalitarian Chinese policies seeking to safeguard 'social stability', but here the goal is even more nebulous and up for ruling technocrats to decide.

If it is not an universal principle, then the Federation just has it as a policy, habit or cultural decision. If another group like the Romulans wants to meddle it is OK, and the Federation can at most wring its hands or send warnings to the Romulans. If the Federation wants to be beneficient towards the protected culture it can use the meddlers as intermediaries (a bit like The Culture). This is of course total hypocricy and "not getting one's hands dirty".

If the directive only applies to every civilisation that is not sufficiently "adult", then one has to define what constitutes civilizational adulthood (in the Star Trek universe I guess spaceflight would be a likely definition, but as we have seen in enough episodes there appears to be bundles of spacefaring primitives out there). But if the value of independent cultures is so high as to merit the sacrifice of many individuals, then it appears likely that this value would be diminished by them getting adult and joining the interstellar community since the individual culture now would be subjected to globalization (OK, galactization).

The directive might be pragmatic in the sense that intervention brings disaster. In the IMHO rather silly novel Omega by Jack McDevitt everybody is firmly convinced that less developed civilizations will be totally culturally crushed by any contact with a more advanced one. This is said to be based on terrestrial experience, but as India and Japan show it is empirically not true. Even if it was true it does not seem to be a strong enough reason to avoid intervening against a threat against the survival of an entire alien species. In the novel what could have been a very straightforward rescue effort is turned inefficient and downright silly just to prevent any cultural contamination. But ethically, delaying/risking saving the inhabitants in order to ensure the survival of their culture is equivalent to first sending in rescuers to salvage all museums, libraries and archives, and once they are securely saved turning to the injured. It takes a very warped moral system to sustain that. And the assumption of this paragraph was that the directive was pragmatically based rather than a moral principle.

I agree with aristos Gabriel (spoiler) - if they got smallpox and I have a vaccine I'm going to give it to them, regardless of what they think. But I'm going to leave it up to them to think it. The authenticity of a culture comes from interacting with the outside universe (including other cultures), not from isolation. Respect for the uniqueness of cultures is an aesthetic value, but it is not strong enough to be an ethical value.

Where in the World is Christine Korsgaard?

My previous philosophy maps are a bit hard to navigate, so I did an annotated version. This helps find out the clusters of favorite philosophers.

It is interesting to see that the philosophers are so unevenly distributed. I used names I knew or that I found using the Wikipedia genealogy (leaving out a few due to graphical constraints). There are "big names" in most parts of the graph, but entire subclusters with no celebrities. I wonder if this is because they are too specialised to be well-known.

Freedom of Movement - in Health and Death

This week's CNE Health blog is about the consultation leading up to the upcoming EU health services directive, defining what medical services patients have a right to, quality of care and who will pay for it.

Meanwhile, Waldemar Ingdahl and Daniel Johanson defends freedom of choice in postmortem treatment (article in Swedish). Just as people no longer remain static with their illnesses and become "health tourists", people are less and less agreeing with the limiting Swedish rules for disposal of bodies. Cryonics is a good example: if you want to suspend yourself you better make sure you travel before death to a cryonics firm (and you better have all the paperwork in order).

The EU has great potential in allowing the free flow of goods, people and services. It doesn't need identical rules, just the option of going where the rules work - whether you are alive, ill or (temporarily) dead.

Genealogy: the Next Generation

My previous post about cultural and academic genealogy of course deserves a sequel.

Mike Love pointed me to his latest dataset, so now I have redone my clustering of his genealogy. I also got interested in extracting a genealogy from the Wikipedia entries for philosophers, which often contain a "influenced by/influence to" box. So here are the results (click for PDF):

Love's network is to the left and the Wikipedia-derived one to the right. The Wikipedia network has a stronger focus just on philosophy, with many more "lesser greats" included (yet leaving out some real greats since they did not have clear lists of influence - I will deal with that later).

The clustering from yEd behaves a bit strangely in the new Love dataset. The greeks get split into three clusters centered around Socrates, Plato and Aristotle. Plato ends up together with Augustine and early Christian thinkers while Aristotle gets Aquinas. We get a very nice renaissance cluster. The central philosopher cluster now includes everyone from Descartes over Kant to Marx and Nietzsche. We still have a number of subdiscipline, author and artist clusters. My guess is that the removal of many A->C links because there is an A->B->C link has weakened many clusters while the web of mutual influence in the German Gang remains equally strong, preventing it from splitting into the classical philosophy and existentialism clusters I liked before.

On the other hand the Wikipedia network has almost the same cluster. Since it has many more people with just one inlink or outlink we get plenty of clusters that are the centered on one Big Name. We get a very nice separation of analytic philosophy, and a cluster just for marxism. But we get a messy cluster of Plato, Aristotle, Locke, Decartes and Adam Smith. Removing all people with just one link to the network cleans everything up nicely, but it makes even more of the classic philosophers end up in the central cluster.

Which is best? I think both have merits. Love's genealogy covers a broader range and does not include every little thinker. The Wikipedia network is much stronger for contemporary philosophy and the borderland towards politics where marxists and libertarians dwell. That we get clusters that make sense and roughly overlap is also nice: there seem to be some ground truth to these networks.

Academic Pedigree

My previous map of the philosophy family tree was nicely circular, but as a Gustaf Arrhenius immediately pointed out after seeing his entry, it lacked the cross-connections of multiple advisors. Adding these destroyed all the neat symmetry. In return we got an even bigger giant component, a big flying cross with Leibniz in the lower left corner and the Russel-Whitehead-Wittgenstein gang on the right arm.

As an aside, there seems to be lots of academic genealogy projects. Of particular note is the neuroscience family tree. That computer scientists through mathematicians can lead their ancestry back to Leibniz isn't that strange. But many neuroscientists also tie into the above network. Eric Kandel has Otto Mencke in his background, and Herman von Helmholtz has neuroscience and psychology descendants beside his physics and philosophy descendants.

It would be fun to actually merge all these academic graphs to see the big picture, but I will likely need a better layout program to handle it. The philosophers are <8000 and yEd handles them well, but the philosopher graph is just 8% of the mathematician graph.

Next step is to show the influence links on the pedigree graph, but I leave that for next week.

[I have also added these images to my Flickr networks photostream]

Duct Tape for the Body

This week's CNE posting is about nano-scaffoldings for rapid hemostasis. It is the same substance that was used earlier to speed nerve regeneration, it looks like a real panacea.

The thing that annoys me a bit is that it is dubbed nanotechnology. It may have a better claim than many other things, but it is still made and applied by bulk processes. The nano part lies in how the peptides change behavior in an ionic fluid, polymerizing into nanofibers. Elegant and applied self assembly, but not very high level of precision. Still, the body seems to like the scaffolding.

The New Robotics... I mean, NEURObotics

Yesterday I attended the opening of the exhibition NEURObotics at the Wellcome Trust wing of the London Science Museum. I had been one of the science advisors so I got the chance to talk a bit with the journalists as well as sample the free sandwiches.

Overall, I like this exhibit. It is simple and really shows some of the devices that can affect the brain - multielectrode chips, TMS, chips for cultured neurons and so on. And there is the ubiquitious brainball game. I won again - maybe I'm relaxed, maybe I'm just empty in the head and no signal registers.

The exhibit has the usual problems with asking challenging ethical questions: it just asks them. This leaves the visitors trying to make up their minds without giving them any tools to do it. But at least it asks them and shows some of the good and bad applications of the technologies. Usually it is enough to show the tech and then ask. But creating the dialogue and discussion real answering would entail is of course very hard with a single exhibit.

I also discovered the Science Museum. Very fun exhibitions, especially the ones that didn't try to be interactive, for kids or "popular". Since England was at the forefront of the industrial revolution they got all the originals and firsts here, the devices you see in the history and textbooks. Seeing Lord Kelvin's mechanical spectrum analyzer, Babbages machines and the instruments of the 18th century physicists was moving.

In particular the exhibit of mathematical surface models was brilliant: just beautiful models and weird devices, with explanatory texts that dared to give the correct equations! Maybe most viewers won't get that, but it doesn't matter because they can feel that this is for real: real objects made long ago by scientists for real scientific use. Or for the sheer beauty of it - the generalised Klein bottles in glass by Alan Bennett were a treat.

Colorful Academic Genealogies

Mike Love's Genealogy of Influence is a fascinating resource tracing who has influenced whom in Western thought.

Of course, I couldn't stay away from the data so I pilfered it from the site and tried my hand at displaying it as static graphs using the excellent yEd. The results are in PDF, so they can be zoomed and searched.

I tried a hierarchical graph, organic (force-directed) layout and a circular layout. Each shows different things. The hierarchy makes the causality of influence stand out, but it is hard to see who belongs together with who. The circular layout is cute and produces some nice clusters, but their logic is hard to see. The organic layout gives a surprisingly good timeline and (as we shall see) reveals a lot of structure. I just wish I had remembered to reflect it so it goes left-to-right.

For color I used graph distance from Xenophanes. While Thales is earlier, his influence is somewhat backward in this dataset since it extends via Pythagoras and Aristotle, so it looks better with Xenophanes. There is a need to build up the pre-socratics; Wikipedia has a nice genealogy of the pre-socratics which ought to be included here.

Plotting size as betweenness centrality reveals an interesting thing: there really does seem to be a backbone of great thinkers in western philosophy. Plato, Aristotle, Aquinas, Kant, Nietzsche and so on stand out due to their size. Of course, there is always a risk of bias in selecting the genealogy (if I know much about philosopher X he is likely to get many links, increasing his importance).

The real gem was when I allowed yEd to find natural clusters. To my great surprise the clusters mostly make sense. There are obvious existentialist, christian, painter and author clusters, as well as clusters apparently corresponding to philosophy of mind and nonviolent resistance. My labelling is probably going to annoy experts, who are also likely to argue that Dante isn't an aristotelian and that Hesse should be in the author cluster, but I leave them to argue with yEd. While Love's data had explicit attributions of people into different groups this clustering is done just on the graph itself. I guess this shows that the genealogy he created makes sense - the structure fits the historical structure we tend to organise things into. Whether this is actually a true structure is another matter, of course.

If I were to extend this, I would of course add a lot more scientists and political thinkers. A more fun thing would be to take a philosophy encyclopedia, extract the entries denoting philosophers, and use occurences of other philosopher names in entries to create links. This kind of automated construction will no doubt be noisier and of lower quality, but could become far larger. The Philosopher Family Tree and the The Mathematics Genealogy Project (and the emerging theoretical and high-energy physics ones) could add a bit of information too. But unfortunately teacher-student links doesn't necessarily imply much influence.

A case in point would be my own genealogy: I can trace much influence to my advisor, but his advisor is in a somewhat different field (computer vision), and his advisor moved into the field from mathematics. My delight in differentiable objects is hardly inhereited from them. Still, it is pretty cool to see that I can claim both Felix Klein and David Hilbert as my (advisor)⁷! Going backward I find C.F. Gauss as (advisor)¹⁰ as well as all the greats of analysis up to Leibniz as (advisor)¹⁵. Of course, that is not surprising. Any genealogical tree will eventually end up with a few greats, especially when there are a few people with two advisors. And the Leibniz family is the biggest one in both philosophy and math.

Still, I couldn't resist plotting the entrie The Philosopher Family Tree:

Since it is a tree (I left out the root nodes) the circular layout works great, although the large number of small 'families' create the elegant ring-like shape. The big fractal to the right is the Leibniz cluster, in this case the philosophical descendants of Mr Monad rather than the mathematical ones (although many are represented). For some reason text searching doesn't work as it should in the document, which is unfortunate. I did a PDF of just the Leibniz cluster, and there it works.

It would be interesting to construct a Wikipedia interface along these lines, so that one could see the links and clusters in idea-space, in academia-space and family-space around a person.

Warning Signs for Tomorrow

The Book of Ratings rates danger symbols (via Ian Albert). Very enjoyable, and brings up the question of how to mark new threats. All the truly cool transhuman technologies are going to require warning signs.

I have added larger versions of these signs to my Flickr photostream.

Maybe we should define a RFID standard for this kind of hazard warnings, if it doesn't already exist? Every container or dangerous object has a RFID tag that enables a "danger detector" to tell that there is a particular problem nearby. It sounds like a reasonable extension of PML. Of course, a warning for "RFID reception impaired" might be in trouble. However, there is a need for signs to tell meddling monkeys to keep away. There are a few obvious concerns: the signs should be clear, easily recognizable, not reliant on color vision and so on. But as the ratings show, the symbols are rather arbitrary.

Note the wall in the second frame for some examples. The second sign from the left is a warning against naked singularities (better depicted in some later strip). The ghost warning is obvious (but a bit strange given Kevyns rationalism during the haunted starship storyline). The DNA helix suggests some alternative to the (IMHO great) biohazard symbol, or that it was too hard to draw. Maybe it is about threats to the genome.

The singularity tornado sign is a good one. It can be used to mark any object with dangerous spacetime metric properties, be it tidal forces, event horizons or bad topology. However, the tornado suggests the erroneous embedding view of space-time, so a better version might just show a spiralling shape. Of course, purists will point out that orbits around black holes are just as stable as around any other mass – they don’t suck things in without some friction mechanism – and that many other nasty metrics are not even symmetric. But some physical realism ought to be sacrificed for visual saliency. A black spiral signals an obvious, dynamical danger.

Antimatter is another obvious danger. My symbol is intended to both remind of a Penning trap (for early applications where we just have a few antiprotons) and a starlike explosion (for bigger amounts). It is also in reverse, to hint at the anti-aspect of antimatter.

Chaos control is likely to be very useful in many future applications. But chaos is sensitive, so interfering with such a system might be unadvisable.

Macroscale quantum systems are likely not personally dangerous, but interfering with them will of course disrupt whatever they are doing. Of course, just seeing the warning might be enough to decohere them. Some might interpret the psi sign as a warning for psychic powers. Maybe the sign could show an alive/dead cat instead, but it is a bit too much of an in-joke. People should learn what symbol is used for the wave function.

If strangelets are stable at zero pressure and have a positive charge, they can absorb ordinary matter and convert it into more strangelets. A great energy source, but also potentially a planet- or star-eater.

Whether nanoparticles are dangerous or not depends likely on their size, composition, surface properties and what system they get into. But it seems reasonable to have a marker for generic nanoparticle problems.

As nanosystems become more common we might need to make their activity visible. There is nothing as worrying as invisible technology that might or might not be there, acting quietly. Hence active nanodevices ought to be shown, either by a warning sign or through some aspect of their activity (color, light etc). Doctorow and Stross had a fun idea for medical nanodevices in Jury Service, where they mark the person being rebuilt with small biohazard signs.

Diamondoids are likely to be very useful, but can be pretty risky in some forms. Bulk diamond shards for example are extremely sharp and unlikely to wear down quickly. The hardness and slipperiness of a perfect diamondoid surface might also pose risks.

Self-replicating devices are potentially the most powerful technology of all, since they can increase their reach exponentially in suitable environments. It also raises risks of arms races (whoever builds and releases one first wins), unwanted replication (the sorcerers’ apprentice) and slight misbehaviour of the devices that becomes dangerous as they become numerous.

Autonomous devices may start acting at any time because they “want” to. While they might have safety programming this is nothing that can be taken for granted. I’m rather happy with the symbol, using the square as an icon for device, the eye as a sign of some form of intelligence or environmental sensitivity, and the arrow for action.

Lack of internet connectivity: today a nuisance, tomorrow a serious problem and possibly even damage risk. As our exoselves become more and more dynamic and linked to our biological minds a loss of connectivity could cause breakups of our extended minds.

Using microdevices, identity technology and pattern recognition software it is not inconceivable that future environments will be 100% privacy-free. If this is not the standard situation but only occurs in some places, this sign would mark the surveilled area. In the case of total surveillance the opposite sign, with a crossed over or blindfolded pyramid, would mark the risky non-observed areas.

A system that evolves freely is potentially very adaptable and creative. It could also become nearly anything, with consequences ranging from the annoying to the disastrous. It is likely unlimited self-evolution will need to be contained carefully even as we mine it for truly new inventions. The arrows nicely hint at a chaos-star as well as replication.

Systems that might launch a “hard takeoff” where they rapidly become smarter and more capable require care. Even software that doesn’t aim at intelligence might still behave unexpectedly, requiring a warning.

Group intellects: don’t go too close, or resistance will be futile… While well-behaved group minds no doubt are selective of who joins (it is after all rather intimate) and unlikely to assimilate everybody nearby, there might be applications or situations where mental firewalls are down and brains easily form group intellects. Maybe the people in the icon should all be raising their hands in the same way, but this is the clipart I found.

Memetic hazards: a black lightbulb to represent really bad ideas. Compare with the Science Related Memetic Disorder in A Miracle of Science. Of course, the line between preventing viral bad ideas from spreading and censorship is a fine one.

Motivation hazards: as we learn to affect our brains better there is an increased risk for addictions, gain pleasure from something harmful or that we edit ourselves to like our current state no matter what. The poppy represents such motivation traps.

Exactly whatother kinds of hazards could occur with mature cognotechnologies is hard to imagine. The staircase sign represents a general hazard, perhaps the induction of inconsistent beliefs, infinite loops or mistaken perception.

Finally, a catch-all sign for things you really don’t want to mess with – existential risks implies threats to the future of humanity as a whole.

Size of danger

For a roleplaying game I came up with a system of signs denoting the size of the threat rather than its type. In that setting I assumed that mankind was spread out across several solar systems, so a species level threat needed to affect all the systems (gamma ray bursts, invasion of enemy aliens). Culture level threats represent dangers to a single system (self-replicating machines, a local nova). Ecosystem level threats could wipe out the biosphere and/or tehcnosphere of a planet. Such threats would be species level threats to us today: large meteors, permanent climate destabilization, ecophagous nanomachines. Nation level threats risk continent-sized destruction or smaller: nuclear war, biological attacks etc.

Perhaps this ranking can be extended. A natural scale would be the logarithm of the number of threatened people.

• Level 0 threats: all humans threatened. True existential threats.
  
• Level 1 threats: around 10% of all humans threatened. Continent-sized threats at present.
  
• Level 2 threats: 1% of all humans threatened.
  
• Level 3 threats: 0.1% of all humans threatened
  
• Level 4 threats: 0.01% of all humans threatened
  
• Level 5 threats: 0.001% of all humans threatened
  
• Level 6 threats: 0.0001% of all humans threatened
  
• Level 7 threats: 0.00001% of all humans threatened. At present <1000 people.
  
• Level 8 threats: 0.000001% of all humans threatened. At present groups <100 people.
  
• Level 9 threats: one billionth of humanity threatened. Individuals and small groups.
  
• Level 10: no humans threatened, but other values (such as unchanged biosphere, aesthetics or economy threatened).
  

Maybe the scale makes better sense in the other direction, with “normal” threats being level 1 and more serious threat as a higher level, ending with level 9 and 10 threats. But this way it acts as a countdown, making it explicit that the scale looks at the threat compared to the total size of humanity.

In my sf setting the signs had color (as well as text) depending on the level of threat. One could perhaps change the warning-yellow background depending on the threat level, but it would likely be bad for recognition. A better way might be to just have a color marker and some text underneath the warning sign. "SELF-REPLICATING DEVICE. LEVEL 0 THREAT: GLOBAL DANGER. DO NOT MESS WITH"

It might also be possible to add a marker or border showing the probability of the threat happening if one disobeys the warning. Again this could be marked using a logarithmic scale:

• Infrared (black): 100% probability
  
• Red: 10%
  
• Orange: 1%
  
• Yellow: 0.1%
  
• Green 0.01%
  
• Turquoise 0.001%
  
• Blue: 0.0001%
  
• Indigo: 0.00001%
  
• Violet: 0.000001%
  

However, this might both give people a feeling of licence (“It is just 1% chance that something will go wrong”) and quickly runs out of colours.

Another addition would be to add an “outrage scale” a la Peter Sandman to show just how outraged people will be if something goes wrong. Maybe this could be marked with a series of symbols or icons, denoting for example that if you mess up this particular thing, the damage is going to involve new technology, children, chronic effects and personal responsibility.

Death, Be Afraid, Very Afraid!

A fun advert for the Mexican Red Cross with a decidedly anti-mortalist theme.

A song on the same theme is Death to the Angel of Death by C.o.G.

"Those who say death cannot be killed
Lack the proper defensive skill"

Continuing in the same morticidal vein, here is cartoon. I'm almost starting to feel sorry for the grim reaper. Like other great mythological horrors it is becoming cute. I guess that is the secret of our modern, postironic world - we can poke fun at anything.

As a final link, here is a great cartoon by Carol Lay.

The Right to Know What Child one is Carrying

This CNE
Health blog was about ultrasound sex determination. In some places
hospitals give screenings to parents but refuse to tell the sex of the
child, out of fear that this would lead to gender selection. I would
argue this is wrong, both in terms of withholding desired information
from patients, due to the often racist undertones of the
anti-selection reasoning (it is often constructed as being ethnicity
specific) and as protection of the children. After all, the parents
can always go to a private clinic to get an extra scan which only
results in a bit of unnecessary, potentially unhealthy ultrasound and
a loss of money for the parents. In fact, forcing parents who think
the sex matters to do something extra to learn it is likely to make
them consider this information more salient, not less.

In general sex selection does not appear to be very prevalent in
western culture except as "family balancing" where parents want an
equal number of children of each sex. If an ethnic minority desires
more boys it may actually be undermining itself: it would either mean
losing in size or require marrying outside women, which presumably
would introduce other cultures into the ethnicity (unless of course
one assumes that women always slavishly adapt to their husband's
culture, a highly doubtful assumption). So maybe from an integration
perspective one should encourage sex selection?

In general the issue brings up an interesting question: what are the limits to medical self-knowledge and information about the child one is carrying? Today most medical systems are somewhat paternalistic about it, preferring not to offer a lot of information. Some of this is due to resource limits (tests do cost), much appears to be the assumption that patients cannot handle it properly and will become hypochondriac or just confused. But ethically it seems that we have a right to know about ourselves - even when this might be bad for us. Outsiders do not understand ourselves as well as we do, even if our self-knowledge is always limited. Our actions must be guided by the knowledge and values we have, including the action of acquiring more knowledge for future actions. Limiting this acquisition limits our ability to act morally. Hence, if the medical system retains a monopoly on this information and refuses to divulge it, it is acting wrongly (we might still have to pay for it, though - there is no right to have medical information paid for by taxpayer money). The proper thing to do would be to create institutions, whether extensions of genetic counseling to other forms of tests or health consumer services, that helps us make sense of and handle the information. How they should be formalised and paid for is a matter for another post (and a lot of deep thinking).

It seems to me that given the above reasoning we also should have a right to know everything about the baby. Prepersons do not have a right to privacy, and even children do not seem to be regarded as having a right to privacy. My general heuristic for rights is that you should get a right when you can understand it and desire it. This suggests that the right to social privacy emerges sometime around 10 years of age or so. But the right to medical privacy is more subtle, and likely to come a few years later. So learning everyhting there is to know about one's baby is OK. Besides, any human will grow up surprising regardless of what we know about its starting conditions.

The Future of Images

The latest issue of ACM Transactions on Graphics (TOG) (July 2006, 25:3) are the Proceedings of ACM SIGGRAPH 2006, Boston, MA conference so it gives a sense of where state of the art in computer graphics is. One of the strongest impressions I got was that photography is going to change. Images are becoming just as dynamic and laden with inferential promiscuity as computer-readable textual information.

Now when rendering has more or less achieved full photorealism one way to go is of course expressive non-photorealism, but merging photography with rendering (image based rendering) is also popular. And one can move into the even harder domain of video (lots of Hollywood interest of course). At the same time computer vision has become involved in manipulating photography. All this points at a situation where there is no distinction between photographing, rendering and image processing.

Daniel Cohen-Or, Olga Sorkine, Ran Gal, Tommer Leyvand, Ying-Qing Xu describe a method of harmonising colors in photos. Apparently there are a few basic "color harmonies" (like having most colors on opposite sides of the color circle, or deparated by 90 degrees etc), so by analysing the existing colors it becomes possible to nudge them to the closest harmonious color scheme. Some trickery is needed to deal with extended fields of colors so they all get transformed the same way.

Noah Snavely, Steven M. Seitz, Richard Szeliski developed a system for "photo tourism", where a large number of images from the same place (such as the results found after a Google images or Flickr search) are matched. it deduces the original camera locations and produces a 3D model of the scene and where the cameras were, enabling the user to jump between the perspectives, zooming in on details by finding pictures of them and allowing image annotation.

Erum Arif Khan, Erik Reinhard, Roland W. Fleming, Heinrich H. Bülthoff do image based material editing: from a photo enough information can be extracted to change the material of an object, turning it transparent, metallic or anything else. The trick is of course that the eye doesn't notice the pretty big approximations that happen and gladly allows itself to believe in the fake refractions or reflections.

Jiaya Jia, Jian Sun, Chi-Keung Tang, Heung-Yeung Shum do drag-and-drop pasting, essentially a much smarter lasso tool enabling the pasting of objects between images that respects object boundaries without having to force the user to painstakingly mark them.

Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman describe a way of removing camera shake from photos. I remember how awed I was when I learned about Wien filtering and how it could remove movement blur, but that required knowledge of how the camera had shook. This method instead figures out a likely blur kernel and then removes it.

Carsten Rother, Lucas Bordeaux, Youssef Hamadi, Andrew Blake have developed an "autocollage" system that takes a set of images, finds representative images that are different, finds the important objects in them (especially taking care with faces), distributes them over a surface and interpolates together a collage.

So far this is enormously complex, requires expertise and is computationally expensive. Almost none of the techniques discussed in the papers look like anything one could put into the hands of a non-expert working in real-time on an ordinary PC. However, that is going to change over the next few years. My prediction is that many of the techniques will become easily packaged elements of standard photo-editing (or embedded camera) software, just as the once upon a time advanced histogram and anti-redeye methods now are everywhere.

De-blurring of images and automatic color harmonisation won't make everybody a photographic artist, but it is sure going to make everything prettier. Smart pasting and composition tools are going to make everybody a potential photo-retoucher or history editor. Of course, it is also going to produce endless visual cliches like overused Photoshop filters and the font fruit salads. The merging of photo collections into 3D databases that enable editing of object appearances or extraction of 3D models of them, as well as moving through "camera space" using online image databases make photo collections into something far more powerful than just images. They actually start to link the real and virtual worlds.

Add the ubiquity of cellphone cameras and other imagers to this. We have already had a democratization of photo thanks to digital cameras and online photosharing. This gets combined with the folksonomies generated by Web 2.0 and geotagging. Suddenly the Great God Internet gets a significantly better visual perceptive system, where every photo starts to add a bit of meaning to the automatic systems in it rather than just for the humans viewing it. And this is even without any advance image recognition that can tell exactly what is in these pictures.

At the same time all this suggests a need to keep track of the providence of images: when we start creating virtual environments based on images, we will need to know how strongly they have been edited and how. So besides the usefulness for intellectual property of adding authorship metadata that cannot easily be removed or lost, we would want such metadata to tell us how changed the image has become compared to the original base reality. Of course, the Great God Internet can likely deal with a bit of false memories and bias just as we do, our own memories are quite constructed anyway.

So my guess, inspired by these papers, is that in the future images are going to be far more dynamic objects than today in about the same way as text has gone from something fixed and stable to something that is endlessly recombined and automatically collated, summarized, analysed and hyperlinked.

A few other papers

Aude Oliva, Antonio Torralba, Philippe G. Schyns have a paper on hybrid images, where they mix the low frequencies of one image with the high of another one, resulting in an image that changes depending on viewer distance. Faces change mood, gender or species, text can become readable only within a certain range and so on. Very fun and simple trick.

Olga A. Karpenko, John F. Hughes describe a system to create smooth 3D objects from simple line sketches. It is a generalisation of the old work in AI and computer vision on interpreting drawings, now in a differential geometry setting. Still rather limited, but it is getting there.

Also worth mentioning are making panoramas out of series of overlapping photos, rendering rain streaks, force field modelling of crowds and making motions more cartoony.