February 10, 2009
How to accidentally mislead experts
I'm very fond of information visualisation and communications issues (copies of Tufte everywhere, I read Information Aesthetics every morning...) Beside the enjoyment of good design and the epistemic insights from well presented information there is the equally important matter of how to discern when and how information is misleading. The most common case is that mis-presented information misleads amateurs or outsiders in the field, but here is an interesting example of how it can mislead experts just outside the community producing the information, while amateurs do not fall into the same trap.
Eric K. Drexler has a great post, Nanomachines: How the Videos Lie to Scientists that describe how a certain kind of nanotechnology visualisations may have been very misleading - to the professionals in the field. He writes:
By now, many scientists have seen videos of molecular-scale mechanical devices like the one shown here, and I have no way to know how many have concluded that the devices are a lot of rubbish (and have perhaps formulated an unfortunate corollary regarding your author). The reasons for this conclusion would seem clear and compelling to someone new to the topic, but because I was too close to it, I saw no problem. Familiarity with the actual dynamics of these devices kept me from recognizing that what the videos seem to show is, in one crucial respect, utterly unlike reality.
The problem is that in a typical animation of molecular machinery parts the speed of motion of the machine is on the same order as the thermal vibrations. This produces an exciting image (wobbling atoms, molecular gears turning) but as Drexler notes, someone who understands molecular systems (but are not versed in the assumptions of the machine) will think that there will be a lot of coupling between the machine parts and thermal motion - lots of friction and heat dissipation. Hence they would just assume that the system will be unworkable.
The intended speed of the parts is several orders of magnitude lower, but that would not make an exciting animation (see some examples in Eric's post). So the visual demands of a striking image, together with the difficulty of recognizing that what will be perceived by a certain group is not what is intended, has led to a plethora of misleading visualisations.
The problem is probably pretty common. Not even researchers understand error bars, and in general few readily understand how experts from other disciplines would (mis)interpret their information. Add to this the illusion of transparency bias that makes us overestimate how well we understand how others think, and it is likely that a lot of professionals disagree due to minor miscommunications. It could be small and localized, like this example where a kind of visualisation does the damage, or it could be more fundamental due to deeper discipline differences ("cornucopian" economists vs. "Malthusian" ecologists might be an example).
Non-experts will of course be misled quite often too, but this is widely recognized and people work hard to fix it. Much less work is spent on making sure experts interpret the information right. Good visualisations are about communication. Sometimes that might necessitate making one for the experts (showing nearly motionless gears with thermal motion, or moving gears with blurred atoms) and one for the non-experts (Look! It is a gear! Made of atoms! That wobble!)
Posted by Anders3 at February 10, 2009 01:44 PM