October 31, 2008
Probing the Improbable
My (and Toby's and Rafaela's) paper is now up as [0810.5515] Probing the Improbable: Methodological Challenges for Risks with Low Probabilities and High Stakes on arxiv. We'll see if we get our houses egged this Halloween by annoyed physicists.
Our basic argument ought to be obvious: without any data, a certain risk may be between zero and X. An argument arrives saying that it is less than Y. So if it is right we should now think the risk is at most Y. But the argument itself has a certain chance P of being wrong. If that chance is large compared to Y, the remaining risk XP will be larger than Y, and we should estimate the risk to be about XP+(1-P)Y or less. So arguments showing a risk to be zero are less effective than they would seem at first glance in convincing rational observers to lower their risk estimates. This doesn't matter much in cases where the risks are larger than P, which is true for most everyday risks. But it matters a lot in the case of really unlikely but big risks, like human extinction from physics disasters.
We suggest that there are ways of fixing the problem. But they all take a bit of extra work. A simple way is having several independent risk estimates. The chance of them all being wrong can be made very small - if each P is about one chance in a hundred, combining four of them gets the total risk of a flawed argument down to 10-8. Another approach is to have red team/blue team competitive evaluation where one side tries to find faults in the other side's approach. This is likely to produce more robust estimates. Having estimates that also make falsifiable predictions is very useful.
The sad part is that no matter if this is done or not, it is not going to be very effective in convincing the worried public that the risks have now been rationally bounded. To achieve that requires entirely different sociological strategies. But from a truth-seeking, rational standpoint it is at least nice to figure out how to handle the slipperiness of small probabilities.
Posted by Anders3 at October 31, 2008 05:57 PM