June 10, 2013
Is the surveillance working?
On practical ethics I blog about secrecy versus transparency, my take on the ongoing PRISM revelation. Basically, it is not about security vs. freedom, but secrecy vs. accountability.
One question is how much good the surveillance dragnets are doing. Terrorism is a minor cause of death but has enormous salience, so it can motivate enormous and dangerous efforts - which might be unnecessary or even counter-productive.
Massive investments in surveillance have been done post-911, and technology certainly has improved. So, are we getting better at stopping terrorism? And is this evidence that surveillance and other improvements in intelligence are paying off?
I used data about US terrorism incidents from complied by Wm. Robert Johnston; one can quibble about inclusion, data quality and so on, but this is a start for a quick-and-dirty preliminary check. I excluded the criminal incidents, although this is of course debatable - there is no clear border. I used the US since this is a place where the maximum amount of effort presumably is applied.
Looking at the time distribution of incidents the pattern seems to be an intermittent rate up until the mid 1960s, followed by a higher but in the large steady rate.
Zooming in on the recent years, it is hard to see any difference in trend after 911. Had there been an increase or decrease in rate the line would bend from the pre 2001 direction.
Binning the numbers shows that the rate seems to remain the same pre- and post-911.
The ability to thwart seems to go up: the fraction of thwarted incidents does seem to be larger than before, although I have no confidence in any statistical significance here. Squinting, it looks like it is a rising trend in thwarting: society may be getting better at it. But this began long before massive surveillance was possible.
My conclusion is that it doesn't look like post-911 efforts have reduced the rate of incidents. They may have made Americans a bit safer. Whether this improvement has been worth the infringement on civil liberties, immoral aspects of the War on Terror and the cost is of course debatable. If the thwarting probability has increased from 30% to 60% the terrorism mortality rate has been decreased from 72 to 41 deaths per year (using the post-1970 data).
One possibility is that there may be a huge number of plots that are being silently thwarted, and we only see the spillover when they become public. But given that the incident rates seem to have remained roughly constant since the 1960s, this must mean that there has been a corresponding huge increase in terrorism that more or less exactly balances this. This balance seems a bit unlikely. It can be tested (with some nontrivial methodological effort) by comparing to other countries that are not part of the current surveillance boom or began it later: are they seeing huge increases in terrorism while surveilling counterpart countries do not?.
I remain unconvinced that national security surveillance is reducing risk a lot. A bit, yes. But it is likely that a lot of the risk reduction is simply better policing and other mundane methods.
Addendum: The Global Terrorism Database has some data, that also seem to suggest a declining number of incidents per year. However, this decline seem to have been ongoing since the 1970s, so it is likely not due to surveillance. Looking at different regions suggests that trends are all over the place, a more careful study is needed. It might be a nice paper to pick out estimated SIGINT intelligence budgets and regress on the terrorism trends from this and the RAND database to see how much effect they have.
Posted by Anders3 at June 10, 2013 02:20 PM