existential risk

Where to hide? Likely with your neighbors

If you are into civilization-scale prepping, this paper is a must read: Jehn, Florian Ulrich, Maximilian Rössler, Luke Kemp, Mike Cassidy, Zachary Kallenborn, Juan Bartolomé García Martínez, Lara Mani, and Matt Boyd. “No place to hide? Regional resilience and vulnerability to global catastrophic risk.” Global Challenges. 10:8 (2026).

The key point is that there is no best place to be in a global disaster: some are better in some circumstances, but none are good against all GCRs.


Fig 8 from the paper: Overview of how often countries were mentioned directly as resilient or vulnerable in cited documents or inferred by the authors of this paper based on factors described in other documents.

Factors that add resilience against some things (e.g. being an island) are at odds with other factors (e.g. volcanism, weak access to global supply chains). Geography is hard to change. Hi-tech societies have resources, but are vulnerable to cyberattacks and HEMP.

There are however synergistic factors that can be improved: governance quality (democratic institutions, low inequality, state capacity, low corruption, and social cohesion are protective), decentralization, preparedness, food system resilience.

(Yes, one can totally imagine an enlightened authoritarian doing great here by harnessing state capacity… except that there are not many Lee Kuan Yews around – try to find examples beside Singapore that reliably has pulled it off. The base rate is low, so one cannot rely on this in a crisis. Democracies are the reliable low variance approach.)

The caricature prepper planning to outlast the End alone with toilet paper and guns was of course wrong, since disaster survival is very much a community thing. The same applies to GCRs: there is no place that can smugly ignore the rest of the world, we need the global community.

Interactive Interface

I really liked the paper, but I was curious about the data. So I quickly vibe-coded an interactive interface.

You can find it at https://aleph.se/references/gcr-resilience-explorer.html – this is the Claude one-shot at interpreting the paper and supplementary data (the eight factors are his too). Very useful for getting a feel for what is in there.

You can change the weightings of the factors to change the ranking. The inference credence is how much you buy the author estimates vs the literature estimates. Net weighted count scoring tends to overweigh much-written-about countries like AUS; balanced divides by # of evidence, letting countries like Iceland shine.

I think what I first noticed is how scarce data is! This is not built on massive data. This is built on scattered reports painstakingly collected. There ought to be much, much more – an obvious follow up project would be to scour the open literature for more.

This is why the regional imputation is doing a lot of work. Which makes sense, since many disasters have a geographical nature and hence regions are hit somewhat similarly.

At least in my visualization it looks like Taiwan has no conflict risk, but that is because the marker just indicates nuclear war issues: makes sense in this context, but easy to misinterpret.

Still, I think the paper findings are solid and a bit obvious (in retrospect). What I really think one could do is to use AI to not just build a deeper database, perform clever imputation of missing data, but start mapping the cross-country links and correlations.

Automating the data-gathering may also allow what the paper points out: the situation is dynamic, and one should see it as a snapshot. Indeed, being able to disregard stale reports or extrapolate trends a bit might be valuable.

This is also a bit of a demonstration of a new way of reading papers: do not just ask for an AI summary, ask for an interactive exploration!

Don’t Worry – It Can’t Happen

(Originally a twitter thread)

When @fermatslibrary  brought up this 1940 paper about why we have nothing to worry about from nuclear chain reactions, I first checked that it was real and not a modern forgery. Because it seems almost too good to be true in the light of current AI safety talk.

Yes, the paper was real: Harrington, J. (1940). Don’t Worry—it Can’t Happen. Scientific American, 162(5), 268-268.

It gives a summary of a recent fission experiment that demonstrate a chain reaction where neutrons released from a split atom induces other atoms to split. The article claimed this caused widespread unease:
“Wasn’t there a dangerous possibility that the uranium would at last become explosive ? That the samples being bombarded in the laboratories at Columbia University, for example, might blow up the whole of New York City ? To make matters more ominous, news of fission research from Germany, plentiful in the early part of 1 939, mysteriously and abruptly stopped for some months. Had government censorship been placed on what might be a secret of military importance ?
The press and populace, getting wind of these possibly lethal goings-on, raised a hue and cry.”
However, physicists were unafraid to of being blown up (and blowing up the rest of the world).
“Nothing daunted, however, the physicists worked on to find out whether or not they would be blown up, and the rest of us along with them.”
Then comes a good description of the recent French experiment in making a self-sustaining chain reaction. The resulting neutrons are too fast to interact much with other atoms, making the net number dwindle despite a few initial induced fission.  And since it runs out, there is no risk.
There are some caveats, but don’t worry, scientific consensus seems to be firmly on the safety side!

“With typical French – and scientific – caution, they added that this was per haps true only for the particular conditions of their own experiment, which was carried out on a large mass of uranium under water. But most scientists agreed that it was very likely true in general.”

This article was 2 years before the Manhattan Project started, so it is unlikely to have been due to deliberate disinformation: it is an honest take on the state of knowledge at the time. Except of course that there was actually a fair bit to worry about soon…
(Foreword from apps.dtic.mil/sti/tr/pdf/ADA , commenting from the coldest part of the Cold War some decades later.)
Note that the concern in the article was merely self-sustaining fission chain reactions, not the atmospheric ignition by fusion discussed later in the Manhattan project and dealt with in the famous (in existential risk circles) report E. J. Konopinski, C. Marvin, and E. Teller, “Ignition of the Atmosphere with Nuclear Bombs,” Los Alamos National Laboratory, LA-602, April 1946. The idea that nuclear chain reactions could blow up the world was actually decades old by this time, a trope or motif that had emerged in the earliest years of the 20th century. Given that the energy content in atomic nuclei was known to be vast, that fissile isotopes occur throughout the Earth, and the possibility of a chain reaction at least conceivable after Leo Szilard’s insight in 1933, this was not entirely taken out of thin air.
Human engineering can change conditions *deliberately* to slow down neutrons with a moderator (making a reactor) or use an isotope where hot neutrons cause fission (the atomic bomb). The natural state is not a reliable indicator of the technical state.
It cannot have escaped the contemporary reader that this is very similar to many claims AI will remain safe. It is not reliable enough to self-improve or perform nefarious tasks well, so the chain reaction runs down. Surely nobody can make an AI moderator or find AI plutonium!
More generally, this seems a common argument failure mode: solid empirical evidence against something within known conditions cannot just be extrapolated reliably outside the conditions. What is needed is for the argument to work is (1) the conditions cannot be changed, (2) the result can be smoothly extrapolated, or (3) the impossibility needs to be relevant to the risk.
For nuclear chain reactions both (1) and (2) were wrong (moderators and plutonium). Arguments that AI will always hallucinate may be true, but that does not mean safety follows, since hallucinating humans (the results apply equally to us) are clearly potentially risky.
I think this is a relative to Arthur C. Clarke’s “failure of nerve” (not following extrapolation implications, often leading to overconfident impossibility claims) and “failure of imagination” (not looking outside the known domain or acknowledging there could be anything out there) he discusses in (1982). Profiles of the Future: An Inquiry to the Limits of the Possible.
Also, when reading the article I thought about my discussions with Tom Moynihan about how many tropes are earlier than the discoveries or events enabling them to become real – in the Scientific American article we already have the planet-destroying explosion and scientists “going dark” for military secrecy.

The funny thing is that this allows enlightened writers to poke fun at those naive people who merely believe in tropes, rather than the real science. The problem is that sometimes we make tropes true.

Thinking long-term, vast and slow

John Fowler "Long Way Down" https://www.flickr.com/photos/snowpeak/10935459325
John Fowler “Long Way Down” https://www.flickr.com/photos/snowpeak/10935459325

This spring Richard Fisher at BBC Future has commissioned a series of essays about long-termism: Deep Civilisation. I really like this effort (and not just because I get the last word):

“Deep history” is fascinating because it gives us a feeling of the vastness of our roots – not just the last few millennia, but a connection to our forgotten stone-age ancestors, their hominin ancestors, the biosphere evolving over hundreds of millions and billions of years, the planet, and the universe. We are standing on top of a massive sedimentary cliff of past, stretching down to an origin unimaginably deep below.

Yet the sky above, the future, is even more vast and deep. Looking down the 1,857 m into Grand Canyon is vertiginous. Yet above us the troposphere stretches more than five times further up, followed by an even vaster stratosphere and mesosphere, in turn dwarfed by the thermosphere… and beyond the exosphere fades into the endlessness of deep space. The deep future is in many ways far more disturbing since it is moving and indefinite.

That also means there is a fair bit of freedom in shaping it. It is not very easy to shape. But if we want to be more than just some fossils buried inside the rocks we better do it.

Existential risk in Gothenburg

This fall I have been chairing a programme at the Gothenburg Centre for Advanced Studies on existential risk, thanks to Olle Häggström. Visiting researchers come and participate in seminars and discussions on existential risk, ranging from the very theoretical (how do future people count?) to the very applied (should we put existential risk on the school curriculum? How?). I gave a Petrov Day talk about how to calculate risks of nuclear war and how observer selection might mess this up, beside seminars on everything from the Fermi paradox to differential technology development. In short, I have been very busy.

To open the programme we had a workshop on existential risk September 7-8 2017. Now we have the videos up of our talks:

I think so far a few key realisations and themes have in my opinion been

(1) the pronatalist/maximiser assumptions underlying some of the motivations for existential risk reduction were challenged; there is an interesting issue of how “modest futures” rather than “grand futures” play a role and non-maximising goals imply existential risk reduction.

(2) the importance of figuring out how “suffering risks”, potential states of astronomical amounts of suffering, relate to existential risks. Allocating effort between them rationally touches on some profound problems.

(3) The under-determination problem of inferring human values from observed behaviour (a talk by Stuart) resonated with the under-determination of AI goals in Olle’s critique of the convergent instrumental goal thesis and other discussions. Basically, complex agent-like systems might be harder to succinctly describe than we often think.

(4) Stability of complex adaptive systems – brains, economies, trajectories of human history, AI. Why are some systems so resilient in a reliable way, and can we copy it?

(5) The importance of estimating force projection abilities in space and as the limits of physics are approached. I am starting to suspect there is a deep physics answer to the question of attacker advantage, and a trade-off between information and energy in attacks.

We will produce an edited journal issue with papers inspired by our programme, stay tuned. Avancez!