Settling Titan, Schneier’s Law, and scenario thinking

Charles Wohlforth and Amanda R. Hendrix want us to colonize Titan. The essay irritated me in an interesting manner.

Full disclosure: they interviewed me while they were writing their book Beyond Earth: Our Path to a New Home in the Planets, which I have not read yet, and I will only be basing the following on the SciAm essay. It is not really about settling Titan either, but something that bothers me with a lot of scenario-making.

A weak case for Titan and against Luna and Mars

Basically the essay outlines reasons why other locations in the solar system are not good: Mercury too hot, Venus way too hot, Mars and Luna have too much radiation. Only Titan remains, with a cold environment but not too much radiation.

A lot of course hinges on the assumptions:

We expect human nature to stay the same. Human beings of the future will have the same drives and needs we have now. Practically speaking, their home must have abundant energy, livable temperatures and protection from the rigors of space, including cosmic radiation, which new research suggests is unavoidably dangerous for biological beings like us.

I am not that confident in that we will remain biological or vulnerable to radiation. But even if we decide to accept the assumptions, the case against the Moon and Mars is odd:

Practically, a Moon or Mars settlement would have to be built underground to be safe from this radiation.Underground shelter is hard to build and not flexible or easy to expand. Settlers would need enormous excavations for room to supply all their needs for food, manufacturing and daily life.

So making underground shelters is much harder than settling Titan, where buildings need to be isolated against a -179 C atmosphere and ice ground full with complex and quite likely toxic hydrocarbons. They suggest that there is no point in going to the moon to live in an underground shelter when you can do it on Earth, which is not too unreasonable – but is there a point in going to live inside an insulated environment on Titan either? The actual motivations would likely be less of a desire for outdoor activities and more scientific exploration, reducing existential risk, and maybe industrialization.

Also, while making underground shelters in space may be hard, it does not look like an insurmountable problem. The whole concern is a bit like saying submarines are not practical because the cold of the depths of the ocean will give the crew hypothermia – true, unless you add heating.

I think this is similar to Schneier’s law:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can’t break.

It is not hard to find a major problem with a possible plan that you cannot see a reasonable way around. That doesn’t mean there isn’t one.

Settling for scenarios

Maybe Wohlforth and Hendrix spent a lot of time thinking about lunar excavation issues and consistent motivations for settlements to reach a really solid conclusion, but I suspect that they came to the conclusion relatively lightly. It produces an interesting scenario: Titan is not the standard target when we discuss where humanity ought to go, and it is an awesome environment.

Similarly the “humans will be humans” scenario assumptions were presumably chosen not after a careful analysis of relative likelihood of biological and postbiological futures, but just because it is similar to the past and makes an interesting scenario. Plus human readers like reading about humans rather than robots. All together it makes for a good book.

Clearly I have different priors compared to them on the ease and rationality of Lunar/Martian excavation and postbiology. Or even giving us D. radiodurans genes.

In The Age of Em Robin Hanson argues that if we get the brain emulation scenario space settlement will be delayed until things get really weird: while postbiological astronauts are very adaptable, so much of the mainstream of civilization will be turning inward towards a few dense centers (for economics and communications reasons). Eventually resource demand, curiosity or just whatever comes after the Age of Ems may lead to settling the solar system. But that process will be pretty different even if it is done by mentally human-like beings that do need energy and protection. Their ideal environments would be energy-gradient rich, with short communications lags: Mercury, slowly getting disassembled into a hot Dyson shell, might be ideal. So here the story will be no settlement, and then wildly exotic settlement that doesn’t care much about the scenery.

But even with biological humans we can imagine radically different space settlement scenarios, such as the Gerhard O’Neill scenario where planetary surfaces are largely sidestepped for asteroids and space habitats. This is Jeff Bezo’s vision rather than Elon Musk’s and Wohlforth/Hendrix’s. It also doesn’t tell the same kind of story: here our new home is not in the planets but between them.

My gripe is not against settling Titan, or even thinking it is the best target because of some reasons. It is against settling too easily for nice scenarios.

Beyond the good story

Sometimes we settle for scenarios because they tell a good story. Sometimes because they are amenable to study among other, much less analyzable possibilities. But ideally we should aim at scenarios that inform us in a useful way about options and pathways we have.

That includes making assumptions wide enough to cover relevant options, even the less glamorous or tractable ones.

That requires assuming future people will be just as capable (or more) at solving problems: just because I can’t see a solution to X doesn’t mean it is not trivially solved in the future.

(Maybe we could call it the “Manure Principle” after the canonical example of horse manure being seen as a insoluble urban planning problem at the previous turn of century and then neatly getting resolved by unpredicted trams and cars – and just like Schneier’s law and Stigler’s law the reality is of course more complex than the story.)

In standard scenario literature there are often admonitions not just to select a “best case scenario”, “worst case scenario” and “business as usual scenario” – scenario planning comes into its own when you see nontrivial, mixed value possibilities. In particular, we want decision-relevant scenarios that make us change what we will do when we hear about them (rather than good stories, which entertain but do not change our actions). But scenarios on their own do not tell us how to make these decisions: they need to be built from our rationality and decision theory applied to their contents. Easy scenarios make it trivial to choose (cake or death?), but those choices would have been obvious even without the scenarios: no forethought needed except to bring up the question. Complex scenarios force us to think in new ways about relevant trade-offs.

The likelihood of complex scenarios is of course lower than simple scenarios (the conjunction fallacy makes us believe much more in rich stories). But if they are seen as tools for developing decisions rather than information about the future, then their individual probability is less of an issue.

In the end, good stories are lovely and worth having, but for thinking and deciding carefully we should not settle for just good stories or the scenarios that feel neat.

How much should we spread out across future scenarios?

Robin Hanson mentions that some people take him to task for working on one scenario (WBE) that might not be the most likely future scenario (“standard AI”); he responds by noting that there are perhaps 100 times more people working on standard AI than WBE scenarios, yet the probability of AI is likely not a hundred times higher than WBE. He also notes that there is a tendency for thinkers to clump onto a few popular scenarios or issues. However:

In addition, due to diminishing returns, intellectual attention to future scenarios should probably be spread out more evenly than are probabilities. The first efforts to study each scenario can pick the low hanging fruit to make faster progress. In contrast, after many have worked on a scenario for a while there is less value to be gained from the next marginal effort on that scenario.

This is very similar to my own thinking about research effort. Should we focus on things that are likely to pan out, or explore a lot of possibilities just in case one of the less obvious cases happens? Given that early progress is quick and easy, we can often get a noticeable fraction of whatever utility the topic has by just a quick dip. The effective altruist heuristic of looking at neglected fields also is based on this intuition.

A model

But under what conditions does this actually work? Here is a simple model:

There are $N$ possible scenarios, one of which ( $j$ ) will come about. They have probability $P_i$ . We allocate a unit budget of effort to the scenarios: $\sum a_i = 1$ . For the scenario that comes about, we get utility $\sqrt{a_j}$ (diminishing returns).

Here is what happens if we allocate proportional to a power of the scenarios, $a_i \propto P_i^\alpha$ . $\alpha=0$ corresponds to even allocation, 1 proportional to the likelihood, >1 to favoring the most likely scenarios. In the following I will run Monte Carlo simulations where the probabilities are randomly generated each instantiation. The outer bluish envelope represents the 95% of the outcomes, the inner ranges from the lower to the upper quartile of the utility gained, and the red line is the expected utility.

Utility of allocating effort as a power of the probability of scenarios. Red line is expected utility, deeper blue envelope is lower and upper quartiles, lighter blue 95% interval.

This is the $N=2$ case: we have two possible scenarios with probability $p$ and $1-p$ (where $p$ is uniformly distributed in [0,1]). Just allocating evenly gives us $1/\sqrt{2}$ utility on average, but if we put in more effort on the more likely case we will get up to 0.8 utility. As we focus more and more on the likely case there is a corresponding increase in variance, since we may guess wrong and lose out. But 75% of the time we will do better than if we just allocated evenly. Still, allocating nearly everything to the most likely case means that one does lose out on a bit of hedging, so the expected utility declines slowly for large $\alpha$ .

The $N=100$ case (where the probabilities are allocated based on a flat Dirichlet distribution) behaves similarly, but the expected utility is smaller since it is less likely that we will hit the right scenario.

What is going on?

This doesn’t seem to fit Robin’s or my intuitions at all! The best we can say about uniform allocation is that it doesn’t produce much regret: whatever happens, we will have made some allocation to the possibility. For large N this actually works out better than the directed allocation for a sizable fraction of realizations, but on average we get less utility than betting on the likely choices.

The problem with the model is of course that we actually know the probabilities before making the allocation. In reality, we do not know the likelihood of AI, WBE or alien invasions. We have some information, and we do have priors (like Robin’s view that $P_{AI} < 100 P_{WBE}$ ), but we are not able to allocate perfectly. A more plausible model would give us probability estimates instead of the actual probabilities.

We know nothing

Let us start by looking at the worst possible case: we do not know what the true probabilities are at all. We can draw estimates from the same distribution – it is just that they are uncorrelated with the true situation, so they are just noise.

Utility of allocating effort as a power of the probability of scenarios, but the probabilities are just random guesses. Red line is expected utility, deeper blue envelope is lower and upper quartiles, lighter blue 95% interval. 100 possible scenarios, with uniform probability on the simplex.

In this case uniform distribution of effort is optimal. Not only does it avoid regret, it has a higher expected utility than trying to focus on a few scenarios ( $\alpha>0$ ). The larger N is, the less likely it is that we focus on the right scenario since we know nothing. The rationality of ignoring irrelevant information is pretty obvious.

Note that if we have to allocate a minimum effort to each investigated scenario we will be forced to effectively increase our $\alpha$ above 0. The above result gives the somewhat optimistic conclusion that the loss of utility compared to an even spread is rather mild: in the uniform case we have a pretty low amount of effort allocated to the winning scenario, so the low chance of being right in the nonuniform case is being balanced by having a slightly higher effort allocation on the selected scenarios. For high $\alpha$ there is a tail of rare big “wins” when we hit the right scenario that drags the expected utility upwards, even though in most realizations we bet on the wrong case. This is very much the hedgehog predictor story: ocasionally they have analysed the scenario that comes about in great detail and get intensely lauded, despite looking at the wrong things most of the time.

We know a bit

We can imagine that knowing more should allow us to gradually interpolate between the different results: the more you know, the more you should focus on the likely scenarios.

Optimal alpha as a function of how much information we have about the true probabilities. N=2. — Optimal alpha as a function of how much information we have about the true probabilities (noise due to Monte Carlo and discrete steps of alpha). N=2 (N=100 looks similar).

If we take the mean of the true probabilities with some randomly drawn probabilities (the “half random” case) the curve looks quite similar to the case where we actually know the probabilities: we get a maximum for $\alpha\approx 2$ . In fact, we can mix in just a bit ( $\beta$ ) of the true probability and get a fairly good guess where to allocate effort (i.e. we allocate effort as $a_i \propto (\beta P_i + (1-\beta)Q_i)^\alpha$ where $Q_i$ is uncorrelated noise probabilities). The optimal alpha grows roughly linearly with $\beta$ , $\alpha_{opt} \approx 4\beta$ in this case.

We learn

Adding a bit of realism, we can consider a learning process: after allocating some effort $\gamma$ to the different scenarios we get better information about the probabilities, and can now reallocate. A simple model may be that the standard deviation of noise behaves as $1/\sqrt{\tilde{a}_i}$ where $\tilde{a}_i$ is the effort placed in exploring the probability of scenario $i$ . So if we begin by allocating uniformly we will have noise at reallocation of the order of $1/\sqrt{\gamma/N}$ . We can set $\beta(\gamma)=\sqrt{\gamma/N}/C$ , where $C$ is some constant denoting how tough it is to get information. Putting this together with the above result we get $\alpha_{opt}(\gamma)=\sqrt{2\gamma/NC^2}$ . After this exploration, now we use the remaining $1-\gamma$ effort to work on the actual scenarios.

Expected utility as a function of amount of probability-estimating effort (gamma) for C=1 (hard to update probabilities), C=0.1 and C=0.01 (easy to update). N=100.

This is surprisingly inefficient. The reason is that the expected utility declines as $\sqrt{1-\gamma}$ and the gain is just the utility difference between the uniform case $\alpha=0$ and optimal $\alpha_{opt}$ , which we know is pretty small. If C is small (i.e. a small amount of effort is enough to figure out the scenario probabilities) there is an optimal nonzero $\gamma$ . This optimum $\gamma$ decreases as C becomes smaller. If C is large, then the best approach is just to spread efforts evenly.

Conclusions

So, how should we focus? These results suggest that the key issue is knowing how little we know compared to what can be known, and how much effort it would take to know significantly more.

If there is little more that can be discovered about what scenarios are likely, because our state of knowledge is pretty good, the world is very random, or improving knowledge about what will happen will be costly, then we should roll with it and distribute effort either among likely scenarios (when we know them) or spread efforts widely (when we are in ignorance).

If we can acquire significant information about the probabilities of scenarios, then we should do it – but not overdo it. If it is very easy to get information we need to just expend some modest effort and then use the rest to flesh out our scenarios. If it is doable but costly, then we may spend a fair bit of our budget on it. But if it is hard, it is better to go directly on the object level scenario analysis as above. We should not expect the improvement to be enormous.

Here I have used a square root diminishing return model. That drives some of the flatness of the optima: had I used a logarithm function things would have been even flatter, while if the returns diminish more mildly the gains of optimal effort allocation would have been more noticeable. Clearly, understanding the diminishing returns, number of alternatives, and cost of learning probabilities better matters for setting your strategy.

In the case of future studies we know the number of scenarios are very large. We know that the returns to forecasting efforts are strongly diminishing for most kinds of forecasts. We know that extra efforts in reducing uncertainty about scenario probabilities in e.g. climate models also have strongly diminishing returns. Together this suggests that Robin is right, and it is rational to stop clustering too hard on favorite scenarios. Insofar we learn something useful from considering scenarios we should explore as many as feasible.

Strategies for not losing things

A dear family member has an annoying tendency to lose things – sometimes causing a delaying “But where did I put the keys?” situation when leaving home, sometimes brief panics when wallets go missing, and sometimes causing losses of valuable gadgets. I rarely lose things. This got me thinking about the difference in our approaches. Here are some strategies I seem to follow to avoid losing things.

This is intended more as an exploration of the practical philosophy and logistics of everyday life than an ultimate manual for never losing anything ever.

Since we spend so much of our time in everyday life, the returns of some time spent considering and improving it are large, even if the improvement is about small things.

Concentric layers

I think one of my core principles is to keep important stuff on me. I always keep my phone in my breast pocket, my glasses on my nose, my wallet and keys in my pocket. On travel, my passport is there too. My laptop, travel/backup drive, business cards, umbrella, USB connectors etc. are in the backpack I carry around or have in the same room. If I had a car, I would have tools, outdoor equipment and some non-perishable snacks in the trunk. Books I care about are in my own bookshelf, other books distributed across my office or social environment.

The principle is to ensure that the most important, irreplaceable things are under your direct personal control. The probability of losing stuff goes up as it moves away from our body.

Someone once said: “You do not own the stuff you cannot carry at a dead run.” I think there is a great deal of truth to that. If things turn pear-shaped I should in principle be able to bail out with what I got on me.

A corollary is that one should reduce the number of essential things one has to carry around. Fewer things to keep track of. I was delighted when my clock and camera merged with my phone. The more I travel, the less I pack. Fewer but more essential things also increases the cost of losing them: there is a balance to be made between resilience and efficiency.

Layering also applies to our software possessions. Having files in the cloud is nice as long as the cloud is up, the owner of the service behaves nicely to you, and you can access it. Having local copies on a hard drive means that you have access regardless. This is extra important for those core software possessions like passwords, one time pads, legal documents or proofs of identity – ideally they should be on a USB drive or other offline medium we carry at all times, making access hard for outsiders.

For information redundant remote backup copies also works great (a friend lost 20 years of files to a burglar – and her backup hard drives were next to the computer, so they were stolen too). But backups are very rarely accessed: they form a very remote layer. Make sure the backup system actually does work before trusting it: as a general rule you want to have ways to notice when you have lost something, but remote possessions can often quietly slip away.

Minimax

Another useful principle, foreshadowed above, is minimax: minimize the max loss. Important stuff should be less likely to be lost than less important stuff. The amount of effort I put into thinking up what could go wrong and what to do about it should be proportional to the importance of the thing.

Hence, think about what the worst possible consequence of a loss. A lost pen: annoying if there isn’t another nearby. A lost book: even more annoying. A lost key: lost time, frustration and quite possibly locksmith costs. Lost credit card: hassle to get it blocked and replaced, loss of chance to buy things. Identity theft: major hassle, long term problems. Lost master passwords: loss of online identity and perhaps reputation. Loss of my picture archive: loss of part of my memory.

The rational level of concern should be below the probability of loss times the consequences. We can convert consequences into time: consider how long it would take to get a new copy of a book, get a new credit card, or handle somebody hijacking your Facebook account (plus lost time due to worry and annoyance). The prior probability of loosing books may be about 1%, while identity theft has an incidence of 0.2% per year. So if identity theft would cause a month of work to you, it is probably worth spending a dedicated hour each year to minimize the risk.

Remember XKCDs nice analysis of how long it is rational to optimize daily tasks.

Things you have experience of losing a few times obviously require more thought. Are there better ways of carrying them, could you purchase suitable fasteners – or is their loss actually acceptable? Conversely, can the damage from the loss be mitigated? Spare keys or email accounts are useful to have.

There is of course a fuzzy border between conscientiousness, rationality and worry.

Scenarios

I have the habit of running through scenarios about possible futures whenever I do things. “If I leave this thing here, will I find it again?” “When I come to the airport security check, how do I minimize the number of actions I will need to take to put my stuff in the trays?” The trick is to use these scenarios to detect possible mistakes or risks before they happen, especially in the light of the minimax principle.

Sometimes they lead to interesting realizations: a bank ID device was stored right next to a card with a bank ID code in my wallet: while not enough to give a thief access to my bank account they would pass by two of the three steps (the remaining was a not too strong password). I decided to move the device to another location near my person, making a loss of both the code and the device significantly less probable in a robbery or lost wallet.

The point is not to plan for everything, but over time as you notice them patch holes in your everyday habits. Again, there is a fine line between forethought and worrying. I think the defining feature is emotional valence: if the thought makes you upset rather than “OK, let’s not do that” then you are worrying and should stop. The same for scenarios you cannot actually do anything about.

When something did go wrong, we should think through how to not end up like that again. But it also helps to notice when something nearly went wrong, and treat that as seriously as if it had gone wrong – there are many more teachable instances of that kind than actual mistakes, although they often are less visible.

Poka-yoke

I love the idea of mistake-proofing my life. The trick is to set things up so my behaviour will be shaped to avoid the mistake: the standard example is putting your keys in your shoes or on the door handle, so that it is nearly impossible to leave home without them.

Often a bit of forethought can help construct poka-yokes. When washing clothes, the sound of the machine reminds me that it is ongoing, but when it ends there is no longer a reminder that I should hang the clothes – so I place coat hangers on the front door handle (for a morning wash) or in my bed (for an evening wash) to make it impossible to leave/go to bed without noticing the extra task.

Another mini-strategy is gestalt: put things together on a tray, so that they all get picked up or there will be an easier noticeable lack of a key item. Here the tray acts as a frame forcing grouping of the objects. Seeing it can also act as a trigger (see below). For travel, I have ziploc bags with currency, travel plugs, and bus cards relevant for different destinations.

Habits

One of the main causes of loss is attention/working memory lapses: you put the thing there for a moment, intending to put it back where it belongs, but something interferes and you forget where you placed it.

The solution is not really to try to pay more attention since it is very hard to do all the time (although training mindfulness and actually noticing what you do is perhaps healthy for other reasons). The trick is to ensure that other unconscious processes – habits – help fix the situation. If you always put stuff where it should be by habit, it does not matter that your attention lapses.

The basic approach is to have a proper spot where one habitually puts the particular thing. First decide on the spot, and start putting it there. Then continue doing this. Occasional misses are OK, the point is to make this an automatic habit.

Many things have two natural homes: their active home when you bring them with you, and a passive home when they are not on you. Glasses on your nose or on your nightstand, cellphone in your pocket or in the charger. As long as you have a habit of putting them in the right home when you arrive at it there is no problem. Even if you miss doing that, you have a smaller search space to go through when trying to find them.

One can also use triggers, a concrete cue, to start the action. When going to be, put the wedding ring on the bed stand. When leaving the car, when you are one pace beyond it turn and lock the door. The trick here is that the cue can be visualized beforehand as leading to the action: imagine it vividly, ensuring that they are linked. Every time you follow the trigger with the action they get strengthened.

Another cause of lost items is variability: habits are all about doing the same thing again and again, typically at the same time and place. But I have a fairly variable life where I travel, change my sleep times and do new things at a fairly high rate. Trigger habits can still handle this, if the trigger is tied to some reliable action like waking up in the morning, shaving or going to bed – look out for habits that only make sense when you are at home or doing your normal routine.

One interesting option is negative habits: things you never do. The superstition that it is bad luck to put the keys on the table serves as a useful reminder not to leave them in a spot where they are more likely to be forgotten. It might be worth culturing a few similar personal superstitions to inhibit actions like leaving wallets on restaurant counters (visualize how the money will flee to the proprietor).

Checklists might be overkill, but they can be very powerful. They can be habits, or literal rituals with prescribed steps. The habit could just be a check that the list of everyday objects are with you, triggered whenever you leave a location. I am reminded of the old joke about the man who always made the sign of the cross when leaving a brothel. A curious neighbour eventually asks him why he, such an obviously religious man, regularly visited such a place. The man responds: “Just checking: glasses, testicles, wallet and watch.”

Personality

I suspect a lot just hinges on personality. I typically do run scenarios of every big and small possibility through my head, I like minimizing the number of things I need to carry, and as I age I become more conscientious (a common change in personality, perhaps due to learning, perhaps due to biological changes). Others have other priorities with their brainpower.

But we should be aware of who we are and what our quirks are, and take steps based on this knowledge.

The goal is to maximize utility and minimize hassle, not to be perfect. If losing things actually doesn’t bother you or prevent you from living a good life this essay is fairly irrelevant. If you spend too much time and effort preventing possible disasters, then a better time investment is to recognize this and start living a bit more.

Andart II

Part of Anders' Exoself

scenarios