Academia

Existential risk in Gothenburg

This fall I have been chairing a programme at the Gothenburg Centre for Advanced Studies on existential risk, thanks to Olle Häggström. Visiting researchers come and participate in seminars and discussions on existential risk, ranging from the very theoretical (how do future people count?) to the very applied (should we put existential risk on the school curriculum? How?). I gave a Petrov Day talk about how to calculate risks of nuclear war and how observer selection might mess this up, beside seminars on everything from the Fermi paradox to differential technology development. In short, I have been very busy.

To open the programme we had a workshop on existential risk September 7-8 2017. Now we have the videos up of our talks:

I think so far a few key realisations and themes have in my opinion been

(1) the pronatalist/maximiser assumptions underlying some of the motivations for existential risk reduction were challenged; there is an interesting issue of how “modest futures” rather than “grand futures” play a role and non-maximising goals imply existential risk reduction.

(2) the importance of figuring out how “suffering risks”, potential states of astronomical amounts of suffering, relate to existential risks. Allocating effort between them rationally touches on some profound problems.

(3) The under-determination problem of inferring human values from observed behaviour (a talk by Stuart) resonated with the under-determination of AI goals in Olle’s critique of the convergent instrumental goal thesis and other discussions. Basically, complex agent-like systems might be harder to succinctly describe than we often think.

(4) Stability of complex adaptive systems – brains, economies, trajectories of human history, AI. Why are some systems so resilient in a reliable way, and can we copy it?

(5) The importance of estimating force projection abilities in space and as the limits of physics are approached. I am starting to suspect there is a deep physics answer to the question of attacker advantage, and a trade-off between information and energy in attacks.

We will produce an edited journal issue with papers inspired by our programme, stay tuned. Avancez!

 

Aristotle on trolling

Watchful AristotleA new translation of Aristotle’s classic “On trolling” by Rachel Barney! (Open Access)

That trolling is a shameful thing, and that no one of sense would accept to be called ‘troll’, all are agreed; but what trolling is, and how many its species are, and whether there is an excellence of the troll, is unclear. And indeed trolling is said in many ways; for some call ‘troll’ anyone who is abusive on the internet, but this is only the disagreeable person, or in newspaper comments the angry old man. And the one who disagrees loudly on the blog on each occasion is a lover of controversy, or an attention-seeker. And none of these is the troll, or perhaps some are of a mixed type; for there is no art in what they do. (Whether it is possible to troll one’s own blog is unclear; for the one who poses divisive questions seems only to seek controversy, and to do so openly; and this is not trolling but rather a kind of clickbait.)

Aristotle’s definition is quite useful:

The troll in the proper sense is one who speaks to a community and as being part of the community; only he is not part of it, but opposed. And the community has some good in common, and this the troll must know, and what things promote and destroy it: for he seeks to destroy.

He then goes on analysing the knowledge requirements of trolling, the techniques, the types or motivations of trolls, the difference between a gadfly like Socrates and a troll, and what communities are vulnerable to trolls. All in a mere two pages.

(If only the medieval copyists had saved his other writings on the Athenian Internet! But the crash and split of Alexander the Great’s social media empire destroyed many of them before that era.)

The text reminds me of another must-read classic, Harry Frankfurt’s “On Bullshit”. There Frankfurt analyses the nature of bullshitting. His point is that normal deception cares about the truth: it aims to keep someone from learning it. But somebody producing bullshit does not care about the truth or falsity of the statements made, merely that they fit some manipulative, social or even time-filling aim.

It is just this lack of connection to a concern with truth – this indifference to how things really are – that I regard as of the essence of bullshit.

It is pernicious, since it fills our social and epistemic arena with dodgy statements whose value is uncorrelated to reality, and the bullshitters gain from the discourse being more about the quality (or the sincerity) of bullshitting than any actual content.

Both of these essays are worth reading in this era of the Trump candidacy and Dugin’s Eurasianism. Know your epistemic enemies.

Big picture thinking

In Michaelmas term 2015 we ran a seminar series on Big Picture Thinking at FHI. The videos of most seminars are online.

I gave a talk on observer selection effects, and here are my somewhat overdone lecture notes. Covers selection bias, anthropic reasoning, anthropic shadows, nuclear war near misses, physics disasters, the Doomsday Argument, the Fermi Paradox, the Simulation Argument, fine tuning and Boltzmann brains.

Pass the pith helmet, we are going to do epistemology!

Recognized outfittersEuronews has a series on explorers. Most are the kind of people you expect – characters who go off to harsh locations. But they also interviewed me, since I do a kind of exploration too: Anders Sandberg : Explorer of the mind.

“Explorer of the mind” sounds pretty awesome. Although the actual land I try to explore is the abstract and ill-defined spaces of the future, ethics, epistemology and emerging technology.

When I gave the interview I noticed how easy it was to slip into the explorer metaphor: we have a pretty clear cultural idea of what explorers are supposed to be and how their adventures look. Explaining how you do something very abstract like come up with robust decision procedures for judging emerging technology is hard, so it is very easy (and ego-stroking) to describe it as exploration. I think there is some truth in the metaphor, though.

Exploration is basically about trying to gather information about a domain. Some exploration is about the nature of the domain itself, some about its topography/topology, some about the contents of the domain. Sometimes it is about determining the existence of a location or not. In philosophical and mathematical exploration we are partially creating the domain as we go there, but because of consistency (and, sometimes, the need to fit with known facts about the world) it isn’t arbitrary. We might say it is procedurally generated (by a procedure we really would like to know more about!) Since the implications of any logical statement can go infinitely far and we have both limited mental resources and limited logical reach (as per Gödel) there will always be unknown and unknowable things out there. However, most of the unknown is boring and random. Real explorers try to find the important, useful, unique or just aesthetic things – something which again is really hard.

One of the things that fascinate me most about intellectual effort is that different domains have different “topographies”. Solving problems in discrete mathematics is very different from exploring probability or ethics. We know some corners are tough and others easy. Part of it is experience: people have been trying to understand consciousness or number theory for a long time and we see that they have moved less far than the people in geometry. But part of it is also a “feel” for how the landscape works. Getting from one useful result to another one requires different amounts of effort in logic (in my mind a mesa landscape where there are many plateaus of easy walk separated by immense canyons and deserts requiring real genius) and future studies (a thick jungle of fog, mud and creepers where you cannot see far and it is a huge slog to even move, but there is fascinating organisms everywhere within arms reach). Maybe category theory is like an Arctic vista of abstraction where one can move far but there is almost nothing to see. I don’t know, I keep to the mathematical tropics of calculus and geometry.

Another angle of exploration is how much exploitation to do. We want to learn things because of some value of knowledge. Understanding the topography of a domain helps us to direct efforts, so it is valuable at the very least for that (we might of course also value the knowledge about the domain itself). In some domains like engineering or surgery exploitation is so valuable that it tends to dominate: inventors or exploratory engineers/surgeons are rare. I suspect that this means these domains are seriously under-explored: were more people to investigate their limits, topography and nature we would probably learn some very valuable things. Maybe this is the curse of being rich in resources: there is little need to go far, and domains that are less useful get explored more widely. However, when such a broadly explored domain becomes useful it might be colonized on a huge scale (consider the shifts from being just philosophy to becoming proper somewhat mapped disciplines like natural science, economics, psychology etc.)

Of course, some domains are underexplored simply because the tools and opportunities for exploration are expensive or few. We cannot try wild surgical ideas on that many patients, and space engineering is still rather expensive. Coming up with a way of reducing these limitations and opening up their explorative frontiers ought to have big effects. We have seen this happening in scientific disciplines when new instruments arrive (think of the microscope, telescope or computer), or when costs come down (think computers, sequencing). If we could do something similar in abstract domains we might discover awesome things.

One of the best reasons to go exploring is to recognize how fantastic the stuff we already know is. Out there in the unknown there is likely equally fantastic things waiting to be discovered – and there is much more unknown than known.

Just outside the Kardashian index danger zone

Renommée des SciencesMy scientific Kardashian index is 3.34 right now. 

This weeks talkie in the scientific blogosphere is a tongue-in-cheek paper by Neil Hall, The Kardashian index: a measure of discrepant social media profile for scientists (Genome Biology 2014, 15:424). He suggests it as the ratio K=F_a/F_c between actual twitter followers F_a and the one predicted by the number of scientific citations a scholar has,  F_c = 43.3 \cdot C^{0.32} . A higher value than 5 indicates scientists whose visibility exceeds their contributions.

Of course, not everybody took it well, and various debates erupted. Since I am not in the danger zone (just as my blood pressure, cholesterol and weight are all just barely in the normal range and hence entirely acceptable) I can laugh at it, while recognizing that some people may have huge K scores while actually being good scientists – in fact, part of being a good scientific citizen is to engage with the outside world. As Micah Allen at UCL said: “Wear your Kardashian index with pride.”

Incidentally, the paper gives further basis for my thinking about merit vs. fame. There has been debate over whether fame depends linearly on merit (measured by papers published) (Bagrow et al.) or increases exponentially (M.V. Simkin and V.P. Roychowdhury,  subsequent paper). The above paper suggests a cube-root law, more dampened than Bagrow’s linear claim. However, Hall left out people on super-cited papers and may have used a small biased sample: I suspect, given other results, that there will be a heavy tail of super-followed scientists (Neil deGrasse Tyson, anyone?)