Popper vs. Macrohistory: what can we really say about the long-term future?

Talk I gave at the Oxford Karl Popper Society:

The quick summary: Physical eschatology, futures studies and macrohistory try to talk about the long-term future in different ways. Karl Popper launched a broadside against historicism, the approach to the social sciences which assumes that historical prediction is their principal aim. While the main target was the historicism supporting socialism and fascism, the critique has also scared away many from looking at the future – a serious problem for making the social sciences useful. In the talk I look at various aspects of Popper’s critique and how damaging they are. Some parts are fairly unproblematic because they demand too high precision or determinism, and can be circumvented by using a more Bayesian approach. His main point about knowledge growth making the future impossible to determine still stands and is a major restriction on what we can say – yet there are some ways to reason about the future even with this restriction. The lack of ergodicity of history may be a new problem to recognize: we should not think it would repeat if we re-run it. That does not rule out local patterns, but the overall endpoint appears random… or perhaps selectable. Except that doing it may turn out to be very, very hard.

My main conclusions are that longtermist views like most Effective Altruism are not affected much by the indeterminacy of Popper’s critique (or the non-ergodicity issue); here the big important issue is how much we can affect the future. That seems to be an open question, well worth pursuing. Macrohistory may be set for a comeback, especially if new methodologies in experimental history, big data history, or even Popper’s own “technological social science” were developed. That one cannot reach certitude does not prevent relevant and reliable (enough) input to decisions in some domains. Knowing which domains that are is another key research issue. In futures studies the critique is largely internalized by now, but it might be worth telling other disciplines about it. To me the most intriguing conclusion is that physical eschatology needs to take the action of intelligent life into account – and that means accepting some pretty far-reaching indeterminacy and non-ergodicity on vast scales.

What is going on in the world?

Inspired by Katja Grace’s list, what do I think are the big narratives, the “plots” that describe a lot of what is going on? Here is a list I hacked together after breakfast.

These are not trends. They are not predictions. They are stories one can tell about what has been happening and what may happen, directing attention towards different domains. Some make for better stories than others. Some urge us to action, others just reflection.

Earth

  • The dance between gravity and entropy: initial high-homogeneity state of universe turns lumpy, producing energy release that drives non-equilibrium processes. Entropy tries to smear out differences, driving further non-equilibria until in the very long run it “wins” through the most convoluted evolution imaginable.
  • The stelliferous era going from the wild galaxy forming youth to the current staid adulthood with peak star formation behind it, peak star number just ahead, and a long middle and old age dominated by red dwarf ellipticals separated by accelerating expansion.
  • A phase transition of molecular matter from non-living forms into living and technological forms, expanding outwards from a small nucleation event.
  • A species with modest intelligence, language and tech ability getting enough of each to make itself the dominant biological and geological force on the planet in a biologically short time. Transitioning from a part of nature to defining nature, making its future evolution contingent on cultural decisions.
  • Humans causing a complex biotic crisis, unique in that the key species involved has become aware of what it is doing.
  • Transition from a biosphere to a cyborg biosphere, where the technosphere is inseparable from the biological and geophysical systems.
  • The transition from scarcity economics to post-scarcity economics.

Water

  • Humanity having being thrust straight into a globalized world, in some cases going from a tribal society to member of the global village in a single generation. Coming to terms with the close presence of vast diversity of nearly any human property causes turbulent transients.
  • The crisis of academia: challenged by new competitors, hemmed in by old structures and sources of money, trying to be the key source or gatekeeper of intellectual capital in society.
  • Rise of the global middle class: much of the world is far more “middle class” than many think. As long as the middle class experiences a rise in living standards or feeling things get better they tend to accept whatever government they have.
  • Rise of the tech billionaires: unlike traditional trade-billionaires not beholden to the standard structure of society, but interested in disruptive new possibilities they can drive. This also goes for non-monetary counterparts (e.g. influence “billionaires”). Individual accumulation of wealth has become much more able to generate individual, idiosyncratic projects.
  • Humanity dealing with attentional heavy tails: traditionally only celebrities had to deal with massive attention, now it can happen to anybody who goes viral (for positive or negative reasons). Humans generally do not handle super-attention well.
  • Split between “somewhere” and “anywhere” people – globalized cosmopolitanism(s) versus nationalist rootedness.
  • Urbanisation shifting: from a steady urbanisation driven by the economics of scale of cities to new styles. Remote working driving at least temporary exurbanisation, forcing accommodations in tax-base losing cities, culturally changing small locations, and people inventing new social structure. Clusters may start forming in new places, or placeless clusters may finally start emerging.
  • Multigenerational society: longer healthy lives mean more generations alive at same time, their different experiences clashing and interacting. Demographic winter drives “beanpole” families with few members per generation but many generations. Time horizons widening due to generations, life extension, better storage, environmental concerns.
  • Controlling the means of production used to be controlling farmers, then factories, now infrastructure. Liberation struggles about infrastructure – platforms, right to repair, open source, micromanufacture, biotechnology, nanotechnology.
  • Broadcasting media created shared cultural and event touchstones, first nationally and then internationally. Network media reduces this impact, making it more subtle for any nation to exert cultural power. Covid-19 may have been the last truly global event.
  • First lifetime servitude to a lord, then a lifetime career, then fluid consulting and gig work. Since average human lifetime considerably longer than average corporate lifetime secure positions require either a lord or long-lived institutions (universities, states, cities).
  • The sexual rights liberations continue (first women, sex in general, homosexuality, trans rights, then furries and technosexuals) while new taboos and puritanisms emerge to channel them into approved forms (debates on prostitution, deepfakes, sexbots…) and make forbidden activities more delightful.
  • Virginia Postrel’s struggle between dynamists (the future is positive, let’s try a diversity of approaches) and stasists (the future is dark and risky, either need reactionary policies or technocracy) playing out across different domains, producing odd bedfellows.
  • The end of the War on Drugs and the start of the Trade Wars on Drugs.
  • Maturation of the social media ecosystem: it takes about 20 years to integrate new technologies into society. Changes to traditional social media function are now resisted by the everyday life and institutional integration, making them less likely to occur. New social media arrive from time to time and will overlay old media.
  • Emergence and death of the lifelong subculture in the broadcast age, followed by the lightweight fandoms in the network age.
  • The battle between traditional small scale degrowth environmentalism and pro-tech “bright greens”.
  • The gradual shift from survival-linked values towards self-expression values, and possibly from traditional values to secular-rational values (the later scale can move far more erratically in response to many factors, and could evolve into new traditionalities if new ideologies became available).
  • The rise of Effective Anything: data-driven, consequentialist analysis pushes for improvement in many domains – and runs straight into vested interests, nonconsequentialist mores, and that “activity X is not about X”. Evidence based medicine case in point. Can lead to shallow A/B testing of minor options and policymaking set by focus group, or actually optimizing important domains.
  • Shifting from a unipolar to multipolar world. Competition between three (?) partial world systems, actually just jockeying for being the central part of the global world system that has emerged. Nobody wants to be periphery, just watch Russia.
  • A shift from society-building ideologies towards a post-ideological risk society. Unhappiness with lack of progress and risk-taking building leads to counter-movements, some that may become real ideologies.
  • Corrosive cynicism undermines any mainstream project attempting to build something, including fixing the cynicism.
  • Systemic risk growth leads to deliberate modularisation of systems (economy, tech, food); only maintainable as long as people remember the last time everything crashed.
  • Rise of religious fundamentalism as reaction to lagging behind, complicated by emergence of integrated and secularized second generation emigrants. Managing intra-family ideological range in a connected world becomes ever more important.
  • The rise of high-tech totalitarian states that may not just lock in their citizens but their leadership. The Algorithm may control the Party.
  • Capitalism getting destroyed by its internal contradictions and reinventing itself, as always.

Air

  • Shift from a text-dominant world to a multimedia world. Influence and status does no longer necessarily accrue to masters of text. Yet search, translation, and curation methods lag and may become AI dependent.
  • The new AI winter when deep learning hype does not immediately deliver everything, followed by a world where at least perception-based jobs are heavily impacted, and possibly many high-prestige and importance domains like design, engineering, planning…
  • The shift from directly experienced reality to mediated reality.
  • Molecular manufacturing going from science fiction to serious futuristic research program, hi-jacking by material science and chemistry, resurgence using protein engineering.
  • A world of rising biotechnological opportunity, risk and knowledge. The biohacker becomes the public hero/villain, while the actual biotechnological institutions grow in power.
  • 70s/80s home computer era seeds a generation comfortable with computers, enabling 90s/00s internet revolution. Wearables, quantified self, neurohacking in 20s seeds enhancement revolution in 40s?
  • Information limits on society radically changed by information technology, with institutions lagging behind. This drives a challenge of explosive change in epistemic systems (how to achieve filtering, authority, trust, etc.), challenges to political institutions (many new forms made possible but not yet invented, tried and tested), as well as personal epistemics (information virtues and habits, new forms of awareness and social links, …)
  • Diagnosis overshoots treatment: better instruments, statistics and AI makes detecting many states and situations easier, but does not necessarily help fix them. Leads to a situation where everybody and everything is diagnosed
  • Identity technology makes everybody and everything identifiable – automatically, remotely, at any time, any purpose.
  • Moore’s law shifting from serial processor performance to parallel performance driving shift in how code works (advantaging things like deep learning, data science, graphics over serial tasks). Next shifts may be towards energy efficiency (may trigger/be triggered by another new device class arriving a la Bell’s law), and/or 3D structures (favouring concentrated computation).
  • The replication crisis in parts of science leading to new methodological orthodoxies, possibly creating better evidence but impairing crisis decision-making and innovation.
  • The shift of energy use from fossil to renewable, linked to shifts in energy infrastructure (transmission, storage, centralization), world politics (loss of importance of middle east), and new forms of problems (fluctuations and instabilities).
  • Technology potentially makes the world as mercurial as the software world, making cultural and institutional constraints now become the main issue.
  • Scientific and technological stagnation as low-hanging fruits are picked and exponentially growing resources needed to make linear progress.
  • Scientific and technological singularity as tools for making progress improve, feeding back on the process and leading to an intelligence explosion (or a capability swell across society).

Fire

  • Existential risk as a way of framing global problems, competing with the old rights based approach. Human security versus national security versus global security.
  • Enlightenment modernists and conservative religious people jointly battling their mutual enemy postmodernism.
  • The broadening moral circles of concern continue to broaden – other tribes, other nations, other races, genders, species, complex systems…
  • Unbundling of key human concepts. Love, sex, reproduction and social roles split apart due to contraception, IVF, online dating etc. Illness, death, pain, ageing and bodies unbundles due to medicine, analgesics, cryonics, ageing interventions, brain emulation… As they unbundle they become legible and subject to individual and social decisions.
  • The search for extraterrestrial intelligence going from pointless (since it is obvious that planets inhabited) to pointless (since they appear dead) to possible (radio telescopes, big galaxy) to exotic and disquieting (exoplanets, Dysonian SETI, great filter considerations) – the choice used to be between loneliness and little green men, but now might include threatening emptiness, postbiology and weirder things.
  • Transition from a mythical worldview (causes follow narrative) to a scientific worldview (causes follow regular, universal rules) to a systems worldview (causes can be complex and entangled, some domains more scientific, some more mythical).
  • A transformation of the world from highly constrained to underconstrained (possibly returning to externally or internally imposed constraints in the future, or expanding into something radically underconstrained).
  • Business as usual: every era has been crucial to its inhabitants, growth and change are perennial, problems are solved or superseded in ways that gives rise to new problems. History does not end.
  • The “hinge of history” where choices, path-dependencies and accidents may set long-term trajectories.