The ever readable Slate Star Codex has a post about checking how accurate the predictions for 2015 were; overall Scott Alexander seems pretty well calibrated. Being a born follower I decided to make a bunch of predictions to check my calibration in a year’s time.
Here is my list of predictions, with my confidence (some predictions obviously stolen):
- No nuclear war: 99%
- No terrorist attack in the USA will kill > 100 people: 95%
- I will be involved in at least one published/accepted-to-publish research paper by the end of 2015: 95%
- Vesuvius will not have a major eruption: 95%
- I will remain at my same job through the end of 2015: 90%
- MAX IV in Lund delivers X-rays: 90%
- Andart II will remain active: 90%
- Israel will not get in a large-scale war (ie >100 Israeli deaths) with any Arab state: 90%
- US will not get involved in any new major war with death toll of > 100 US soldiers: 90%
- New Zeeland has not decided to change current flag at end of year: 85%
- No multi-country Ebola outbreak: 80%
- Assad will remain President of Syria: 80%
- ISIS will control less territory than it does right now: 80%
- North Korea’s government will survive the year without large civil war/revolt: 80%
- The US NSABB will allow gain of function funding: 80%
- US presidential election: democratic win: 75%
- A general election will be held in Spain: 75%
- Syria’s civil war will not end this year: 75%
- There will be no NEO with Torino Scale >0 on 31 Dec 2016: 75%
- The Atlantic basin ACE will be below 96.2: 70%
- Sweden does not get a seat on the UN Security Council: 70%
- Bitcoin will end the year higher than $200: 70%
- Another major eurozone crisis: 70%
- Brent crude oil will end the year lower than $60 a barrel: 70%
- I will actually apply for a UK citizenship: 65%
- UK referendum votes to stay in EU: 65%
- China will have a GDP growth above 5%: 65%
- Evidence for supersymmetry: 60%
- UK larger GDP than France: 60%
- France GDP growth rate less than 2%: 60%
- I will have made significant progress (4+ chapters) on my book: 55%
- Iran nuclear deal holding: 50%
- Apple buys Tesla: 50%
- The Nikkei index ends up above 20,000: 50%
The point is to have enough that we can see how my calibration works.
Looking for topics leads to amusing finds like the predictions of Nostradamus for 2015. Given that language barriers remain, the dead remain dead, lifespans are less than 200, there has not been a Big One in western US nor has Vesuvius erupted, and taxes still remain, I think we can conclude he was wrong or the ability to interpret him accurately is near zero. Which of course makes his quatrains equally useless.
As many people have commented, the best way to do accurate forecasts is to look at everything that is gradually getting worse and predict that path will continue. Pretty much a dead certainty.
(In theory the same should apply to things that are gradually improving, but nothing springs to mind). 🙂
Many events are 50/50. e.g. either Trump will become president or he won’t.
So to improve your forecast score don’t try to guess 50/50 events.
Unexpected events (black swans) are almost (by definition) impossible to predict and these may be the most significant events. So ‘black swan’ predictions will almost always be wrong and be ridiculed. But if you do get one correct you will be hailed as a genius forecaster. Providing the usual bias applies that people forget all the wrong forecasts. 😉
I am not trying to make a list of forecasts that *will* come true. Then I would just have listed eclipses or continental drift predictions. I am not trying to demonstrate that I am great at predicting stuff. I am trying to see how well calibrated I am about my guesses of the future: are half of my 50% confidence predictions right? Are 90% of my 90% confidence predictions right? If not, I need to update my confidence.
And I doubt Trump is a 50/50 event. Either Gary Johnson, the libertarian candidate will win or not, but that does not mean it is 50/50 even if we know nothing else about the situation. The whole point of checking calibration is to see whether your probability estimates are in the right ballpark and you know it.
Forecasting black swans is a moot point almost by definition, but it is my job to think about smart strategies to rein them in. Resilient systems (and arguments) handle surprises better than brittle systems. So figuring out example black swans and robust counterstrategies is a good use of time.
Chinese GDP as measured by the World Bank I would assume, and the OECD countries by OECD statistics. Or simply go by what The Economist prints, since that will be the easiest and most practical solution 🙂
/You nit-picking Hakan