Helping future researchers to better understand long-term forecasting

post by gabriel_wagner · 2020-11-25T18:55:01.084Z · EA · GW · 1 comments

Epistemic status: a random idea I got while taking a shower, would love to hear if other people have thought about this a bit more seriously. 

Key idea: Longterm forecasts might be relevant for many longtermist interventions. However,  we lack evidence about the accuracy of such forecasts. Could we maybe help future researchers to better understand how good or bad long-term forecasting is, by creating a database of long-term predictions today that they could then evaluate in maybe 500 years?

The starting point of my thoughts was this article [LW · GW] that is skeptical of longtermist interventions because they are based on very speculative forecasts. The argument roughly is this: We can not really be confident in the forecasts that many longtermist interventions are - either explicitly or implicitly - based on. It seems reasonable to assume that our predictions about the very long-run are pretty bad. But it’s not just that they are bad. Maybe even more importantly, we also simply do not really know how bad our predictions are.

One way to presumably solve the issue of uncertainty in our predictions about the long-run is to resort to conservative order of magnitude estimates: something along the lines of “even if my predictions are wrong by a factor of 1000, cause xyz would still be a good thing to do”. The question is, maybe they are not just wrong by 3 orders of magnitude, but by 5. Or 10? Or 50? Really, who knows? We essentially have no empirical track record of how far off long-term forecasting actually is.

Yet, it seems like clarity on this would be extremely helpful in order to evaluate how realistic longtermist interventions are. This made me think of just how nice it would be if some people 500 years ago had created a huge number of predictions about the world of today. Then we could look at these forecasts now and compare them to the real world. The comparison could offer a good baseline estimate as to how feasible it is to make reasonably accurate forecasts about long-term trends. 

Unfortunately, to my knowledge such a project has not been done 500 years ago, so we cannot use it today. But what about creating a database today in which skilled forecasters try to predict trends over the next 50-1000 years? Ideally, this would put researchers in 500 or 1000 years into a much better position to evaluate longterm forecasting accuracy because they could work with some real evidence. 

Projects like seem to have contributed a lot to our understanding of how forecast on timelines of maybe 1-2 years work. Has anyone ever thought about creating a similar project, just that it has timelines of, let’s say, 50-1000 years? Does it maybe already exist?

I could imagine such a project do be beneficial, but also have quite a number of uncertainties about its usefulness, feasibility, and implementation:

Ironically, some of these questions themselves also require some sort of forecast about the far future :p 

Currently, I do not have any concrete plans to actually implement this, but I would love to hear if people think it would be worth considering. 


Comments sorted by top scores.

comment by DonyChristie · 2020-11-29T03:34:51.313Z · EA(p) · GW(p)

The Long Now Foundation started something in this direction: "Long Bets".