Near-term focus, robustness, and flow-through effects

post by AshwinAcharya · 2019-02-04T20:58:26.023Z · score: 26 (13 votes) · EA · GW · 8 comments

I recently read Open Phil's 2018 cause prioritization update, which describes their new policy of splitting funding into buckets that make donations according to particular worldviews. They divide worldviews into animal-inclusive vs human-centric, as well as long-term vs near-term focused. I think these two factors cover a lot of the variance with EA viewpoints, so they're pretty interesting to examine. As someone who's generally pretty focused on the long-term, I found this a good jumping-off point for thinking about arguments against it, as well as general concerns about robustness and flow-through effects.

The discussion of near-term focus brings up many good points I've heard against naive long-term-EV maximization. It's hard to predict the future; it means you can't get feedback on how your actions turn out; acting on these predictions has a mixed or bad track record; it can involve confusing moral questions like the value of creating new beings. Aiming simply to preserve civilization runs the risk of carrying bad values into the far future; aiming at improving human values or averting worst cases gives you an even harder target to hit.[1]

On a more intuitive level, it feels uncomfortable to be swayed by arguments that imply you can achieve astronomical amounts of value, especially if you think you're vulnerable to persuasion; if so, a sufficiently silver-tongued person can convince you to do anything. You can also couch this in terms of meta-principles, or outside view on the class of people who thought they'd have an outsize impact on future utility if they did something weird. (I'm not sure what the latter would imply, actually; as Zhou Enlai never said, "What's the impact of the French Revolution? Too soon to tell.")

I think these are mostly quite good objections; if other long-termist EAs are like me, they've mostly heard of these arguments, agreed with them, adjusted a bit, and continued to work on long-term projects of some sort.

The part of me that most sympathizes with these points is the one that seeks robustness and confidence in impact. It's hard for me to adapt to cluster-thinking, which I suspect underlies strong near-termist positions, so I mostly think of this as a constrained optimization problem: either minimizing max badness given some constraint on EV, or maximizing EV - [robustness penalty]. If you don't include a heavy time discount, though, I think it's plausible that this still leads you to "long-term-y" interventions, such as reducing international tension or expanding moral circles of concern [EA · GW]. This is partly due to the difficulty of accounting for flow-through effects. I confess I haven't thought too much about those for short-term human-focused interventions like global health and poverty, but my sense is that unless you optimize fairly hard for good flow-through you're likely to have a nontrivial chance of negative effects.

Another way of thinking about this is to consider what you should have done as an EA at some point in the past. It seems plausible that, while you might not be able to avert nuclear or AI catastrophe directly in 1500, you could contribute to meaningful moral growth, or to differential advances in e.g. medicine (though now we're already in the realm of plausible negative flow-through, via earlier bioweapons -> death, offense-favoring dynamics, lack of norms against "WMDs" as a category). Maybe it's more obvious that ministering to the poor and sick that you could would be the best thing?

I haven't built up much knowledge or deep consideration about this, so I'm quite curious what you guys think. If you support short-termism, is it mainly out of robustness concerns? How do you deal with flow-through uncertainty in general, and how do you conceptualize it, if naive EV maximization is inadequate? Open Phil's post suggests capping the impact of an argument at 10-100x the number of persons alive today, but choosing benchmarks/thresholds/tradeoffs for this kind of thing seems difficult to do in a principled way.

[1] Another object-level point, due to AGB [EA · GW], is that some reasonable base rate of x-risk means that the expected lifespan of human civilization conditional on solving a particular risk is still hundreds or thousands of years, not the astronomical patrimony that's often used to justify far-future interventions. Of course, this applies much less if you're talking about solving an x-risk in a way that reduces the long-term base rate significantly, as a Friendly AI would.

8 comments

Comments sorted by top scores.

comment by Milan_Griffes · 2019-02-05T15:12:42.959Z · score: 5 (5 votes) · EA · GW

Most of my impulse towards short-termism arises from concerns about cluelessness, which I wrote about here [EA · GW].

Holding a person-affecting ethic is another reason to prioritize the short-term; Michael Plant argues for the person-affecting view here [EA · GW].

comment by Pablo_Stafforini · 2019-02-05T19:20:24.681Z · score: 2 (2 votes) · EA · GW
Another object-level point, due to AGB

Would you mind linking to the comment left by that user, rather than to the user who left the comment? Thanks.

comment by AshwinAcharya · 2019-02-05T19:47:07.269Z · score: 4 (4 votes) · EA · GW

He brought this up in a conversation with me; I don't know if he's written it up anywhere.

comment by Max_Daniel · 2019-02-08T23:32:50.078Z · score: 5 (4 votes) · EA · GW

If I recall correctly this paper by Tom Sittler also makes the point you paraphrased as "some reasonable base rate of x-risk means that the expected lifespan of human civilization conditional on solving a particular risk is still hundreds or thousands of years", among others.

comment by Pablo_Stafforini · 2019-02-05T22:12:35.762Z · score: 1 (1 votes) · EA · GW

I see. Thanks.

comment by Denkenberger · 2019-02-08T04:36:50.661Z · score: 4 (5 votes) · EA · GW

I think the argument was written up formally on the forum, but I'm not finding it. I think it goes like if the chance of X risk is 0.1%/year, the expected duration of humans is 1000 years. If you decrease the risk to 0.05%/year, the duration is 2000 years, so you have only added a millennium. However, if you get safe AI and colonize the galaxy, you might get billions of years. But I would argue if you reduce the chance that nuclear war destroys civilization (from which we might not recover), then you increase the chances of getting safe AI and colonization, and therefore you can attribute overwhelming value of mitigating nuclear war.

comment by AGB · 2019-02-09T19:27:22.386Z · score: 6 (5 votes) · EA · GW

> But I would argue if you reduce the chance that nuclear war destroys civilization (from which we might not recover), then you increase the chances of getting safe AI and colonization, and therefore you can attribute overwhelming value of mitigating nuclear war.

For clarity's sake, I don't disagree with this. This does mean that your argument for overwhelming value of mitigating nuclear war is still predicated on developing a safe AI (or some other way of massively reducing the base rate) at a future date, rather than being a self-contained argument based solely on nuclear war being an x-risk. Which is totally fine and reasonable, but a useful distinction to make in my experience. For example, it would now make sense to compare whether working on safe AI directly or working on nuclear war in order to increase the number of years we have to develop safe AI is generating better returns per effort spent. This in turn I think is going to depend heavily on AI timelines, which (at least to me) was not obviously an important consideration for the value of working on mitigating the fallout of a nuclear war!

comment by Denkenberger · 2019-02-12T04:48:52.656Z · score: 2 (1 votes) · EA · GW

I should have said develop safe AI or colonize the galaxy, because I think either one would dramatically reduce the base rate of existential risk. The way I think about the value of nuclear war mitigation being affected by AI timelines is that if AI comes soon, there are fewer years that we are actually threatened by nuclear war. This is one reason I only looked out about 20 years for my cost-effectiveness analysis [EA · GW] for alternate foods versus AI. I think these risks could be correlated, because one mechanism of far future impact of nuclear war is worse values ending up in AI (if nuclear war does not collapse civilization).