Utility Cascades

post by Aaron Gertler (aarongertler) · 2020-07-29T07:16:03.528Z · EA · GW · 3 comments

This is a link post for https://academic.oup.com/analysis/article-abstract/doi/10.1093/analys/anaa011/5834865


  My notes
  The good
  The bad

Original paper written by Max Khan Hayward (thanks for engaging with EA, and for doing so much work to further public philosophy!).

Thanks to Sci-Hub for the article access.

Epistemic status: This is a basic summary and commentary that I didn't spend much time on, and my analysis may be too simple // full of holes. I'd love to hear additional thoughts from anyone who finds this interesting!


Utility cascades occur when a utilitarian’s reduction of support for an intervention reduces the effectiveness of that intervention, leading the utilitarian to further reduce support, thereby further undermining effectiveness, and so on, in a negative spiral. 

This paper illustrates the mechanisms by which utility cascades occur, and then draws out the theoretical and practical implications. 

Theoretically, utility cascades provide an argument that the utilitarian agent should sometimes either ignore evidence about effectiveness or fail to apportion support to effectiveness. Practically, utility cascades call upon utilitarians to rethink their relationship with the social movement known as Effective Altruism, which insists on the importance of seeking and being guided by evidence concerning effectiveness. 

This has particular implications for the ‘Institutional Critique’ of Effective Altruism, which holds that Effective Altruists undervalue political and systemic reforms. The problem of utility cascades undermines the Effective Altruist response to the Institutional Critique.

My notes

From the paper's conclusion:

Probably the only way to address the root causes of world misery is through structural reforms – the interventions with the highest utility were they to work are systemic and political. Whether or not they do work is in part dependent on how many people pursue them. But, in a world increasingly influenced by Effective Altruists, the likelihood of people pursuing these reforms is reduced by arguments that this is an inefficient strategy. Perhaps the world would be better, in utilitarian terms, if Effective Altruists would keep quiet about the difficulty of political reform.

The good

The bad

Overall, the paper identifies a real risk that does come up in EA funding, but I think the author is too quick to dismiss EA's chances of reducing that risk in ways other than "selectively ignoring new evidence about effectiveness."


Comments sorted by top scores.

comment by MichaelPlant · 2020-07-29T14:24:36.524Z · EA(p) · GW(p)

I enjoyed reading the paper but was unconvinced any serious problem was being raised (rather than merely a perception of a problem resulting from a misunderstanding).

Put very simply, the structure of the original case is that person chooses option B instead of option A because new information makes option B look better in expectation. It then turns out that option A, despite having lower expected value, produced the outcome with higher value. But there's nothing mysterious about this: it happens all the time and provides no challenge to expected value theory or act utilitarianism. The fact that I would have won if I'd put all my money on number 16 at the roulette table does not mean I was mistaken not to do so.

comment by trammell · 2020-07-29T17:39:25.549Z · EA(p) · GW(p)

One Richard Chappell has a response here: https://www.philosophyetc.net/2020/03/no-utility-cascades.html

comment by Max_Daniel · 2020-07-29T12:33:27.019Z · EA(p) · GW(p)

[Only skimmed Aaron's notes, didn't read the paper, so might be quite off.]

At first glance, this seems like a special case of (e.g.) Parfit's observation in the first part of Reasons and Persons that consequentialist views can imply it'd be better if you didn't follow them, didn't believe in them etc. (similar to how prudential theories can imply that in some situations it'd be better for you if you were 'rationally irrational'). Probably the basic idea was already mentioned by Sidgwick or earlier utilitarians.

I.e. the key insight is that, as people often put it, utilitarianism as a 'criterion of rightness' does not imply we ought to always use utilitarianism (or something that looks like a 'direct' application of it) as a 'decision procedure'. Instead, consequentialist criteria of rightness transform the question which decision procedures to use into a purely empirical one. It's trivial to construct contrived thought experiments where the 'correct' decision procedure is arbitrarily bizarre.

I think this kind of cuts both ways:

  • On one hand, to say something interesting, papers like the above need to engage in empirical investigations: They need to say something about when and how often situations in which it'd be best for the world to use some 'non-consequentialist' decision procedure actually occur. E.g., does this paper give convincing examples for 'utility cascades', or arguments for why we should expect them to be common?
  • On the other hand, it means that (by consequentialist lights) the appropriateness of EA's principles, methods etc. is a purely empirical question as well. They depend on one's normative views as much as they depend on a host of contingent facts, such as the track record of science, how others react to EA, etc.