EA Forum Reader

The talk is here

[a brief note on altruistic coordination in EA]

1. EA as a community has a distribution over people of values and world-views (which themselves are uncertain and can bayesianly be modeled as distributions).
2. Assuming everyone have already updated their values and world-view by virtue of epistemic modesty, each member of the community should want all the resources of the community to go a certain way.
   • That can include desires about the EA resource allocation mechanism.
3. The differences between individuals undoubtedly causes friction and resentment.
4. It seems like the EA community is incredible in it's cooperative norms and low levels of unneeded politics.
   • There are concerns about how steady this state is.
   • Many thanks to anyone working hard to keep this so!

There's bound to be a massive room for improvement, a clear goal of what would be the best outcome considering a distribution as above, a way of measuring where we're at, an analysis of where we are heading under the current status (an implicit parliamentary model perhaps?), and suggestions for better mechanisms and norms that result from the analysis.

This makes sense, but the type of things that tend to convince me to believe in an ethical theory generally depend a lot on how much I resonate with the main claims of the theory. When I look at the premises in this theorem, none of them seem to be type of things that I care about.

If you want to deal with moral uncertainty with credences, you could assign each of the 3 major assumptions an independent credence of 50%, so this argument would tell you should be utilitarian with credence at least $\frac{1}{2^3}=\frac{1}{8}=12.5\%$. (Assigning independent credences might not actually make sense, in case you have to deal with contradictions with other assumptions.)

On the other hand, pointing out that utilitarians care about people and animals, and they want them to be as happy as possible (and free, or with agency, desire satisfaction) that makes me happy to endorse the theory. When I think about all people and animals being happy and free from pain in a utilitarian world, I get a positive feeling.

Makes sense. For what it's worth, this seems basically compatible with any theory which satisfies the Pareto principle, and I'd imagine you'd also want it to be impartial (symmetry). If you also assume real-valued utilities, transitivity, independence of irrelevant alternatives, continuity and independence of unconcerned agents, you get something like utilitarianism again. In my view, independence of unconcerned agents is doing most of the work here, though.

I want to point out that both assumptions 2, and 1 and 3 together have been objected to by academic philosophers.

Assumption 2 is ex post consequentialism: maximize the expected value of a social welfare function. Ex ante prioriatarianism/egalitarianism means rejecting 2: we should be fair to individuals with respect to their expected utilities, even if this means overall worse expected outcomes. This is, of course, vNM irrational, but Diamond defended it (and see my other comment here [EA · GW]). Essentially, even if two outcomes are equally valuable, a probabilistic mixture of them can be more valuable because it gives people fairer chances; this is equality of opportunity. This contradicts the independence axiom specifically for vNM rationality (and so does the Allais paradox).

Assumptions 1 and 3 together are basically a weaker version of ex ante Pareto, according to which it's (also) better to increase the expected utility of any individual(s) if it comes at no expected cost to any other individuals. Ex post prioritarianism/egalitarianism means rejecting the conjunction of 1 and 3, and ex ante Pareto: we should be more fair to individuals ex post (we want more fair actual outcomes after they're determined), even if this means worse individual expected outcomes.

There was a whole issue of Utilitas devoted to prioritarianism and egalitarianism in 2012, and, notably, Parfit defended prioritarianism in it, arguing against ex ante Pareto (and hence the conjunction of 1 and 3):

When Rawls and Harsanyi appeal to their versions of Veil of Ignorance Contractualism, they claim that the Equal Chance Formula supports the Utilitarian Average Principle, which requires us to act in ways that would maximize average utility, by producing the greatest sum of expectable benefits per person. This is the principle whose choice would be rational, in self-interested terms, for people who have equal chances of being in anyone’s position.

We can plausibly reject this argument, because we can reject this version of contractualism. As Rawls points out, Utilitarianism is, roughly, self-interested rationality plus impartiality. If we appeal to the choices that would be rational, in self-interested terms, if we were behind some veil of ignorance that made us impartial, we would expect to reach conclusions that are, or are close to being, Utilitarian. But this argument cannot do much to support Utilitarianism, because this argument’s premises are too close to these conclusions. Suppose that I act in a way that imposes some great burden on you, because this act would give small benefits to many other people who are much better off than you. If you object to my act, I might appeal to the Equal Chance Formula. I might claim that, if you had equal chances of being in anyone’s position, you could have rationally chosen that everyone follows the Utilitarian Principle, because this choice would have maximized your expectable benefits. As Scanlon and others argue, this would not be a good enough reply.9 You could object that, when we ask whether some act would be wrong, we are not asking a question about rational self-interested choice behind a veil of ignorance. Acts can be wrong in other ways, and for other reasons.

He claimed that we can reject ex ante Pareto ("Probabilistic Principle of Personal Good"), in favour of ex post prioritarianism/egalitarianism:

Even if one of two possible acts would be expectably worse for people, this act may actually be better for these people. We may also know that this act would be better for these people if they are worse off. This fact may be enough to make this act what we ought to do.

Here, by "worse off" in the second sentence, he meant in a prioritarian/egalitarian way. The act is actually better for them, because the worse off people under this act are better off than the worse off people under the other act. He continued:

We can now add that, like the Equal Chance Version of Veil of Ignorance Contractualism, this Probabilistic Principle has a built-in bias towards Utilitarian conclusions, and can therefore be rejected in similar ways. According to Prioritarians, we have reasons to benefit people which are stronger the worse off these people are. According to Egalitarians, we have reasons to reduce rather than increase inequality between people. The Probabilistic Principle assumes that we have no such reasons. If we appeal to what would be expectably better for people, that is like appealing to the choices that it would be rational for people to make, for self-interested reasons, if they had equal chances of being in anyone’s position. Since this principle appeals only to self-interested or prudential reasons, it ignores the possibility that we may have impartial reasons, such as reasons to reduce inequality, or reasons to benefit people which are stronger the worse off these people are. We can object that we do have such reasons.

When Rabinowicz pointed out that, in cases like Four, Prioritarians must reject the Probabilistic Principle of Personal Good, he did not regard this fact as counting against the Priority View. That, I believe, was the right response. Rabinowicz could have added that similar claims apply to Egalitarians, and to cases like Two and Three.

Yeah, it doesn't (obviously) follow. See the appendix on equality.

My guess is I consider the activities you mentioned less valuable than you do. Probably the difference is largest for programming at MIRI and smallest for Hubinger-style AI safety research. (This would probably be a bigger discussion.)

I don't think that peculiarities of what kinds of EA work we're most enthusiastic about lead to much of the disagreement. When I imagine myself taking on various different people's views about what work would be most helpful, most of the time I end up thinking that valuable contributions could be made to that work by sufficiently talented undergrads.

Independent of this, my guess would be that EA does have a decent number of unidentified people who would be about as good as people you've identified. E.g., I can think of ~5 people off the top of my head of whom I think they might be great at one of the things you listed, and if I had your view on their value I'd probably think they should stop doing what they're doing now and switch to trying one of these things. And I suspect if I thought hard about it, I could come up with 5-10 more people - and then there is the large number of people neither of us has any information about.

I am pretty skeptical of this. Eg I suspect that people like Evan (sorry Evan if you're reading this for using you as a running example) are extremely unlikely to remain unidentified, because one of the things that they do is think about things in their own time and put the results online. Could you name a profile of such a person, and which of the types of work I named you think they'd maybe be as good at as the people I named?

It might be quite relevant if "great people" refers only to talent or also to beliefs and values/preferences

I am not intending to include beliefs and preferences in my definition of "great person", except for preferences/beliefs like being not very altruistic, which I do count.

E.g. my guess is that there are several people who could be great at functional programming who either don't want to work for MIRI, or don't believe that this would be valuable. (This includes e.g. myself.)

I think my definition of great might be a higher bar than yours, based on the proportion of people who I think meet it? (To be clear I have no idea how good you'd be at programming for MIRI because I barely know you, and so I'm just talking about priors rather than specific guesses about you.)

---

For what it's worth, I think that you're not credulous enough of the possibility that the person you talked to actually disagreed with you--I think you might doing that thing whose name I forget where you steelman someone into saying the thing you think instead of the thing they think.

I once read a comment on the effective altruism subreddit that tried to explain why aging didn't get much attention in EA despite being so important, and I thought it was quite enlightening.

For background, here's the comment I wrote:

Longevity research occupies an unstable position in the space of possible EA cause areas: it is very "hardcore" and "weird" on some dimensions, but not at all on others. The EAs in principle most receptive to the case for longevity research tend also to be those most willing to question the "common-sense" views that only humans, and present humans, matter morally. But, as you note, one needs to exclude animals and take a person-affecting view to derive the "obvious corollary that curing aging is our number one priority". As a consequence, such potential supporters of longevity research end up deprioritizing this cause area relative to less human-centric or more long-termist alternatives.

Another one for bees: information integration across or generalization between senses.

"Bumble bees display cross-modal object recognition between visual and tactile senses" by Cwyn Solvi, Selene Gutierrez Al-Khudhairy and Lars Chittka.

Humans excel at mental imagery, and we can transfer those images across senses. For example, an object out of view, but for which we have a mental image, can still be recognized by touch. Such cross-modal recognition is highly adaptive and has been recently identified in other mammals, but whether it is widespread has been debated. Solvi et al. tested for this behavior in bumble bees, which are increasingly recognized as having some relatively advanced cognitive skills (see the Perspective by von der Emde and Burt de Perera). They found that the bees could identify objects by shape in the dark if they had seen, but not touched, them in the light, and vice versa, demonstrating a clear ability to transmit recognition across senses.

Many animals can associate object shapes with incentives. However, such behavior is possible without storing images of shapes in memory that are accessible to more than one sensory modality. One way to explore whether there are modality-independent internal representations of object shapes is to investigate cross-modal recognition—experiencing an object in one sensory modality and later recognizing it in another. We show that bumble bees trained to discriminate two differently shaped objects (cubes and spheres) using only touch (in darkness) or vision (in light, but barred from touching the objects) could subsequently discriminate those same objects using only the other sensory information. Our experiments demonstrate that bumble bees possess the ability to integrate sensory information in a way that requires modality-independent internal representations.

The list of questions can be accessed here.

Thanks for your input. Option value struck me as a subject that is not only relevant to EA, but also has not disseminated effectively from the academic literature to a larger audience. It’s very hard to find concrete information on option value outside of the literature. For example, the Wikipedia article on the subject is a garbled mess.