Retention in EA - Part I: Survey Data

post by Ben_West · 2021-02-05T19:09:18.450Z · EA · GW · 16 comments

Contents

  Executive Summary
  Background
  Summary of interviews
    
    do people leave EA?
    do people stay in EA?
    
  Prior work
    
    related to EA
  Footnotes
None
16 comments

Executive Summary

Background

Retention is one of CEA’s three major goals. More precisely, we attempt to retain what I will refer to in this sequence as “Engaged EAs” (EEAs). These are people who meet a relatively high bar of engagement: they have, for example, taken the GWWC pledge or have a career largely motivated by EA principles, and are actively engaged in applying EA thinking to those actions. It roughly corresponds to somewhere between “considerable engagement” and “high engagement” on the EA Survey’s self-reported engagement level [EA · GW].

We believe that there are 1,500 - 2,500 people in the world who meet this definition.

The “I” in this post is me (Ben West), and my views don’t necessarily represent the views of CEA. I would view this sequence as a major input into CEA’s thinking on retention over the next few months, but it’s not dispositive, and I wouldn’t be surprised if our thinking six months from now is substantially different.

Summary of interviews

Method

I had 30-60 minute calls with each of ~20 people. In a few cases, they had published Forum articles about retention already, which I also used as a basis for this document (see Prior Work section).

People were selected by asking my coworkers at CEA for recommendations about the people who would be best to speak to about retention, and then asking some of those people who they thought would be best to speak to. I time boxed this to having a proposal (which you’re reading now) publishable by the end of January.

This “Summary of interviews” section is intended to be an unbiased summary of my interviewees’ responses, and the next “Possible Projects” post is where I have a more opinionated synthesis of the results. That being said, interviewees don’t necessarily agree with any of this document.

Why do people leave EA?

ReasonNumber of people who listed that reason (without prompting)
They can’t find a way to contribute11
Cultural fit/cause area disagreement/interpersonal conflict[1]9
Some big life event occurred which made them temporarily step back and then never rejoin7
Burnout/mental health5

Several interviewees clarified that it’s rarely one single thing which causes disengagement, and one suggested that some sort of interpersonal conflict is almost always at least a partial driver of why heavily-engaged people disengage.

Why do people stay in EA?

ReasonNumber of people who listed that reason (without prompting)
Social (friends/community)12
Career benefits7
Moral motivation/wanting impact6
Having responsibilities/recognition for work6
Learning important/interesting things4

Interventions

These are interventions that interviewees either found useful in the past or thought would be useful in the future. They are roughly organized by reasons people leave/stay, but there’s a lot of overlap in these categories and I didn’t try that hard to make this a perfect ontology. Note that this section is trying to faithfully represent what interviewees suggested, and these are not necessarily endorsed by me or CEA.

Prior work

These are some pieces of prior research that informed my work. This is not intended to be a comprehensive list.

EA-Specific

  1. A Qualitative Analysis of Value Drift in EA [EA · GW], Marisa Jurczyk
  2. The community's conception of value drifting is sometimes too narrow [EA · GW], Mati Roy
  3. Concrete Ways to Reduce Risks of Value Drift and Lifestyle Drift [EA · GW], Darius Meißner
  4. Empirical data on value drift [EA · GW], Joey Savoie
  5. More empirical data on 'value drift [EA · GW]', Ben Todd
  6. EA Survey 2018 Series: How Long Do EAs Stay in EA? [EA · GW] Peter Hurford
  7. Value drift and how not to be evil part I, part II, Daniel Gambacorta
  8. Identifying And Mitigating Burnout In Animal Advocacy, Krista Hiddema
  9. Burnout: What is it and how to Treat it [EA · GW]. Elizabeth Van Nostrand
  10. Why do EA events attract more men than women? Focus group data [EA · GW]. Khorton
  11. EA Survey 2019 Series: Community Information [EA · GW]. David Moss

These are some publications which informed my research, but were not about EA. I’m including key quotes from each paper for easy reference.

  1. To eat or not to eat. A comparison of current and former animal product limiters. “Current limiters were more likely to have made a gradual rather than abrupt transition to animal product limitation and were more likely to have joined a vegetarian or vegan group than former limiters. Furthermore, current limiters indicated that their eating pattern was a part of their self identity.”
  2. A sociological look at ex-vegetarians. “During the course of my 14 interviews I ascertained 6 main themes or factors that acted as barriers to vegetarian maintenance. They are: family relationships, identity, guidelines and cleansing, gender roles, peer influence and social networks, and trend participation.”
  3. Recruitment to High-Risk Activism: The Case of Freedom Summer.  “Participants were distinguished from withdrawals primarily on the basis of their (a) greater number of organizational affiliations, (b) higher levels of prior civil rights activity, and (c) stronger and more extensive ties to other participants.”
  4. Social Predictors of Retention in and Switching from the Religious Faith of Family of Origin: Another Look Using Religious Tradition Self-Identification. “many life course transitions involving social disruptions — marriage, divorce, geographical relocation, etc. — significantly increase the chances of religious switching and dropping out [of religion]”.

Footnotes

  1. One person mentioned cause area disagreements, 4 mentioned interpersonal conflict, and 5 mentioned cultural fit.

16 comments

Comments sorted by top scores.

comment by David_Moss · 2021-02-09T11:34:22.766Z · EA(p) · GW(p)

I was a little surprised not to see the 2019 EA Survey mentioned here since we included around 8 questions about these issues that were written and requested by CEA.

  • As in your interviews, the 2019 EA Survey also asked people why people that the respondent knew left EA
  • We also asked whether people’s level of interest in EA had (increased or) decreased and what lead to that change
  • We asked what factors were important for people’s retention
  • And we also asked about what barriers people faced to greater involvement in EA

All of these questions can be analysed looking only at the 926 people who were levels 4-5 on the self-reported EA engagement scale. (They could also be analysed looking only at people who reported actually doing specific things like taking the GWWC pledge or changing career plan largely motivated by EA principles.)

I think a sizable advantage of the interviews is that the responses were to open rather than fixed questions (only the ‘reasons why people’s level of interest changed’ question was open comment). Since the categories included in these questions were not very comprehensive, clear or consistent, the results are probably somewhat arbitrarily skewed towards particular categories, while ignoring others, and may not be easy to interpret. On the other hand, the survey results have the advantage of drawing from a much larger sample of people. More than sheer sample size, the fact that the survey respondents are probably a broader/more representative sample of EAs seems important. It’s not clear how representative the views of people who are thought to know about retention are, or whether there are individuals who know much about what is important for retention in EA.

As it stands, I’m not sure which source of information I prefer, but I think I’d strongly prefer the results from an EA Survey with better questions (for example, we could base survey options around the categories you identified in your interviews or our own qualitative data). 

Reasons why people left EA

These are the reasons mentioned as to why people (known to the respondents) who were levels 4-5 engagement left EA (based on n=178 responses):

Barriers to higher involvement

Barriers to higher involvement are not (necessarily) the same as reasons to stop being involved, but comparing responses to this question to your table, we can see some overlap. (The results below are for level 4-5 EAs only).

Among the open comment responses, personal issues and being too busy, which may correspond to your last two categories were also commonly mentioned, though unfortunately they weren’t offered as fixed category responses (in which case they probably would have received more responses).

One thing that stands out is that lack of EA friends was a much less commonly cited issue for engaged EAs than among less engaged EAs (see below):

This open comment data is for the whole sample, not only EAs who were level 4-5 on the engagement scale (since it was qualitative data which we analysed separately it would take a while to narrow it down to only highly engaged EAs). Still, it highlights that overall personal issues and people being too busy were commonly mentioned in the broader sample (despite not being included among the fixed options) and these plausibly correspond somewhat to the ‘life event’ and ‘burnout/mental health’ categories you mention.

Factors important for retention

These are the factors selected as being important for retention by level 4-5 EAs. Unfortunately the categories are very different to the other questions and the categories that came up in the interview so it's hard to compare.

Reasons for reduced interest

Below are the reasons (based on people’s qualitative comments) for their having less interest in EA than they did 12 months ago. Only ~18% of respondents reported that their level of interest had decreased, so these number are pretty low.

Labels for these categories are included in Appendix 1 [EA · GW] of our post and pasted below.

Replies from: Ben_West
comment by Ben_West · 2021-02-10T22:23:26.343Z · EA(p) · GW(p)

Thanks David – an earlier draft of this post had a table cross-referencing which factors had been listed in which previous work, including EA Survey data, but it got too confusing since every post used its own categorization scheme. I decided to just publish my synthesis without trying to clean that up since I don’t want the perfect to be the enemy of the good, and I appreciate you doing some of that cross-referencing in this comment!

Overall, it seemed like different sources more or less agreed about the most common retention risks, which is encouraging and seems consistent with your analysis in this comment.

And I do see that I linked to the 2018 EA survey but not the 2019 one; I’ve added that as a link now, thanks!

comment by Khorton · 2021-02-06T13:54:09.043Z · EA(p) · GW(p)

I did a couple of focus groups in London about why highly engaged members were involved and that also matches your findings. https://forum.effectivealtruism.org/posts/znuJ2Z48YnEjrGLvA/why-do-ea-events-attract-more-men-than-women-focus-group [EA · GW]

Replies from: Ben_West
comment by Ben_West · 2021-02-08T23:25:29.562Z · EA(p) · GW(p)

Thanks! I remember finding that post helpful when it came out. I've added it to the list above

comment by Benjamin_Todd · 2021-02-10T09:51:58.568Z · EA(p) · GW(p)

Hey Ben, thank you for this!

I had a quick question. With this category:

Cultural fit/cause area disagreement/interpersonal conflict

Cause area disagreement seems fairly different from interpersonal conflict to me. Is there something I've missed about how you're thinking of the categories? How might it look if you broke this group out into sub-categories?

Replies from: Ben_West
comment by Ben_West · 2021-02-10T22:24:11.231Z · EA(p) · GW(p)

Hey Ben, one person mentioned cause area disagreements, 4 mentioned interpersonal conflict, and 5 mentioned cultural fit.

One person mentioned to me that there is almost always some sort of interpersonal conflict involved in driving people out of EA, even if other factors are also important.

Replies from: Benjamin_Todd
comment by Benjamin_Todd · 2021-02-11T12:57:36.865Z · EA(p) · GW(p)

Thanks!

comment by trammell · 2021-02-06T09:53:03.800Z · EA(p) · GW(p)

Sorry, what’s REI work?

Replies from: Ben_West
comment by Ben_West · 2021-02-08T23:25:46.935Z · EA(p) · GW(p)

It stands for Representation, Equity, and Inclusion. It’s an alternative to the more common Diversity, Equity, and Inclusion, which some people prefer because it’s often more accurate to describe an organization’s goals as trying to be representative of some population then it is to say they want  “diversity” per se. I’ve edited the post to clarify this as well.

Replies from: trammell
comment by trammell · 2021-02-09T13:22:29.958Z · EA(p) · GW(p)

Thanks!

comment by Ben_West · 2021-02-05T19:10:38.503Z · EA(p) · GW(p)

As a meta-comment: I’m trying to share more earlier-stage thinking publicly, both to solicit feedback to make my own thinking better and to help others’ investigations. There are obvious downsides to this (e.g. people interpreting my draft thoughts as an authoritative statement from CEA). 

If anyone has feedback about the structure of these posts or the process of me sharing earlier-stage ideas, I would be interested to hear them.

comment by CarolineJ · 2021-02-22T19:04:27.053Z · EA(p) · GW(p)

I randomly found this research and thought it could be interesting to inform yours: https://link.springer.com/article/10.1007/s10551-006-9138-x 

"Abstract

Lapses in ethical conduct by those in corporate and public authority worldwide have given business researchers and practitioners alike cause to re-examine the antecedents to personal ethical values. We explore the relationship between ethical values and an individual’s long-term orientation or LTO, defined as the degree to which one plans for and considers the future, as well as values traditions of the past. Our study also examines the role of work ethic and conservative attitudes in the formation of a person’s long-term orientation and consequent ethical beliefs. Empirically testing these hypothesized relationships using data from 292 subjects, we find that long-term perspectives on tradition and planning indeed engender higher levels of ethical values. The results also support work ethic’s role in fostering tradition and planning, as well as conservatism’s positive association with planning. Additionally, we report how tradition and planning mediate the influence of conservatism and work ethic on the formation of ethical values. Limitations of the study and future research directions, as well as implications for business managers and academics, are also discussed."

Replies from: Ben_West
comment by Ben_West · 2021-02-22T23:45:28.723Z · EA(p) · GW(p)

Thanks! I will check it out.

comment by tamgent · 2021-02-09T19:37:25.610Z · EA(p) · GW(p)

I'd be interested to zoom in on the "can't find a way to contribute" response, and wonder if follow-up questions were asked. It's extra hard because you're another degree removed by asking for group leaders impressions rather than speaking to "leavers" directly. I'd bet that people define contributing in very different ways, and as a result it's pretty unclear what exactly is going wrong here, if anything at all. For example, maybe people can't find a way to contribute via working at EA organisations specifically, but could contribute in highly impactful careers in non-EA organisations (there is a spectrum and I'm oversimplifying). Maybe some "left" to do that. Personally, I wouldn't count this as leaving the EA movement, or at least the model of the EA movement that I have and want to continue having.

But maybe others have a different model?

Replies from: Ben_West
comment by Ben_West · 2021-02-10T22:23:49.482Z · EA(p) · GW(p)

Good question! I, and I think most of the people I talked to, would not consider that leaving the movement. I would look to whether the career decision was motivated by EA considerations, rather than whether the employer officially considers itself "EA".

That being said, I do think some people who left EA might have left because of this misunderstanding: they were not a good fit for some small number of "EA careers" (e.g. 80 K priority paths), and therefore assumed there wasn't a place in EA for them, even though that small list of careers is not a definitive list of what it means to be in EA. 80 K has tried to clarify this (e.g. 

here [EA · GW]), which I think is helpful, but there is probably still more to be done.

comment by Milan_Griffes · 2021-02-12T04:48:21.977Z · EA(p) · GW(p)

What principles & practices does Christianity use to maintain its forms & institutional coherence over many generations? (Islam? Buddhism?)