Is EA Growing? Some EA Growth Metrics for 2017

post by Peter_Hurford · 2017-09-05T23:36:39.591Z · score: 19 (17 votes) · EA · GW · Legacy · 27 comments

Contents

  Type of data
  Sep 2014 to Aug 2015
  Sep 2015 to Aug 2016
  Sep 2016 to Aug 2017
  Type of data
  January 2014 to Dec 2014
  January 2015 to December 2015
  January 2016 to December 2016
  Type of data
  EA Survey 2014
  EA Survey 2015
  EA Survey 2017
  Additional Facebook Data
None
27 comments

This post was co-authored by Peter Hurford and Joey Savoie.

 

The EA Survey team at Rethink Charity (including myself) recently released initial data from the 2017 EA Survey and will have more to follow it up. KBog made a comment on the EA Forum noticing that the 2015 EA Survey had 2532 participants, whereas the 2017 EA Survey only had 1837 participants. Does this mean that EA is declining in growth?

It’s hard to say, and the EA Survey team will have more analysis on this, looking deeper at what the data in the EA Survey tells us about EA membership growth and churn, if anything. However, Joey Savoie (unrelated to the EA Survey team) and I (Peter Hurford) were curious to look a bit more at other metrics of potential growth to get a clearer picture. Prior analysis on this has been done by Vipul Naik at the end of 2016 and by Eric Yu at the beginning of 2016 (in a post that I sponsored the creation of).

Joey and I thought it would be a good time to try to re-check the data and see if things have changed. We compiled the following data from various sources to look at EA growth rates.

Type of data

Sep 2014 to Aug 2015

Sep 2015 to Aug 2016

Sep 2016 to Aug 2017

The EA Facebook group (via sociograph.io)[1]

1547 posts

471 authors

920 commenters

1924 reactors[2]

1374 posts

521 authors

1055 commenters

2710 reactors

696 posts[3]

400 authors

1089 commenters

2872 reactors

New Giving What We Can pledgers

618

770

1150

Number of 80,000 Hours impact-adjusted significant career changes

184.8

631.3

1202

Google interest in “effective altruism” (relative scoring)[4]

16.94

28.4

38.2

EA wikipedia data (desktop pageviews)[5]

51,100

70,500

77,800

New EA Newsletter sign-ups

Didn’t really exist[6]

11,167

35,554

EA Forum pageviews

309,961

345,202

390,463

New EA Reddit subscribers[7,8]

308

869

1484

 

Type of data

January 2014 to Dec 2014

January 2015 to December 2015

January 2016 to December 2016

Non-OpenPhil GiveWell donations

$13.3M

$40M

$30-40M (math)

OpenPhil GiveWell donations[9]

$8M

$70M

$50M

Total OpenPhil donations

$30.2M

 

$34.2M

$126.4M

OpenPhil + non-OpenPhil GiveWell donations

$21.3M

$110M

~$80-90M

Total non-OpenPhil donors

9044

14,287

No data available[10]

Total recorded money actually donated (not pledges) from Giving What We Can members

$7.1M

$7.0M[11]

No data available

80,000 Hours Newsletter Subscribers

262

23,000

76,000

 

Type of data

EA Survey 2014

EA Survey 2015

EA Survey 2017

EA Survey respondents

813

2532

1837

Total amount of recorded donations in EA Survey

$5.2M

$6.8M

$9.9M[12]

 

Additional Facebook Data

 As an admin of the EA Facebook group, I was also able to access the built-in statistics, but they only go back 60 days. Right now, as of 30 August 2017, the EA FB group has about 8629 “active users” in the last 60 days as defined by Facebook (user has viewed, posted, commented, or reacted in the group). The last 60 days also saw 99 posts, 2073 comments, and 7042 reactions. 339 members were added in the past 60 days, to a total of 13,407 members right now. Comparing the past sixty days to the past 28 days shows roughly linear growth, but I would not expect to be able to see a trend in only two months of data, even if EA was growing very quickly.

This data is not very useful right now since it only goes back 60 days, but it might be valuable as a time capsule, since a year from now we could re-do this post and compare the numbers for the group as of 2018 to these archived numbers.

 

Commentary 

Overall, it’s hard to get a good sense of a trend from only two to three data points for each trend. It looks like EA is growing somewhat and that the growth rate is somewhat linear, but it varies depending on the source and I would definitely want to see more data to be sure. A lot of the growth rate can vary depending on how much marginal resources organizations like the Center of Effective Altruism put into intentionally growing a particular source (e.g., the EA Newsletter) versus letting a source grow organically (e.g., the EA Forum). 

I don’t really know what kind of growth rate I was expecting prior to seeing this data, so I can’t say if this does or doesn’t support my hypotheses about movement growth. Joey and I are trying to hold off from having firm additional opinions until we can look at the data more. I think it would be ideal to create concrete predictions around where these numbers will be within one year or so (e.g., see herehere, and here). We certainly invite discussion and predictions on this.

 

Acknowledgements 

This post was co-authored by Peter Hurford and Joey Savoie. Thanks to Vipul Naik and Issa Rice for putting together some of the source data used in this report and referring us to the correct places for getting EA Forum and EA Wikipedia pageview data, and thanks Pascal Zimmer for supplying data on the EA Newsletter. Thanks to Kerry Vaughan for providing data on pageviews for the EA Forum in 2017, thanks to Rob Wiblin for some 80,000 Hours statistics, and thanks to Zeke Sherman for pointing out Reddit statistics.

 

Endnotes 

[1]: Sociograph does offer to show member growth over time as a paid feature, which I got via a 14-day free trial, but the data looked incorrect and unusable. 

[2]: I believe “reactors” refer to the number of people who like, haha, love, etc. I am not sure if this includes shares, but I do not think that it does.

[3]: It’s worth noting that the EA Facebook group engages in reasonably heavy moderation of Facebook posts and probably (my intuition as a Facebook mod) rejects about five posts for every one post that reaches the page. Given that moderation may have changed over time, it’s not clear how much to read into this decline in the post count.

[4]: These numbers are not search volumes -- they’re the mean relative “score” for that year, relative to the search volume for the highest day between January 2004 and the end of August 2017.

[5]: See some more wiki data here and here.

[6]: The EA Newsletter was relaunched and first had regular content and marketing in October 2015.

[7]: Both r/EffectiveAltruism and r/smartgiving have been simultaneous EA subreddits since September 2012. r/smartgiving was the default EA subreddit until an intentional migration on 28 Feb 2016. From Sep 2016 - Aug 2017, r/effectivealtruism had +1484 subscribers (1068->2552) and r/smartgiving had +93 (1466->1559); from Sep 2015 - Aug 2016, +869 (199->1068) for r/effectivealtruism and +230 (1236->1466) for r/smartgiving; and from Sep 2014 - Aug 2015, +91 (108->199) for r/effectivealtruism and +308 (928->1236) for r/smartgiving. I will use r/smartgiving numbers for the 2014-2015 period and r/effectivealtruism numbers for all periods after that, to reflect the transition. Note that this growth will therefore involve some inherent double-counting as people who were subscribed on r/smartgiving re-subscribe on r/effectivealtruism. Pageviews for reddit were calculated via http://redditmetrics.com/.

[8]: Reddit admin stats go back a year for pageviews. Looking at this, there were ~65,350 pageviews between Sep 2016 and August 2017. We can keep this number in this comment as a time capsule and then if/when we do a Growth Metrics 2018 we can compare the pageviews to this number.

[9]: Money from the Open Philanthropy Project is counted for the year in which the grant is announced, which may be different from the year the grant is decided or the year the grant money is actually dispersed.

[10]: GiveWell has not yet published their Metrics Report for 2016 data.

[11]: These numbers come from different sources, so I don’t know if they are directly comparable. 

[12]: The EA Survey 2017 recorded donations from both 2015 and 2016. This is 2016 data only. There was $6.7M in donations recorded for 2015 in the 2017 EA Survey.

27 comments

Comments sorted by top scores.

comment by Robert_Wiblin · 2017-09-07T02:08:24.988Z · score: 3 (3 votes) · EA(p) · GW(p)

Regarding the rate of EA Facebook group posts - a year or two ago there was an effort to introduce moderation to slow the rate of low quality posts (often from entirely new people) that added noise and reflected poorly on the group. The lower engagement there seems to be a sign of success and professionalisation rather than slow movement growth.

comment by Peter_Hurford · 2017-09-07T03:04:05.172Z · score: 0 (0 votes) · EA(p) · GW(p)

Definitely. I note that in endnote #3, but it definitely should be pointed out. The heavier moderation started in December 2015.

comment by Robert_Wiblin · 2017-09-07T21:32:30.007Z · score: 0 (0 votes) · EA(p) · GW(p)

Sorry I missed the endnote. I think in that case, where the change is most naturally interpreted as neutral or positive, it's worth giving it a stronger caveat than a footnote, for people just skimming the numbers. :)

comment by Robert_Wiblin · 2017-09-06T07:27:09.963Z · score: 3 (3 votes) · EA(p) · GW(p)

I strongly recommend using the impact-adjusted plan change metric rather than the unadjusted one for 80,000 Hours. Those figures:

Sep 2014 to Aug 2015 - 184.8

Sep 2015 to Aug 2016 - 631.3

Sep 2016 to Aug 2017 - 1202

There's also our newsletter growth. New subscribers each year:

2014 - 262

2015 - 23,000

2016 - 76,000

2017 so far - 57,000.

comment by Peter_Hurford · 2017-09-06T11:28:30.677Z · score: 1 (1 votes) · EA(p) · GW(p)

Ok, done. Thanks. Impact-adjusted numbers are fairer, since that is what you are actually targeting, though there is some subjectivity in the impact adjusting process.

comment by MichaelPlant · 2017-09-06T12:34:32.870Z · score: 3 (3 votes) · EA(p) · GW(p)

Couldn't you put in both types of data so readers can draw their own conclusions?

comment by Robert_Wiblin · 2017-09-06T17:24:22.450Z · score: 2 (2 votes) · EA(p) · GW(p)

Thanks. It's not so much that it's a matter of what we're targetting - I think some of these plan change are more than 100x more valuable than others. If you take the raw number of customers a company has, when some customers spend 100x more than others, the total number of customers could look good while the business is actually shrinking. Our raw plan change figures are dominated by the smallest, least important plan changes so could be quite misleading in future.

comment by allister clark · 2018-11-21T19:12:44.419Z · score: 2 (2 votes) · EA(p) · GW(p)

Hi, I was thinking that the sum of local EA group facebook numbers might serve as a good proxy for the size of the EA movement per se.

I am part of the local team building an EA presence in Brussels, and for local groups this is a useful metric to measure success in one sense i.e. as an easy-to-generate quantification of the size of our group at any one time (as well as the historical trend) as well as for the movement as a whole.

It could also be combined with other metrics e.g.

with impact (e.g. GiveWell donations, 80k Hours career changes) to assess communications effectiveness.

with time (of the local EA team) and cost invested to achieve to assess operational effectiveness.

If compared across local groups, the metrics would highlight local success stories and potentially where groups might need help.

Regards,

Allister.

comment by David_Moss · 2018-11-21T19:47:56.872Z · score: 3 (2 votes) · EA(p) · GW(p)

the sum of local EA group facebook numbers might serve as a good proxy for the size of the EA movement per se

This is an interesting proposition, but one thing that will limit its usefulness, I think, is that lots of EAs are members of multiple local group Facebook groups, presumably either to show support or out of interest in their content. Aside from that, many members of the online groups appear to be not engaged in the local EA community (and perhaps not really engaged in EA at all): for example, EA London has around 2000 members of the Facebook group but many fewer people who are actively engaged and attend events and so.

It could also be combined with other metrics e.g. with impact (e.g. GiveWell donations, 80k Hours career changes) to assess communications effectiveness. with time (of the local EA team) and cost invested to achieve to assess operational effectiveness. If compared across local groups, the metrics would highlight local success stories and potentially where groups might need help.

The Local Groups Survey [EA · GW] did this to some extent: measuring (self-reported) number of group members, Pledges, Career changes, funds raised or donations influenced, among other things. We don't publicly release a breakdown of particular groups, of course, but we did look at the correlations between different variables and performance on different metrics. As you'd expect there was a fairly good correlation between success on different metrics, though with plenty of exceptions. I agree it would be valuable to have more systematic investigation of group performance of this kind to identify trends and where things seem to be working particularly well or not well.

comment by weeatquince · 2017-09-07T11:33:52.266Z · score: 2 (2 votes) · EA(p) · GW(p)

Hi, In case helpful for considering the additional Facebook information, I have a bunch of data on EA social media presence to help me compare growth in London to other locations, including a lot of downloaded Sociograph data from 2016.

For example the EA Facebook group size over the last year:

03/06/2016 _ 10263

13/01/2017 _ 12070

10/06/2017 _ 12,953

Obviously you'd expect these things to grow as people join then do not leave (but might ignore it), even if the movement was shrinking.

comment by Zeke_Sherman · 2017-09-06T14:36:39.352Z · score: 2 (2 votes) · EA(p) · GW(p)

The effective altruism subreddit is growing in traffic: https://i.imgur.com/3BSLlgC.png (August figures are 2.5k and 9.5k)

The EA Wikipedia page is not changing much in pageviews: https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&start=2015-07&end=2017-08&pages=Effective_altruism

comment by vipulnaik · 2017-09-06T18:50:01.606Z · score: 0 (0 votes) · EA(p) · GW(p)

The subreddit stats used to be public (or rather, moderators could choose to make them public) but that option was removed by Reddit a few months ago.

https://www.reddit.com/r/ModSupport/comments/6atvgi/upcoming_changes_view_counts_users_here_now_and/

I discussed Reddit stats a little bit in this article: https://www.wikihow.com/Understand-Your-Website-Traffic-Variation-with-Time

comment by Peter_Hurford · 2017-09-06T15:04:35.197Z · score: 0 (0 votes) · EA(p) · GW(p)

Thanks. Good idea to include the subreddit. How do you get those stats? We can add traffic from the r/smartgiving subreddit too (r/effectivealtruism precursor) to go back prior to 2016.

I have EA Wikipedia pageview data in my post already. :)

comment by Zeke_Sherman · 2017-09-06T16:31:17.240Z · score: 1 (1 votes) · EA(p) · GW(p)

You can find the stats by going to the right of the page in moderation tools and clicking "traffic stats". They only go back a year though. Redditmetrics.com should show you subscriber counts from before that, but not activity.

comment by Peter_Hurford · 2017-09-06T17:38:30.625Z · score: 1 (1 votes) · EA(p) · GW(p)

Thanks, I added the Reddit stats to the article!

comment by [deleted] · 2018-02-21T13:39:07.441Z · score: 1 (1 votes) · EA(p) · GW(p)

Vaguely interesting data point: In January 2018 I listed ~240 people in London who I considered - based on personal acquaintance - to be members of the EA community ("EAs"). Compare that with only 66 Londoners taking the EA Survey 2017.

Lots of caveats that mean you shouldn't assume the current global EA community has ~6,680 (=240/66*1837) members e.g. although I'm now a group leader, I imagine there are dozens/hundreds of EAs in London who I don't know; I expect the London community is growing faster than others because it's a popular destination for young, talented, ambitious people; I don't know what proportion of survey takers identified as EAs; I think EA London promoted the survey more than any other group did etc.

Still. Vaguely interesting data point.

comment by [deleted] · 2017-10-27T18:20:44.392Z · score: 1 (1 votes) · EA(p) · GW(p)

In mid-March 2017, EA Wikipedia page views hit 3-4 times as many views as any day in preceeding or succeeding months...What happened in March?

comment by kbog · 2017-10-27T18:22:54.734Z · score: 0 (0 votes) · EA(p) · GW(p)

I think it was one of those big podcasts, like MacAskill on Joe Rogan's show or something like that.

comment by vipulnaik · 2017-09-06T18:47:16.772Z · score: 1 (1 votes) · EA(p) · GW(p)

I have been using PredictionBook for recording predictions related to GiveWell money moved; see http://effective-altruism.com/ea/xn/givewell_money_moved_in_2015_a_review_of_my/#predictions-for-2016 for links to the predictions. Unfortunately searching on PredictionBook itself does not turn up all the predictions because they use Google, which does not index all pages or at least doesn't surface them in search results.

comment by Peter_Hurford · 2017-12-04T15:02:43.714Z · score: 0 (0 votes) · EA(p) · GW(p)

Note to self for next year: ACE tracks money moved.

comment by Benjamin_Todd · 2017-11-05T03:33:09.447Z · score: 0 (0 votes) · EA(p) · GW(p)

These data are helpful, but they somewhat miss what we really care about.

First, we care more about growth in impact on top causes than the numbers of people involved etc.

I think the key story here is that Open Phil has ramped up donations from $30m to over $120m (4-fold growth in a year), and is expected to increase that several times more in the next few years, but this would be easy to miss in the presentation above.

I'd also prefer to try to look at even more fundamental measures of progress in the top causes, though this is hard since they'll be qualitative. For instance, I think taking AI risk mainstream as a field 2015-2017 has been a huge success.

When it comes to measures of the number of people engaging in EA, we also have to be careful. Many in the community have stepped back from increasing the number of people involved & media outreach, towards increasing the quality of engagement, as reflected in our recent survey: https://80000hours.org/2017/11/talent-gaps-survey-2017/

See some defence of this idea here: https://www.centreforeffectivealtruism.org/blog/the-fidelity-model-of-spreading-ideas/

This means some of these declines in web hits and so on might actually be intentional. We could have easily driven up visits to the EA Forum or wikipedia page if we had tried.

That said, unfortunately, we don't yet have a good measure of the quality of engagement, so it's hard to know if that strategy is working either (though I feel like we're making good progress within 80k at least).

comment by jayd · 2018-01-04T19:00:07.867Z · score: 2 (2 votes) · EA(p) · GW(p)

I think the key story here is that Open Phil has ramped up donations from $30m to over $120m (4-fold growth in a year), and is expected to increase that several times more in the next few years, but this would be easy to miss in the presentation above.

That's not EA as a movement though.

comment by Robert_Wiblin · 2017-09-06T07:18:40.562Z · score: 0 (2 votes) · EA(p) · GW(p)

You should really add non-'global health and development' OpenPhil grants as that's now almost as much as all the other donations here listed together (e.g. $105m since the start of 2017, vs $60m through all of 2016 and little before that). Easy to get all the data here: http://www.openphilanthropy.org/giving/grants .

Adding in that the growth rate looks much faster.

comment by Peter_Hurford · 2017-09-06T11:40:16.355Z · score: 0 (0 votes) · EA(p) · GW(p)

Good idea. I added that in.