Survey of 2018 EA Survey

post by DavidNash · 2018-12-20T17:29:02.631Z · score: 26 (9 votes) · EA · GW · 2 comments

A few months ago, before the results of the 2018 EA survey [EA · GW] came out, I asked if people would be interested in making predictions about the answers, and 35 people completed a survey of the survey. I think that the majority of people answering this were from the group organisers group on Facebook and so probably have a better idea of what the EA community looks like in general, and hopefully by testing our predictions we can improve our calibration.

For each question the respondent was supposed to give lower and upper bounds for which they thought there is an 80% chance that the answer is inside those bounds. All but the first question would have answers as a percentage between 0 and 100.

Here is the program, run by guided track. If you want to see the raw results data for the survey of the survey you can email me using

A few things that might be interesting

The chart below is showing the correct answer percentages and average interval ranges for the 35 participants.

Here is a table looking at each question and seeing what percentage of the 35 gave a correct answer to that question.

Here is a table looking at the median midpoint answer for each question and how that compares to the correct answer, with a positive difference suggesting people thought that the answer would be higher and a negative difference suggesting they thought it would be lower (on average).


Comments sorted by top scores.

comment by Aaron Gertler (aarongertler) · 2018-12-20T23:18:23.395Z · score: 1 (1 votes) · EA(p) · GW(p)

Do your "correct answer" numbers correct for the people who put something like "no answer" or "prefer not to answer"?

I'd guess that most survey respondents were actually guessing something like "percentage of people who give an answer, and for whom the answer is X", even if they were supposed to be guessing "percentage of all people who answer X".

comment by DavidNash · 2018-12-21T20:10:33.609Z · score: 1 (1 votes) · EA(p) · GW(p)

"Correct answer" is maybe not the best wording, it means the answer that Rethink Charity used to describe their results, and is also consistent with how they reported the results in previous years. I should have pointed this out at the beginning of the survey.

In terms of the political questions, that is also shifted by a large response from people supporting libertarian or other political views.