Hey Luis, to be clear, I agree with your post (and using a new name - I like context-specific GPR).

My main proposal which may have been unclear is that you proactively reach out to folks who uses the old definition in their work, in case they don't see this post, and suggest they edit their posts.

I really appreciated this report - especially the attempt to map out the impact of volunteering on the talent pipeline. From the report: 

List of possible impact pathways:

• Better at EA research: Volunteers may become more capable at performing EA research because of their experience performing research, and the support and mentorship they get from the SoGive team.
• Excited about EA: Volunteers may feel more enthusiasm about EA through their work or the sense of community which develops within SoGive
• Community: Building network and sense of community – building networks enables people to achieve more.
• Pathfinding: Volunteers may become more confident about which type of EA career they want (generalist research? Ops? AI alignment?) 
• Introductions: Facilitate recruiters reaching out to people to encourage them to apply for EA roles sooner
  • This is a new impact pathway; we have not connected directly with EA employers before
• Credentialising people makes it easier for employers to make a lower-risk hiring decision. It could happen through enabling a candidate to produce a piece of research an employer can see as evidence of their research capabilities.
  • There is a risk that this could be counterproductive. For example, credentialising a candidate could mean that they displace an equally good or somewhat better candidate; this would be exacerbated if the SoGive-related article has been produced with the support of the SoGive team.
  • Having discussed this with potential EA employers, we see the counterarguments, including the view that seeing high quality work having already been produced is valuable because it significantly derisks an otherwise difficult hiring decision. 

I also appreciated the attempt to actually estimate how much a sample of volunteers was affected by volunteering. I'm more skeptical of some of these estimates than perhaps SoGive is, but I appreciate that the specificity because it gives a concrete place to have a discussion.  

Some thoughts on these estimates : 

• They have "not run these assessments past the relevant people" (in their words) but are partly based on the relevant people's assesments. 
  • It seems in this early stage of evaluating the impact SoGive volunteering, it would be especially important to run this both by the volunteers themselves (especially for 2,3,4) and the orgs (especially for 5)
• I'm pretty uncertain on the value volunteering has, as someone who's both volunteered and worked with many volunteers over the years. At least 3 people who've volunteered on projects I've run now work at 80,000 Hours and CEA. I have not asked any of these orgs whether them volunteering played a role, but I think this would have been a small part of the reason. The main reason was that these folks were just really competent and aligned. I am uncertain what effect volunteering had on retention rates (Excited re EA), it's possible that it did help them. 

Meta: In general, I'd encourage orgs to share more of their novel / new theory of change thinking on the Forum for others to see - not many folks pour through the reports unfortunately, so a lot of good content is missed. 

Hi Luis, thanks for disambiguating between these two defintions! I found this post easy to read, clear and convincing. 

When I was first started working on local priorities research, I definitely intended it to be what you call "Contextualisation Research", and projects that I ran focused on LPR were all CR related. I think (if Yi-Yang agrees with your proposal) it might be helpful to get the original LPR post re-named or add a disclaimer to the top to prevent people who are finding this later assuming that the original post only calls for "LPR" as you define it. 

Names: How about context-specific GPR? (or some variation on this). It takes into account resources / location etc. 

"administration" ? but that sounds quite unappealing, which is why I think the EA movement has used operations. 

• Many would benefit from shorter training programs. While we don’t have the wherewithal to create those now, we hope to move in that direction as the organization grows. We are also offering the option to do an accelerated course that meets bi-weekly instead of monthly.

I would love to know more about what exact feedback you received here, and why you're choosing to deprioritize those for now. It seems like this could be fairly cheap (in terms of resources to run).

As I understand it event apps (maybe not swapcard beyond the basics?) can allow for different categories of attendees - e.g. speakers and attendees. You could add a third category "virtual attendees". It would take some onboarding work, but I could imagine it becoming normal to have folks online but not in-person? 

My view: I think many of the issues raised here are closely related to leadership and management are much harder to address. Examples from the brainstorms above include: 

• Lack of prestige / respect
• Unrealistic expectations
• Improper delegation of tasks
• Exclusion from the decision-making and distance from org strategy

These seem like deeper cultural issues with how certain organisations or members of the community view operations, which is difficult to chang. I’m not really sure what the best way forward there is - I hope that this post could raise awareness about the issue, and hopefully spark conversations at relevant orgs, and make future founders aware of the existing issues in the ecosystem. 

I / Pineapple Operations  have publishing FAQs so that potential candidates have a more realistic picture of working in ops at EA orgs, and why you might consider it. We hope that potential employers and org leaders will also read the FAQ. We will also be publishing a post soon with information for folks who’ve recently joined operations roles at EA orgs. Slightly farther afield,  Elika Somani and I have written a few posts around issues and improvements that could be made to EA events.

I'd be curious to about the team's decision not to publish an annual report in 2021. Based on the 2020 report, it seemed like there were a number of big updates (regarding 80K's impact on the community, and updates in how much impact 80K had) that seemed important to receive updates on. 

(from the full report) 

For product, our priorities in 2023 are:

• Increase investment in research and in improving the content substantively
• Consider releasing an updated version of our out-of-date career guide (which continues to be popular with our audience)
• Take low-hanging fruit in a range of other areas — e.g. updating key articles to be more compelling and to better reflect our current views.

I would be really excited for the web team to spend time revamping the 2017 career guide (I personally found it very useful). From the 2022 user survey (which I recommend reading) : 

• The 2017 career guide is still frequently influential for people who make plan changes — even those who found 80,000 Hours after the 2017 career guide was deprioritised in April 2019. The career guide was mentioned as influential by over 1/4 of CPBCs who found 80,000 Hours in 2021-2022. It was also more likely to be mentioned by plan changes that seemed more impressive. We are now considering releasing an updated version of this career guide.

My hypothesis on why the guide is useful: It teaches skills and approaches, rather than object level views. It gives people tools to compare between options, and encourages people to be more proactive and ambitious in achieving those goals. I think the guide is one of the few complete / comprehensive resources that embodies "EA/EA thinking as a practically useful framework for making decisions about your life". I think we could do a lot more in this space. 

It would be helpful to have information on lag metrics (actual change in user behavior) in the summary and more prominently in the full report. 

My understanding is that plan changes (previously IASPC's then DIPY's) were a core metric 80K used in previous years to evaluate impact. It seems that there has been a shift to a new metric - CBPC's (see below).

From the 2022 user survey :

Criteria Based Plan Changes

1021 respondents answered that 80,000 Hours had increased their impact. Within those, we identified 266 people as having  made a “criteria based plan change” (CBPC) i.e. they answered they’d taken a different job or graduate course of study and there was at least a 30% chance they wouldn’t have if not for 80,000 Hours programs.

The CBPCs are across problem areas. They also vary in how impressive they seem - both in terms of how much of a counterfactual impact 80,000 Hours specifically seems to have had on the change, and how promising the new career choice seems to be as a way of having impact. Most often the website was cited as most important for the change. In reading the responses, we found 19 that seemed the most impressive cases of counterfactual impact and 69 that seemed moderately impressive.

I'd be curious to know the following:

• How does the CBPC differ from the previous metrics you've used? 
• How important is the CBPC metric to informing strategy decisions compared to other metrics? How do you see the CBPC metric interacting with other  metrics like engagement? Are there other lag metrics you think are directly impactful apart from plan changes (e.g. people being better informed about cause areas, helping high impact orgs recruit, etc.?
• Were you on track with predicted CBPC's or not (and were there any predictions on this - perhaps with the old metrics)? The 2021 predictions doc doesn't mention them (as compared to the 2020 prediction doc) 
• By what process do you rate different plan changes? Does predicted impact vary by cause, or other factors? Are these ratings reviewed by advisors external to 80,000 Hours? 

Love the cost per connection idea, I think it would be cool to calculate cost per impact and counterfactual adjusted connection (similar to how animal advocacy careers measures career impact)

Finally, you could make clear that living in a "EA house," attending parties thrown by EAs or with EAs in attendance, etc. do not count as involvement with EA for statistical or action purposes. It's understandable that the original poster may be looking at EA more as a social group, but as a practical matter CEA isn't in a position to take any action against these sorts of individuals. I also don't think it's necessary for EA as a movement to somehow accept responsibility for these individuals.

I think that since CH does let peopel come to them with issues in scenarios like EA group house or events thrown by EAs (but not parties that EAs are in attendance at) , I'm not sure this makes sense. I think the problem is you can't really separate EA as social group vs EA as professional group, especially in the Bay Area. But it can be really hard to figure out the boundaries here. 

Thanks for this! I would broaden this criteria somewhat.

Someone who meets any of the following criteria relevant time period (really like the way you defined the period as "the year in which the incident occured, the year prior, and the following year (to the extent the following year has transpired)"):

• Accepted to EA-branded events such as an EAG(x), retreat, summit or workshop
• Worked (temporarily or full-time) for an EA-aligned organization  or  one that received a grant from a list of funders and could reasonably be considered part of the EA movement (e.g., the Gates Foundation does not count);
• Received a grant, scholarship or fellowship from any of those funders or programs they have funded during the relevant time period;
• Is an organizer or regular member (as defined by attending some number of events in the given period, or is considered as such by the group organizer) in an EA group that has received funding from EA funders 

Rationale:

• Expanded EAG to EA branded events because there are lots of other EA branded events, but didn't include e.g. cause-specific events that might have a large number of non-EAs
• Mentioned temporary work (e.g. contractor) in case it's unclear
• Expanded grant to include scholarship / fellowship
• Expanded to include being a member of a group, because i think it's really important to count that person as invovled in EA. Very uncertain of exact boundaries there.  

Curious if people have tried to estimate the cost of burnout. 

The things I think we should care about, but without numbers or estimates: 

1. How much directly the burnout reduce productivity?
   1. E.g. 6-12 months on average, the longer the grind the longer the burnout
2. The long-term reduction in total max capacity over time or something 
   1. lets say you had 60K hours before burnout, after you have like 40K because you just can't work as hard.
3. How much does burnout increase the likelihood the person doesn't puruse high impact career  (i.e. leaves direct work roles) 
4. What effect does burnout of a person have on their EA network (e.g. their colleagues, friends etc.?)
   1. E.g. if they're on a team, it could marginally increase the chance other team members burn out because they now have more work (+ creating a negative associations to work) 
   2.  e.g. their friends & local community might have a more negative view of the community as one where your friends burnout
5. others?

Strong +1 to paying people for writing concrete, actionable proposals with clear success criteria etc. - but I also think that DEI / reform is just really, really hard, and I expect relatively few people in the community to have 1) the expertise 2) the knowledge of deeper community dynamics / being able to know the current stsances on things. 

(meta point: really appreciate your bio Jason!)

I'm not sure Amber is saying that a community should have no norms at all? It sounded more like she was saying that in the domain of personal relationships there shouldn't be norms.

One could argue, of course, that there are implicit norms around this already in the community, so maybe the argument I think is stronger is something like:

"There are implicit norms in the community around personal relationships already - we should evaluate this norms and figure out if we think they are actually good or not"

(I think this might be a fruitful exercise that it think many individual organizations are doing now as a result of recent events, but is harder to do for the more nebulous "community" and network of people)

Somewhat agree, there are some concerns I have with totally silo'ing community:

1) People might mistag their posts community if they are actually about "building effective altruism". E.g. posts like Community vs Network feel important and useful for professional movement builders (not just local group organizers). I could imagine people accidentally tagging something like this: Evidence from two studies of EA careers advice interventions as a community post or something. 

Perhaps having a moderator occassionally check the community tag could be a good way to ensure that such things don't happen. 

2) There are a number of posts announcing new orgs e.g. Introducing Ayuda Efectiva which would be useful for folks to know to actually help the world directly (e.g. donate in Spain) - and it's also nice to hear about new projects to do good on the frontapage. 

Of course, it's possible that once there is a much more obvious separation folks will know to avoid it. You could also add helpful language when people are drafting a post so they understand what it means for their post to be a community post. (e.g. rather than it just being 1 tag out of 10, you could have it on a separate line to emphasise it more, so people are more aware of it)

I think this gets a bit complicated if e.g. someone makes a really good critique of an existing meta org or something - I think that person should get karma for doing good work and improving our meta efforts (e.g. the post red-teaming CEA was very high quality and took a lot of effort, george rosenfeld's post about spending was similarly well written / made good points). I think it would make sense if the karma were down-weighted or something, but not having it count at all seems bad to me.

(disclaimer: i write / read a lot of meta / community things and find them valuable so might be a bit biased here)

The World Values Survey: https://www.worldvaluessurvey.org/wvs.jsp

You might also want to check out the inglehart welzel cultural map which is a really cool map of changes in values over the last 50ish years. 

I ran a similar survey on race / ethnicity a few years ago and the main issue was that I wasn't sure it was very representative. You'll probably get people who feel very strongly (one way or another) about the issue, so it's important to think about ways to encourage people who don't have strong feelings to participate in the survey and when you write up results keep in mind that saying things like x% of respondents could overstate (in either direction) whatever you're reporting on.

(Due to Jeff's comment I'm now think  it's likely that the both this and OP's suggestion have more downsides than upsides). 

Thanks for sharing this Jeff - your points about discussions on the Forum being more balanced and newcomers finding things that they have doubts about more easily have updated me towards this maybe not being a good idea (I've been uncertain, but overall leaned more on the pro side).

I really like this suggestion! There's also another option of having posts be visible only to logged in users - definitely not private, but at least not public.

One risk of both options is if people know it's less public they may think it's more private than it is (since anyone can make an account / use the forum search of they are really interested). So a key part of the ux would be clearly communicating the change.

Been looking forward to this write up, thank you for sharing this! Really appreciate the concrete suggestions for field builders.

Thank you for responding to the suggestions in detail Ollie - I really appreciate the recent efforts by the Events team to be more transparent and communicate decisions to the community! And also thanks for engaging with our draft feedback earlier in the process. 

• We were trialling out a new application form, so previous responses didn’t load onto the new form. This also meant applications launched later than we wanted to.

I (personally + 1-2 people I know) did find bugs on previous application forms as well fwiw.  
 

I wonder if attendees could be blocked from scheduling virtual evnts (or virtual attendees are blocked from scheduling virtual events) during the conference itself?

Thanks for sharing this Catherine! (and congrats on your first forum post! woop!) 

I really appreciate you sharing your views so openly and where you found you were wrong - and your steps for next time sound really great!  I'm curious if you've come across Charity Entrepreneurship / their resources on intervention prioritization during your research?

Also, I think it would be really cool if there were group intervention-prioritization projects where people could investigate causes they think are neglected / promising. 

-

(Finally, on small grammar thing: when I read the summary I was confused by this sentence:

I then found a source that undermined a key premise: child marriage is clearly harmful across a number of health metrics. 

I might phrase the last sentence to be more clear that the source gave evidence against child marriage having clearly harmful health effects. At first I read it the opposite and was confused. )

And put the blurb in the body of the post!

This is so cool, thanks for doing this!

Top features for me would be:

• multi-criteria search (e.g. I want to see al biosec grants made by Open Philanthropy)
• CSV download to analyze the grants
• standardize cause areas (possibly the first thing since people might not be aware of this)
• consider talking to gwwc is it's possible for people to link their accounts and port giving history from there. You could also consider letting people link and list their donations anonymously.

Also, how are the grants currently being added? Is it just feeding from the database that Vipul/Issa made ?

Thanks! For full transparency / possible underestimating of our numbers (the post was written quickly):

• it includes part-time roles, and we haven't adjusted for this (but should)
• I imagine there's a distribution in the value of each role we placed (I expect some are much more valuable than others) but we haven't figured out how to judge them.
• it doesn't count my time (since I'm unpaid), which might reduce the cost-effectiveness (I'd estimate it was maybe 20% of the billed hours)
• we don't help with vetting other than providing a list of candidates (but this might be particularly useful for folks doing closed hiring rounds)
• we probably need to apply some discount for candidates who would have been hired anyway

So I wouldn't be surprised if the adjusted cost per placement was 2-3x higher right now (although, we expect this to drop since the maintenance costs are very low)

Do you see Bob's question as moral blackmail? I feel like it ta a reasonable question to ask l, but maybe I'm missing something

Really appreciate the thoroughness and transparency of this post and looking forward to digging more into it at some point! Thank you for sharing :)

Thanks Abby! I'm glad you found it useful. Great to get feedback on the quotes too - I had a theory it was more readable (also put them in the consulting and ea guide), so it's great to hear positive feedback!

Thanks for flagging, have changed! That tag used to be called epistemic modesty (looks like the original link just redirected to humility, somewhat confusingly).  

Couldn't see if someone already suggested this but:

• Have a separate field for org name on profile 
• Option to select if you're writing a post on behalf of an organisation or as an individual (this is very important imo often people write posts and it's not clear who they work for) 
  • Auto-tag with org name + "org updates" or similar tag
• Organisation tag shows all the people who've listed org name on their profile 

Related:

• If co-authors add posts to a sequence, have it be considered "canonical" (e.g. when you open the post it automatically shows the sequence)
• Co-authors should automatically receive comment notifications

Curious where the 60% number is coming from - is that because people make 60% of the connections?

It's not just the money, it's also the collective time costs. My guess is that it would cost quite a lot although it would great to get numbers on what % of attendees are locally based for SF & London respectively, and then do rough estimates. 

re my previous comment (and I noticed Nathan brought it up in another comment, but haven't seen it addressed), i'm curious what portion of spend comes from drinks / alcohol?

To add another datapoint, a lot of my trust comes from :

• Stories from more engaged EAs about difficult trade-offs or considerations made my EA leaders / orgs (which were not available publicly - often personal interactions, semi-private facebook discussions, or just observations of behavior over time which are hard to absorb coming in new to a community)
  • This is also not scalable, not accessible to people without networks, and not very reliable (e.g. what if my friend misremembered some facts over the years? or had gaps in their knowledge?)
• Personal experiences with mostly people i consider friends, but also people i have worked with and developed professional working relationships with (many friendships have come out of working relationships) 

If you don't feel comfortable sharing which orgs they were publicly, I'd strongly recommend speaking to the community health team at CEA about this. 

quick reflections on draft amnesty day 

• i wish i'd known about it sooner, but had missed the notification. ideas
  • When you sub to forum newsletter, include a "be notified by email on upcoming forum activities / events" or append them to the forum newsletter. 
  • Have a little banner on top of the forum with upcoming events that is non obtrusive and dismissable that this event is coming up (as someone who goes on the forum ~ daily, i definitely would have noticed that, but the post annoucement got hidden, forum is a newsfeed etc. 
• i wish the window had been longer - a number of people posted a day late. I'd suggest e.g. Wed- Sun for next time. Why?
  • people don't just publish the draft without looking over it at all (at least i didn't)
  • however, they are posting it just with less edits than before. (sometimes my posts are like half-written, such that they wouldn't make sense to post without at least reviewing it a little, or i want to make sure i still 
  • Given the high standards on the forum, i wouldn't be surprised if that was a common way for people to treat draft amnesty day
• I posted 2 posts i knew weren't amazing, and probably wouldn't be very popular, and it wasn't the most positive experience maybe (like Yonaton mentioned) but I'm glad I did it because I felt like I could move on from those posts and focus on my other writing that is more interesting / valuable. in particular, i published a survey that had been in draft for over a year. 
• It gave me a boost of energy to work on posts, and I made progress on other posts that I haven't yet posted but I think are much more valuable than the things i actually posted. (mostly just speeding it up, but i think in this partiuclar case it was p valuable because the posts are particularly relevant right now) 
• I have a 1-2 posts I will probably still post under draft amnesty after the deadline, just because I want people to see it as a work in progress
  • I wonder if a slightly more permanent version of draft amnesty is really the move here to experiment?
  • Like your post could have a 🚧 emoji next to it to imply WIP or something. People would not expect these to be as great, maybe they'd go off the homepage quicker in general 
  • And then maybe when you post the polished version you can link the posts together so there's like an evolution of your thinking which could be helpful for final posts that become really popular then people can see that the person initially wrote something rougher and that was okay too. 

an updated version later with a note on the top.

Do you mean create a new post or update the old one? ( I can see arguments for both) 

Thanks for this awesome comment - when I have a little more time im going to address it properly!

1. Travel grant funding will likely be more restrictive. Previously we’ve funded people to travel to any EA conference they’ve been accepted to. We expect to retain some amount of travel funding moving forwards, but we’ll likely have to be much more conservative about how much we give and who we give it to. When planning around an event, we’d recommend you act under the assumption that we will not be able to grant your travel funding request (unless it has already been approved). 

I think it would be good to clarify how this affects people with less financial stability and/or who are based in countries that are farther away (specifically for EAG's). To me, providing funding for these groups seems like the biggest positives, since they would be least likely to attend without support. [edit: and generally have less opportunities for meeting the international EA community]

these are mostly educated guesses:

• some people (20%?) will arrive later to the conference venue if they need to get food first (30-60m later if you incl. transport, buying, eating outside the venue etc.). for those people, they might have had either 1-1s or casual conversations over breakfast. 
• some people (10?) might be more tired or have a less good experience if they come early but don't have time to get breakfast (so it's not really buying hours, but quality-adjust hours?)
• some people (1%? 5%?) might come in much later if they know there won't be food til lunch

Good to know a counterpoint - I guess different people have different preferences (e.g. breakfast is very important for me to be focused / able to fully participate in the mornings). I think I would still make the trade-off of to have breakfast over snacks (for reasons I mention in response to Neel below)

And now that I think about it, breakfast could just be snacks if it's cheaper. 

1. Catering will likely be cut down. We’ll likely have to stop providing all three of breakfast, lunch, and dinner on each day for our conferences — we still expect to have some food or snacks available, but it’s currently unclear exactly what we’ll be able to provide.

I'd be curious to see a breakdown of catering costs in SF, DC & London if you have them - specifically costs per meal, for drinks, and for snacks and what % of the total budget food has been. Things I think are more or less important (but don't have a sense of where the budget cut-offs are, hence the request for breakdown)

• I think having some kind of breakfast (even if it's just bread/peanut butter/ hummus/bananas) feels super important for essentially buying an extra 1-2 hours per person. If catering doesn't allow for a lighter breakfast and/or doesn't allow outside food etc. [edit: breakfast could also just be snacks/protein bars/etc.]
• From my limited catering knowledge, drinks are usually the most expensive, so if not serving drinks during opening / closing speeches could potentially subsidise breakfasts / more food, that seems worth it. 
• Snacks also feel more optional