I don’t know the answer, but I suggest asking this on LessWrong as well.

Some thoughts I had while reading that I expect you'd agree with:

1. There is probably a lot of overlap in the kinds of interventions that (some) AI safety folks would be on board with and the kinds of interventions that (some) AI ethics folks would be on board with. For example, it seems like (many people in) both groups have concerns about the rate of AI progress and would endorse regulations/policies that promote safe/responsible AI development.
2. Given recent developments in AI, and apparent interest in regulation that promotes safety, it seems like now might be a particularly good time for people to think seriously about how the AIS community and the AI ethics community could work together.
3. Despite differences, it would be surprising if there was rather little that the "two" communities could learn from each other.
4. I appreciate the links and examples. I'll probably go through them at some point soon and possibly DM you. I think a lot of people are interested in this topic, but few have the time/background to actually "do research" and "compile resources". It seems plausible to me that more "lists of resources/examples/case studies" could improve reasoning on this topic (even moreso than high-level argumentation, and I say that as someone who's often advocating for more high-level argumentation!)

Some thoughts I had while reading that you might disagree with (or at least I didn't see acknowledged much in the post):

1. The differences between the two groups are not trivial, and they'll often lead to different recommendations. For example, if you brought ARC Evals together with (hypothetical) AI Ethics Evals, I imagine they would both agree "evals are important" but they would have strong and serious disagreements about what kinds of evals should be implemented.
2. In general, when two groups with different worldviews/priorities join coalitions, a major risk is that one (or both) of the groups' goals get diluted. 
3. It's harder to maintain good epistemics and strong reasoning + reasoning transparency in large coalitions of groups who have different worldviews/goals. ("We shouldn't say X because our allies in AI ethics will think it's weird.") I don't think "X is bad for epistemics" means "we definitely shouldn't consider X", but I think it's a pretty high cost that often goes underappreciated/underacknowledged (Holden made a similar point recently). 
4. In general, I think the piece could have benefitted from expressing more uncertainty around certain claims, acknowledging counterarguments more, and trying to get an ITT of people who disagree with you. 

Exciting! Have you considered linkposting this to LessWrong? (Some technical AIS folks and AI governance folks check LW more than EAF)

Thanks! I found this context useful. 

Thanks for this context. Is it reasonable to infer that you think that OpenAI would've got a roughly-equally-desirable investment if OP had not invested? (Such that the OP investment had basically no effect on acceleration?)

Personally, I see this as a misunderstanding, i.e. that OP helped OpenAI to come into existence and it might not have happened otherwise.

I think some people have this misunderstanding, and I think it's useful to address it.

With that in mind, much of the time, I don't think people who are saying "do those benefits outweigh the potential harms" are assuming that the counterfactual was "no OpenAI." I think they're assuming the counterfactual is something like "OpenAI has less money, or has to take somewhat less favorable deals with investors, or has to do something that it thought would be less desirable than 'selling' a board seat to Open Phil."

(I don't consider myself to have strong takes on this debate, and I think there are lots of details I'm missing. I have spoken to some people who seem invested in this debate, though.)

My current ITT of a reasonable person who thinks the harms outweighed the benefits says something like this: "OP's investment seems likely to have accelerated OpenAI's progress and affected the overall rate of AI progress. If OP had not invested, OpenAI likely would have had to do something else that was worse for them (from a fundraising perspective) which could have slowed down OpenAI and thus slowed down overall AI progress."

Perhaps this view is mistaken (e.g., maybe OpenAI would have just fundraised sooner and started the for-profit entity sooner). But (at first glance), giving up a board seat seems pretty costly, which makes me wonder why OpenAI would choose to give up the board seat if they had some less costly alternatives.

(I also find it plausible that the benefits outweighed the costs, though my ITT of a reasonable person on the other side says something like "what were the benefits? Are there any clear wins that are sharable?")

Personally, I see this as a misunderstanding, i.e. that OP helped OpenAI to come into existence and it might not have happened otherwise.

I think some people have this misunderstanding, and I think it's useful to clarify this.

With that in mind, much of the time, I don't think people who are saying "do those benefits outweigh the potential harms" are assuming that the counterfactual was "no OpenAI." I think they're assuming something like "OpenAI has less money, or has to take somewhat less favorable deals with investors, or has to do something that it thought would be less desirable than 'selling' a board seat to Open Phil."

(I don't consider myself to have strong takes on this debate and I think there are lots of details I'm missing. I have spoken to some people who seem invested in this debate, though.)

My current ITT of a reasonable person who thinks the harms outweighed the benefits says something like this: "OP's investment seems likely to have accelerated OpenAI's progress and affected the overall rate of AI progress. If OP had not invested, OpenAI likely would have had to do something else that was worse for them (from a fundraising perspective) which could have slowed down OpenAI and thus slowed down overall AI progress."

Perhaps this view is mistaken (e.g., maybe OpenAI would have just fundraised sooner and started the for-profit entity sooner). But (at first glance), giving up a board seat seems pretty costly, which makes me wonder why OpenAI would choose to give up the board seat if they had some less costly alternatives.

(I also find it plausible that the benefits outweighed the costs, though my ITT of a reasonable person on the other side says something like "what were the benefits? Are there any clear wins that are sharable?")

Ah, thanks for the summary of the claim. Apologies for misunderstanding this in my initial skim. 

I agree with Rameon's comment.

I appreciate the red-team of a commonly-discussed AI governance proposal. I'm confused about the section on race dynamics:

However, if it is true that the primary effect of the Windfall Clause is to concentrate money/power in the hands of AGI startup CEOs, this argument will likely not work. Rather, from the point of view of the CEOs, the spoils available to winning are now even larger than before, because you will get a massive slush fund for political pet projects, and the losers will not even get the consolation prize of their 401(k) doing so well.

How are the spoils of winning larger under the Windfall clause? I suppose the spoils of being part of the Windfall clause are large (i.e., if the top 20 labs get to participate in the clause, then it's really important to be in the top 20). But it seems to be like the dangers of race dynamics are primarily concentrated in the desire to be in 1st place-- that is, the top 3-5 actors racing really hard against each other seem like the thing that actually causes people to develop/deploy systems prematurely.

The 401(k) point seems very confusing to me. I think you mention somewhere that "many people own some Microsoft and Google", but this seems pretty unpersuasive. The Rich and Powerful People who own way more Microsoft and Google than everyone else are the ones who will disproportionately benefit. If people who own 0.0000000000000000001% of Microsoft are going to be able to influence the post-TAI future, I don't think it's going to be because of their investment portfolio (but maybe it'll happen if the people who are in control of TAI do something like deliberative democracy).

In summary, I don't see how the Windfall Clause reduces race dynamics (particularly the racing between the top 1-5 actors, which is where I expect most of the danger comes from). When I imagine myself being one of the top 5 actors, I feel more comfortable "losing the race" if I've signed a Windfall Clause than if I'm relying on traditional corporate governance.

Can you say more about which longtermist efforts you're referring to?

I think a case can be made, but I don't think it's an easy (or clear) case.

My current impression is that Yudkowsky & Bostrom's writings about AGI inspired the creation of OpenAI/DeepMind. And I believe FTX invested a lot in Anthropic and OP invested a little bit (in relative terms) into OpenAI. Since then, there have been capabilities advances and safety advances made by EAs, and I don't think it's particularly clear which outweighs.

It seems unclear to me what the sign of these effects are. Like, maybe no one thinks about AGI for decades. Or maybe 3-5 years after Yudkowsky starts thinking about AGI, someone else much less safety-concerned starts thinking about AGI, and we get a world with AGI labs that are much less concerned about safety than status-quo.

I'm not advocating for this position, but I'm using it to illustrate how the case seems far-from-easy. 

A lot of longtermist effort is going into AI safety at the moment. I think it's hard to make the case that something in AI safety has legibly or concretely reduced AI risk, since (a) the field is still considered quite pre-paradigmatic, (b) the risk comes from systems that are more powerful than the ones we currently have, and (c) even in less speculative fields, research often takes several years before it is shown to legibly help anyone.

But with those caveats in mind, I think:

1. The community has made some progress in understanding possible risks and threats from advanced AI system. (See DeepMind's review of alignment threat models). 
2. Interpretability research seems relatively legible. The basic case "we're building powerful models and it would be valuable to understand how they work" makes intuitive sense. There are also several more nuanced ways interpretability research could be helpful (see Neel's longlist of theories for impact).
3. The fact that most of the major AGI labs have technical safety teams and governance teams seems quite concrete/legible. I'm not sure how much credit should go to the longtermist communities, but I think several of these teams have been inspired/influenced by ideas in the AI safety community. (To be fair, this might just be a case of "get lots of people to think seriously about reducing x-risk", but I think it's a bit more tangible/concrete.)
4. In AI governance, the structured access approach seems pretty common among major AGI labs (again, a bit unclear how much credit should go to longtermists but my guess is a non-negligble amount). 
5. In AI governance, some work on reducing misuse risks and recognizing the dual-use nature of AI technologies seems somewhat legible. A lot of people who did this research are now working at major AGI labs, and it seems plausible that they're implementing some of the best practices they suggested (which would be especially legible, though I'm not aware of any specific examples, though this might be because labs keep a lot of this stuff confidential). 

+1 on questioning/interrogating opinions, even opinions of people who are "influential leaders."

I claim people who are trying to use their careers in a valuable way should evaluate organizations/opportunities for themselves

My hope is that readers don't come away with "here is the set of opinions I am supposed to believe" but rather "ah here is a set of opinions that help me understand how some EAs are thinking about the world." Thank you for making this distinction explicit.

Disagree that these are mostly characterizing the Berkeley community (#1 and #2 seem the most Berkeley-specific, though I think they're shaping EA culture/funding/strategy enough to be considered background claims. I think the rest are not Berkeley-specific).

I'd be excited about posts that argued "I think EAs are overestimating AI x-risk, and here are some aspects of EA culture/decision-making that might be contributing to this."

I'm less excited about posts that say "X thing going on EA is bad", where X is a specific decision that EAs made [based on their estimate of AI x-risk]. (Unless the post is explicitly about AI x-risk estimates).

Related: Is that your true rejection?

Thanks for writing this, Eli. I haven't read WWOTF and was hoping someone would produce an analysis like this (especially comparing The Precipice to WWOTF).

I've seen a lot of people posting enthusiastically about WWOTF (often before reading it) and some of the press that it has been getting (e.g., cover of TIME). I've felt conflicted about this.

On one hand, it's great that EA ideas have the opportunity to reach more people.

On the other hand, I had a feeling (mostly based on quotes from newspaper articles summarizing the book) that WWOTF doesn't feature AI safety and doesn't have a sense of "hey, a lot of people think that humanity only has a few more decades [or less] to live." 

I hope that EAs concerned about AIS champion resources that accurately reflect their sense of concern, feature AI safety more prominently, and capture the emotion/tone felt by many in the AIS community. (List of Lethalities is a good example here, though it has its own flaws and certainly isn't optimizing for widespread appeal in the same way that WWOTF seems to be).

Not yet, and I'm to blame. I've been focusing on a different project recently, which has demanded my full attention. 

Will plan to announce the winners (and make winning entries public, unless authors indicated otherwise) at some point this month.

+1. The heuristic doesn’t always work.

(Though for an intro talk I would probably just modify the heuristic to “is the the kind of intro talk that would’ve actually excited a younger version of me.”)

Thanks for writing this, Emma! Upvoted :)

Here's one heuristic I heard at a retreat several months ago: "If you're ever running an event that you are not excited to be part of, something has gone wrong."

Obviously, it's just a heuristic, but I actually found it to be a pretty useful one. I think a lot of organizers spend time hosting events that feel more like "teaching" rather than "learning together or working on interesting unsolved problems together." 

And my impression is that the groups that have fostered more of a "let's learn together and do things together" mentality have tended to have the most success.

This seems like a good time to amplify Ashley's We need alternatives to intro EA Fellowships, Trevor's University groups should do more retreats, Lenny's We Ran an AI Timelines Retreat, and Kuhan's Lessons from Running Stanford EA and SERI.

Thank you for writing this, Ben. I think the examples are a helpful and I plan to read more about several of them. 

With that in mind, I'm confused about how to interpret your post and how much to update on Eliezer. Specifically, I find it pretty hard to assess how much I should update (if at all) given the "cherry-picking" methodology:

Here, I’ve collected a number of examples of Yudkowsky making (in my view) dramatic and overconfident predictions concerning risks from technology.

Note that this isn’t an attempt to provide a balanced overview of Yudkowsky’s technological predictions over the years. I’m specifically highlighting a number of predictions that I think are underappreciated and suggest a particular kind of bias.

If you were apply this to any EA thought leader (or non-EA thought leader, for that matter), I strongly suspect you'd find a lot clearcut and disputable examples of them being wrong on important things. 

As a toy analogy, imagine that Alice is widely-considered to be extremely moral. I hire an investigator to find as many examples of Alice doing Bad Things as possible. I then publish my list of Bad Things that Alice has done. And I tell people "look-- Alice has done some Bad Things. You all think of her as a really moral person, and you defer to her a lot, but actually, she has done Bad Things!"

And I guess I'm left with a feeling of... OK, but I didn't expect Alice to have never done Bad Things! In fact, maybe I expected Alice to do worse things than the things that were on this list, so I should actually update toward Alice being moral and defer to Alice more. 

To make an informed update, I'd want to understand your balanced take. Or I'd want to know some of the following:

• How much effort did the investigator spend looking for examples of Bad Things?
• Given my current impression of Alice, how many Bad Things (weighted by badness) would I have expected the investigator to find?
• How many Good Things did Alice do (weighted by goodness)? 

Final comment: I think this comment might come across as ungrateful-- just want to point out that I appreciate this post, find it useful, and will be more likely to challenge/question my deference as a result of it.

Hey, Jay! Judging is underway, and I'm planning to announce the winners within the next month. Thanks for your patience, and sorry for missing your message.

Thank you, Caleb! 

Miranda, your FB profile & EA profile are great examples of #3 :) 

Thank you, Chana!

Thanks, Evie!

Thanks for mentioning this, Rohin! I agree that longer write-ups and retrospectives can be valuable. And if someone determines that it's valuable for them to spend 40 hours on a write-up, I'd encourage them to do so.

For this contest, I don't want the "norm" or "expectation" to be a 20+ hour write-up. I'm expecting many submissions that take the form "here's an idea that I was already thinking about, and now this contest nudged me to sit down and write it up" or "I sat down and spent a few hours reflecting on X, and here's what I learned."

This is partially motivated by finm's comment here:

I'm a bit worried that contests with large prizes can have distortionary effects. That is, they might pull EAs towards using their time in ways which are not altruistically/impartially best. This would happen when an EA switches her marginal time to some contest with a big prize, where she otherwise would have been doing something expected to be more impactful (e.g. because she's a better fit for it), but which doesn't stand to win her as much money or acclaim.

Most importantly, I think people entering this contest should ask themselves if spending marginal hours on their entries would be a good use of their time (relative to their counterfactual). My guess is that most entrants would benefit from reflecting for 1-10 hours, and a smaller subset would benefit from reflecting for 10-100 hours.

I'd prefer a Google Doc. (Judges might want to add in-text comments). But if this is a barrier for some reason, a forum post would suffice.

As of now, per the EAIF grant, this only applies to new posts (posted between March 11 and April 30). 

But I think one could make a case that people who posted before the contest could be compensated/recognized for their work. And I like that the Creative Writing Contest, for instance, allowed retroactive submissions.

If you have a retroactive submission, feel free to email me at akashwasil133@gmail.com. I can't promise anything right now, but if I get several of these, I could check in with EAIF about awards for previously-posted submissions.

Thanks for raising this, Khorton!

Yup! It's fine if you post the entry to a blog, as long it could also be posted or cross-posted on the forum! 

Update: The deadline has been changed to April 30. Several people pointed out that the deadline felt tight & it would limit their ability to participate.

To encourage early submissions, we are offering three "early-bird prizes" ($1000 each) to the three best submissions we receive by March 31. 

Special thanks to Vaidehi, Kaleem, and those of you who emailed me with feedback about the deadline.

UPDATE: Deadline has been changed to April 30. Added a few "early-bird prizes" for submissions received before March 31. Hoping that this will encourage early submissions while allowing people to participate later. Thank you for the feedback!

Original comment below:

Thanks for pointing this out, Vaidehi and Kaleem! Setting a deadline is tricky, and this is helpful feedback. Here are a few considerations that went into the March 31 deadline:

• Spring break is in mid-March for many universities, and I wanted the deadline to be shortly after spring break.
• I think there are benefits to having people reflect after conferences, and I also think there are benefits to having people reflect before conferences. EAGxOxford is March 26-27 and EAGxBoston is April 1-3. So I figured this deadline would be a useful "midpoint."
• I think it would be great to start seeing some of these posts published before EAGxOxford and EAGx Boston.
• My impression is that three weeks is enough time, though I might be poorly calibrated on this. I expect most submissions will take <10 hours to write over the course of 1-3 days.
• I spoke with a few community builders about the deadline and they thought it seemed reasonable & they wouldn't feel rushed.

For now, I think it makes sense to keep the deadline, but I could see a world in which I extend it by a bit (especially if community builders reach out saying they need more time). If you're reading this and the deadline would prevent you from submitting, feel free to email me at akashwasil133@gmail.com.

Thank you for this write-up, Claire! I will put this in my "posts in which the author does a great job explaining their reasoning" folder.

I noticed that you focused on mistakes. I appreciate this, and I'm also curious about the opposite:

• What are some of the things that went especially well over the last few years? What decisions, accomplishments, or projects are you most proud of?
• If you look back in a year, and you feel really excited/proud of the work that your team has done, what are some things that come to mind? What would a 95th+ percentile outcome look like? (Maybe the answer is just "we did everything in the Looking Forward" section, but I'm curious if some other things come to mind).

1. How does your theory of change or understanding of the alignment problem differ from that of other orgs? (e.g., ARC, Redwood, MIRI, Anthropic). Note that I see you answered a similar question here, though I think this question is a bit different.
2. How would you explain what value extrapolation is & why it's important to a college freshman?
3. What kinds of skills/backgrounds/aptitudes are you looking for in new employees? What kinds of people would you be really excited to see join the team?
4. Are there any skills/aptitudes that would be a uniquely good fit for value extrapolation research? (As in, skills that would make someone an especially good fit for working on this problem as opposed to other problems in AI alignment research)

(Feel free to skip any of these that don't seem like a good use of time!)

Thank you for this post, Neel! I think it's really useful to amplify stories about how people developed (and are continuing to develop) inside views. 

I've recently been thinking about an analogy between "developing inside views" and "thinking deliberately about one's career." I think there's a parallel with 80,000 Hours' general advice. It's like, "We have 80,000 Hours in our career; career planning is really hard, but wouldn't it make sense to spend at least a few dozen or a few hundred hours planning before making big choices? Isn't is super weird that we live in a society in which the "default" for career planning is usually so unhelpful? Even if we won't land on the perfect career, it seems worth it to spend some time actually trying to figure out what we should do, and we might be surprised with how far that takes us."

The same frame might be useful for developing inside views in AI safety. When I think about someone with 80,000 Hours in their career, I'm like "wow, wouldn't it be nice to spend at least a few dozen hours upfront to try to see if they can make progress developing an inside view?. It's quite plausible they just spend a few dozen hours being super confused, and then they can go on and do what they were going to do anyways. But the potential upside is huge!"

This has motivated me to ask myself these questions, and I might experiment with asking others: How much have you really tried to form an inside view? If you could free up a few hours, what would you do to make progress?

I've found these questions (and similar questions) to be pretty helpful. They allow me to keep my eye on the objective (developing an inside view on a highly complicated topic in which many experts disagree) without losing sight of the trail (there are things I can literally do, right now, to continue taking small steps in the right direction).

This is great! Here are a few more (though some of these overlap a lot with the ones you've listed):

Explore-- Do something that allows you to get exposed to different kinds of tasks, skills, and people. (Seems especially useful early on when thinking about fit. Also lets people find things that they might not have been able to brainstorm or might have prematurely ruled out). Exploring in sprints may be better than exploring 3-5 things at once (consistent with "optimize one thing at a time").

Leaveability-- Do something that allows you to leave if you find something better. 

Anticipate the bottlenecks of the future-- Think about which skills will be the bottleneck in 3-5 years. Learn those. (This is a theme explored in High Output Management).

The average of five-- Consider the heuristic "you are the average of the five people you spend the most time with." Who are the people you would be spending the most time with, and how would you feel about becoming more like them? (Shoutout to Jake McKinnon for discussing this with me recently).

Location-- Do something that allows you to work in a location that satisfies you professionally and emotionally. I think it's easy to underestimate how much location can affect people (especially when location is tied so strongly to community/mentorship-- e.g., EA hubs).

Updated-- thanks for pointing this out, yiyang!

"Service/support mindset" reminds me of healers in role-playing games. You don't show up on the damage charts, but you kept everyone alive (and allowed them to optimize their builds for damage)!

Thank you, Joy! "How do I know if I'm a good fit for X?" is a question that I commonly hear. I'll be sending this post along to people who are asking about ops!

Two specific things I liked:

1. You mentioned several specific examples of things you've done, and I felt like I could visualize some of the hours of your day.
2. The questions at the end (the italicized sections) seem quite useful and balanced. I sometimes see lists of questions that feel pretty unhelpful (e.g., "are you ambitious? Are you a team player?"). I appreciate that your list includes several questions that many EAs could actually say "no" to, which is super useful information (and also makes "yes" answers a stronger and more trustworthy signal).

Thank you, Vaidehi! I think your post is great; I have been encouraging myself & others to lower our bars for applying to things, and I love your points about testing personal fit & gaining information. I've linked your post at the end of mine.

Also, it seems that we posted within the same hour... neat that we were reflecting on similar topics at a similar time! 

Excellent— thank you for clarifying!

I'm curious about why the deadline was extended. I can think of two possible reasons:

1. Not enough applicants
2. Applicants rarely had the required skills/interests/qualifications

It seems like either would be great information value. (#1 might imply that interest in this kind of program is lower than expected for some reason. #2 helps people improve the quality of their referrals).

Also, thank you for doing this! Even if #1 is true, this seems like a valuable experiment, and I'm grateful to you for running it.

Penn EA organizer here! Thanks for raising this discussion, Aaron. Penn EA paid intro fellows $500 this semester, and we plan to write-up a reflection soon. For now, though, I have a few quick thoughts on the points you raised:

1.1-- This seems quite reasonable. Does this require paying everyone, or just having an optional financial aid policy? I suppose people might feel bad applying for financial aid, especially from a group focused on altruism. 

1.2-- I actually see this as an argument against paying fellows. I think one common failure mode of fellowships is that you end up with groups where like 1 person is engaged and 3-5 people are clearly not engaged. After week 2 or 3, I would love it if we just said "hey, if you're not super interested in these ideas, no worries, you can stop coming." Paying people (for completion) increases the likelihood that people will stay out of a sense of commitment/obligation, even if they have no plans to take EA ideas seriously. I think this is quite harmful. It makes fellowship discussions less interesting/engaging. It makes going to the fellowship meetings feel more like a chore than something that feels really special/exciting. And I think it makes the experience worse for the fellows who are really committed.

1.3-- I think this is mostly a naming thing. It's completely normal for "weekly reading groups" to be unpaid. I think the term "fellowship" usually implies a much more involved experience (e.g., a 40hr/week 8-week research fellowship), which is why we associate the term "fellowship" with "paid."

1.4-- Agreed. Again, though, I worry about the concern I expressed in 1.2. I want people to spend time on these ideas because they're important, serious, inspiring, etc.

1.5-- Same thoughts as 1.2 and 1.4.

2.1-- Agreed. I think there's probably two things going on here: a) people who come for the money and b) people who stay out of a feeling of commitment/obligation that is reinforced by the money. I'm more worried about B than A. If we were offering stipends of, say, $5000, then I would become more worried about A.

2.2-2.5-- Pretty much agree with these risks and your ideas for managing them. The reputational risks in 2.3 seem especially important. The tail risk here seems pretty bad (e.g., media coverage that frames paying fellows in an uncharitable light). I believe CEA has talked about how tech companies and management consulting companies spend a lot of money on recruiting/outreach at universities. And there's no equivalent for high-impact work, which is a shame. So perhaps this can be framed as "we need to be competitive with these other organizations that are trying to attract top talent." I'm not sure how well this would work, though, since tech companies and management consultancies generally don't have super altruistic/feel-good/admirable reputations.

Final thought-- you claim:

I believe that the arguments for [paying fellows] are significantly more compelling than
the arguments against.

I think this is a pretty strong statement, and I currently don't agree with it. Even if the benefits outweighed the costs on average, I think there would be substantial variability across groups. At some universities, this may cause more reputational risk; at others, the added value from prestige might be especially high. 

Thank you for raising this discussion! Excited to say more once the Penn EA reflection is out.

Great points, Michael! Some of these questions (e.g., why does it make sense to pay people for learning about EA?) came up during Penn EA's fellowships this year.

In my experience as a facilitator, these conversations were almost always productive. I would generally use these moments as opportunities to explain why careers in high-impact cause areas are so valuable. I'd say something like "EA is willing to invest resources into student groups because they've seen that student groups (and intro fellowships) can help people apply EA principles in their careers, which is ultimately worth the financial investment. People who resonate with these ideas & take them seriously often go on to have really important careers in AI alignment, biosecurity, animal welfare, and other really important areas."

This explanation generally seemed persuasive (or at least helped people understand this perspective). 

*Also quick note of clarification: Penn EA experimented with $500 stipends this semester. Just to be clear, I don't believe this is something that Penn EA has ever done in the past, and it might not be something we do in the future.

Some of the comments point out ways that career conversations can go wrong (e.g., people see this as manipulative, people get turned off to EA if someone is telling them they need to change their career). Some comments also point out alternative strategies that would be helpful (e.g., asking people open-ended questions about their career, talking about personal fit/happiness in addition to impact).

With that in mind, I just want to express support for the original strategies endorsed by Nikola and Daniel.

• having facilitators in intro fellowships share their own motivated reasoning upon discovering EA,
• adding readings about changes in career plan (for instance, the part in Strangers Drowning that talks about Julia Wise or the piece On Saving the World, but the latter doesn't seem newcomer-friendly).

Both of these strategies seem clever, useful, practical, and unlikely to backfire if implemented well. And both of these seem quite compatible with other techniques (e.g., asking questions, discussing personal fit).

Quick Example of a Strawman and a Steelman of These Strategies

If someone says "well, you just want to be a doctor because of motivated reasoning! I had motivated reasoning too. And you should read this post about other people who had motivated reasoning until they found EA, and then they decided to give up their childhood dreams to do something that was actually impactful." --> This is probably bad

If someone says, "I find thinking about careers really difficult and messy sometimes. I think for a long time, I was set on a particular path, and it took a lot for me to let go of that path. There are also several examples of this happening in the EA community, and I'd be happy to share some of those narratives if you'd find it interesting." --> This is probably good

I think the new title is better. To nit-pick, "more objective" might imply a sense of "we're right and they're wrong". 

Maybe an alternative could be something like "Helping newcomers feel more comfortable considering new career paths." 

(I don't think you actually need to change the title again-- just throwing this out there because I see how discussions around these kinds of strategies can be perceived as manipulative, and I think the wording/framing we use in group discussions can matter).

I don't have anything useful to say, except that I've appreciated your recent posts & I hope to keep seeing more from you. Thank you for sharing, Nikola!

Thanks for sharing this, Aaron! Really interesting pilot work.

One quick thought-- I wouldn't rely too heavily on statistical significance tests, particularly with small sample sizes. P-values are largely a function of sample size, and it's nearly impossible to get statistical significance with 44 participants (unless your effect size is huge!). 

Speaking of effect sizes, it seems like you powered to detect an effect of d=0.7. For a messaging study with rather subtle manipulations, an effect of d=0.7 seems huge! I would be pretty impressed if giving people CE info resulted in an effect size of d=0.2 or d=0.3, for instance. I'm guessing you were constrained by the # of participants you could recruit (which is quite reasonable-- lots of pilot studies are underpowered). But given the low power, I'd be reluctant to draw strong conclusions.

I also appreciate that you reported the mean scores in the results section of your paper, which allowed me to skim to see if there's anything interesting. I think there might be!

There was no significant difference in Effective Donation between the Info (M = 80.21, SD = 18.79) and No Info (M = 71.79, SD = 17.05) conditions, F(1, 34) = 1.85, p = .183, η_p² = .052. 

If this effect is real, I think this is pretty impressive/interesting. On average, the Effective Donation scores are about 10% higher for the Info Group participants than the No Info group participants (and I didn't do a formal calculation for Cohen's d but it looks like it'd be about d=0.5). 

Of course, given the small sample size, it's hard to draw any definitive conclusions. But it seems quite plausible to me that the Info condition worked-- and at the very least, I don't think these findings provide evidence against  the idea that the info condition worked.

Would be curious to see if you have any thoughts on this. If you end up having an opportunity to test this with a larger sample size, that would be super interesting. Great work & excited to see what you do next!

As an example, I currently think All Possible Views About Humanity's Future are Wild would be on my list because it is a) thought-provoking and b) short; only takes 10-15 mins to read.

In contrast, The Case for Strong Longtermism is thought-provoking but would likely take >1hr to read, so I probably wouldn't include it on my list. (Though I would certainly recommend it to people who want to learn more.)

+1! For debate, I believe the EA Debate Championship and Lecture Series would offer some useful lessons. It was for college debate, and I haven't heard of anything similar being done for HS debate (but it's quite possible that I've missed something). 

Many top universities in the US host HS debate tournaments (often as a money-making activity for the club). It seems plausible to me that at least one of the major uni debate societies would be open to sponsoring an EA-related tournament (or perhaps have one or two rounds that are focused on EA topics). Has anyone tried this?

If someone wants to try this: I did HS debate semi-competitively and college debate (American Parliamentary Debate) during my first two years of college [at Harvard]. If anyone is interested in learning more about the debate circuit, feel free to reach out!  (I'm sure there are other EAs who were more involved in the HS/college debate scene, so if one of them comments, reach out to them instead!). 

Thanks,  Nikola! +1 on so much of this. A few specific thoughts:

I often hear people say things like “they’re so cool” or “they're superhuman” about individuals who are highly engaged with EA. Saying this makes newcomers want to emulate these individuals more and become more engaged to get some of that sweet, sweet status.

I agree that having high-status role models can be really inspiring/motivating. At the same time, I think some of these comments can have unintended harmful effects. I've seen a few ways this can go wrong:

1. "I'm not superhuman, so this isn't for me"-- The "some people are super cool/superhuman" mentality can reinforce the idea that there are "Super Cool/Smart/Amazing people" and then "All of the Regular People." I worry that people-- especially newcomers-- might assume they're in the latter category and then get turned off. 
2. "This feels weird-- why are they worshipping these people?" I think a lot of people are generally put off by hero-worship/cults of personality. This is especially true of people with high epistemic standards. I worry that these people will here "X is superhuman", see a bunch of people nodding along, and then think, "wow, this community is weird and puts people on pedestals. Not for me!"

An idea for how to solve this (epistemic status: uncertain and hastily written)-- emphasize the actions that the high-status people are taking. My guess is that it's generally better for people to tie status to actions rather than people. But talking about people can be a good way to help people see which actions are considered "high-status."

Ex: Instead of "X is superhuman", I currently prefer the framing "I really admire X. X is super agentic and always looking for ways to have a higher impact. As soon as X changed their mind about their cause prioritization, they started thinking seriously about changing their career plan." I think the latter frame places the emphasis on specific concrete things the person is doing-- and ideally these are things that other members could do (i.e., admiring someone for being agentic signals "you should try to be more agentic", whereas admiring someone for being naturally smart/talented/gifted might be less likely to foster motivation to grow/improve.)

Would love to know what you think about this! (And to be clear, I don't think your initial statement was incompatible with any of this. It's quite possible you agree with everything & didn't want to use space to go into all of this).

The second point comes mostly from intuition: If you imagine a highly engaged person surrounded by non-engaged people and a non-engaged person surrounded by highly engaged people, the non-engaged person’s engagement will probably increase faster than the engaged person’s engagement will decrease. 

Agreed. I also think this is one of the biggest flaws of the current fellowship model. At Penn, my estimate is that roughly 20% of fellows were moderately-to-highly engaged (i.e, taking the ideas seriously & considering changes to their studies/careers). 

In practice, this meant that a given fellowship group often had one promising person, a facilitator, and 3-4 people who weren't as engaged. I think this had the effect of making the promising person less excited (to borrow your analogy, "few people want to be more Catholic than the other bishops.")

A few possible ways to solve this: Filtering more at the beginning (e.g., having 1-on-1s with fellowship applicants and coming up with a good evaluation system) placing people into fellowship groups based on how likely they are to be engaged (I think some other unis do this), or experimenting with alternatives to the intro fellowship. I'd be curious if you have any thoughts on this challenge & these possible solutions.

A similar thing seems to have happened at UPenn recently when Sydney and Thomas from Stanford EA helped University of Pennsylvania EA get off the ground

Strong +1 (+2?) on residencies. I think being around Sydney/Thomas was quite influential for me/other Penn organizers. My guess is that aggressively tabling (not just at club fairs, but pretty much every day for the first few weeks of the semester) and 1-on-1s (with potential organizers and other people who seemed engaged) were the two most important components of the residency. Would be curious to hear what you think the most important parts of the Cambridge residencies.

Thank you for this excellent post! I'm a student group organizer & I'll be recommending this to other members/organizers :) 

A few specific thoughts that came up as I read:

Rather than trying to convince the employer that you have to be chosen, you can flip it and concentrate on helping their employer to find the right fit for the position

I love this advice, and I think it can have a special implication for EAs. The question flips from "how can I get this job" to "how can I help this employer make an informed assessment of how impactful I will be at this job".

A traditional approach to job applications might be something like "I want to impress the interviewer, so I maximize my chance of getting a job offer. Then, I will select the offer that is best for me." 

An EA alternative to this might be something like "I want to be highly transparent with the interviewer, so I maximize the amount of information they have when determining who would be most impactful for this role. Then, I will compare my options and consider which one generates the most counterfactual impact [alongside other considerations like personal fit]."

Try and think of the main concerns of the potential employer about your candidacy and use your application to reassure them that the risk of these concerns is low. 

With this "transparency frame" in mind, I'm not sure how I feel about this point. I think this line suggests that people should optimize for "reassuring employers that you are a good hire." I would rather have people optimize for "being transparent and providing information that helps employers assess whether or not you are a good hire." 

As a result, I would be inclined to encourage people to explicitly state their key uncertainties about taking the role & be upfront about potential weaknesses/doubts. (Of course, this is assuming someone is reasonably well-calibrated about their skills/aptitudes. For people who underestimate themselves, this advice would backfire). 

I also think this is easier said than done-- optimizing for transparency inherently means that you might reduce your odds of getting the job. I'd guess that the tradeoff generally isn't super high, though, and it's also quite plausible to me that EA employers would be enthusiastic about people who are upfront with their weaknesses/doubts. It could also help them recommend the applicant for a role that's more suited to their particular aptitudes.

What do you think? (I could be very wrong about this, and I've never been in a hiring role!)

Whoops-- definitely meant my comment as a response to "what content can be cut?" And the section about activities was meant to show how some of the activities in the current fellowship are insufficient (in my view) & offer some suggestions for other kinds of activities.

Regardless of whether we shift to a radically new model, or we try to revamp the existing structure, I think it'll be useful to dissect the current fellowship to see what content we most want to keep/remove.

Will try to respond to the rest at some point soon, but just wanted to clarify!