AI Governance Course - Curriculum and Application 2021-11-29T13:29:30.156Z
Takeaways on US Policy Careers (Part 2): Career Advice 2021-11-08T08:42:57.801Z
Many Undergrads Should Take Light Courseloads 2021-10-25T00:12:25.653Z
Consider "Semester in DC" Programs, if You're a US Student Interested in Policy Work 2021-08-31T05:32:56.482Z
Takeaways on US Policy Careers (Part 1): Paths to Impact and Personal Fit 2021-08-26T02:45:29.209Z
[Past Event] US Policy Careers Speaker Series - Summer 2021 2021-06-18T20:01:21.338Z
[Short Version] What Helped the Voiceless? Historical Case Studies 2020-12-15T03:40:36.628Z
What Helped the Voiceless? Historical Case Studies 2020-10-11T03:38:57.632Z


Comment by Mauricio on [deleted post] 2022-01-07T20:55:08.237Z

How about "College/undergraduate advice"? It's clunky, but I'd guess that's outweighed by the significant boost to the chances that someone looking for this finds it. (E.g. if I didn't already know about this tag and was looking for it, I'd have searched for "undergraduate" in the search bar, but I don't think I'd have looked for "college." Maybe that's a rare case?)

Alternatively, maybe just putting "undergraduate" into the description is good enough for making it easier to find? If so, seems good to also throw in other related terms ("undergrad", "university", "student") into the description.

Comment by Mauricio on Think about EA alignment like skill mastery, not cult indoctrination · 2022-01-06T23:53:38.511Z · EA · GW

I think both (1) and (2) are sufficiently mild/non-nefarious versions of "repeating talking points" that they're very different from what people might imagine when they hear "techniques associated with cult indoctrination"--different enough that the latter phrase seems misleading.

(E.g., at least to my ears, the original phrase suggests that the communication techniques you've seen involve intentional manipulation and are rare; in contrast, (1) and (2) sound to me like very commonplace forms of ineffective (rather than intentionally manipulative) communication.)

(As I mentioned, I'm sympathetic to the broader purpose of the post, and my comment is just picking on that one phrase; I agree with and appreciate your points that communication along the lines of (1) and (2) happen, that they can be examples of poor communication / of not building from where others are coming from, and that the "skill mastery" perspective could help with this.)

Comment by Mauricio on Think about EA alignment like skill mastery, not cult indoctrination · 2022-01-06T17:38:05.344Z · EA · GW

Thanks! Seems like a useful perspective. I'll pick on the one bit I found unintuitive:

Summary: People who try to get more people to be EA-aligned often use techniques associated with cult indoctrination, such as repeating talking points and creating closed social circles.

In the spirit of not repeating talking points, could you back up this claim, if you meant it literally? This would be big if true, so I want to flag that:

  • You state this in the summary, but as far as I can see you don't state/defend it anywhere else in the post. So people just reading the summary might overestimate the extent to which the post argues for this claim.
  • I've seen lots of relevant community building, and I more often see the opposite: people being such nerds that they can't help themselves from descending into friendly debate, people being sufficiently self-aware that they know their unintuitive/unconventional views won't convince people if they're not argued for, and people pouring many hours into running programs and events (e.g. dinners, intro fellowships, and intro-level social events) aimed at creating an open social environment.

(As an aside, people might find it interesting to briefly check out YouTube videos of actual modern cult tactics for comparison.)

Comment by Mauricio on AI Governance Course - Curriculum and Application · 2022-01-03T08:18:34.054Z · EA · GW

Thanks! Good to know that wasn't easy enough to find--I've now added links in several additional spots in the post, including near the top. (The link you had was also right.)

Comment by Mauricio on Democratising Risk - or how EA deals with critics · 2021-12-30T20:05:12.403Z · EA · GW

Other thoughts:

  • Some other comment hinted at this: another frame that I'm not sure this paper considers is that non-strong-longtermist views are in one sense very undemocratic--they drastically prioritize the interests of very privileged current generations while leaving future generations disenfranchised, or at least greatly under-represented (if we assume there'll be many future people). So characterizing a field as being undemocratic due to having longtermism over-represented sounds a little like calling the military reconstruction that followed the US civil war (when the Union installed military governments in defeated Southern states to protect the rights of African Americans) undemocratic--yes, it's undemocratic in a sense, but there's also an important sense in which the alternative is painfully undemocratic.
    • How much we buy my argument here seems fairly dependent on how much we buy (strong) longtermism. It's intuitive to me that (here and elsewhere) we won't be able to fully answer "to what extent should certain views be represented in this field?" without dealing with the object-level question "to what extent are these views right"? The paper seems to try to side-step this, which seems reasonably pragmatic but also limited in some ways.
    • I think there's a similarly plausible case for non-total-utilitarian views being in a sense undemocratic; they tend to not give everyone equal decision-making weight. So there's also a sense in which seemingly fair representation of these other views is non-democratic.
      • As a tangent, this seems closely related to how a classic criticism of utilitarianism--that it might trample on the few for the well-being of a majority--is also an old criticism of democracy (which is a little funny, since the paper both raises these worries with utilitarianism and gladly takes democracy on board, although that might be defensible.)
  • One thing I appreciate about the paper is how it points out that the ethically loaded definitions of "existential risk" make the scope of the field dependent on ethical assumptions--that helped clarify my thinking on this.
Comment by Mauricio on Is EA compatible with technopessimism? · 2021-12-30T19:31:17.852Z · EA · GW

I think you're using a stronger assumption in your ethical theories that situations are even comparable, if you ignore when they occur

Hm I wouldn't endorse that assumption. I avoided specifying "when"s to communicate more quickly, but I had them in mind something like your examples--agree the times matter.

trying to get the first thing to happen (evolve to stable society) instead of second or third is worth doing if it were the only thing we could do in 2021

Agreed but only if we add another condition/caveat: that trying to get the first thing to happen also didn't trade off against the probability of very good scenarios not covered in these three scenarios (which it would mathematically have to do, under some assumptions). As an oversimplistic example with made-up numbers, suppose we were facing these probabilities of possible futures:

  • 20% -- your first scenario (tech stagnation) (10 goodness points)
  • 5% -- your second scenario (mass suffering) (-1,000,000 goodness points)
  • 20%--your third scenario (extinction) (-10 goodness points)
  • 55%--Status quo in 2021 evolving to technologically sophisticated utopia by 2100 (1,000,000 goodness points)

And suppose the only action we could take in 2021 would change the above probabilities to the following:

  • 100% -- your first scenario (tech stagnation) (10 goodness points)
  • 0% -- your second scenario (mass suffering) (-1,000,000 goodness points)
  • 0%--your third scenario (extinction) (-10 goodness points)
  • 0%--Status quo in 2021 evolving to technologically sophisticated utopia by 2100 (1,000,000 goodness points)

Then the expected value of not taking the action is 500,000 goodness points, while the expected value of taking the action is 10 goodness points, so taking the action would be very bad / not worthwhile (despite how technically the action falls under your description of "trying to get the first thing to happen (evolve to stable society) instead of second or third [...] if it were the only thing we could do in 2021").

Comment by Mauricio on Democratising Risk - or how EA deals with critics · 2021-12-29T12:27:02.988Z · EA · GW

Thanks for sharing this! Responding to just some parts of the object-level issues raised by the paper (I only read parts closely, so I might not have the full picture)--I find several parts of this pretty confusing or unintuitive:

  • Your first recommendation in your concluding paragraph is: "EA needs to diversify funding sources by breaking up big funding bodies." But of course "EA" per se can't do this; the only actors with the legal authority to break up these bodies (other than governments, which I'd guess would be uninterested) are these funding bodies themselves, i.e. mainly OpenPhil. Given the emphasis on democratization and moral uncertainty, it sounds like your first recommendation is a firm assertion that two people with lots of money should give away most of their money to other philanthropists who don't share their values, i.e. it's a recommendation that obviously won't be implemented (after all, who'd want to give influence to others who want to use it for different ends?). So unless I've misunderstood, this looks like there might be more interest in emphasizing bold recommendations than in emphasizing recommendations that stand a chance of getting implemented. And that seems at odds with your earlier recognition, which I really appreciate--that this is not a game. Have I missed something?
  • Much of the paper seems to assume that, for moral uncertainty reasons, it's bad for the existential risk research community to be unrepresentative of the wider world, especially in its ethical views. I'm not sure this is a great response to moral uncertainty. My intuition would be that, under moral uncertainty, each worldview will do best (by its own lights) if it can disproportionately guide the aspects of the world it considers most important. This suggests that all worldviews will do best (by their own lights) if [total utiliarianism + strong longtermism + transhumanism]* retains over-representation in existential risk research (since this view cares about this niche field to an extremely unusual extent), while other ethical views retain their over-representation in the many, many other areas of the world that entirely lack these longtermists. These disproportionate influences just seem like different ethical communities specializing differently, to mutual benefit. (There's room to debate just how much these ethical views should concentrate their investments, but if the answer is not zero, then it's not the case that e.g. the field having "non-representative moral visions of the future" is a "daunting problem" for anyone.)

*I don't use your term "techno-utopian approach" because "utopian" has derogotary connotations, not to mention misleading/inaccurate connotations re: these researchers' typical levels of optimism regarding technology and the future.

Comment by Mauricio on Is EA compatible with technopessimism? · 2021-12-29T10:37:44.440Z · EA · GW

I'm not sure I follow. [...] I assume all ethical views prefer status quo to extinction or totalitarianism

I wonder if we might be using "net negative" differently? By "net negative" I mean "worse than non-existence," not "worse than status quo." So even though we may prefer a stable status quo to imminent extinction, we might still think the latter leaves us at roughly net zero (i.e. not net negative, or at least not significantly net negative).

I also suspect that, under many ethical views, some forms of totalitarianism would be better than non-existence (i.e. not net-negative). For example, a totalitarian world in which freedoms/individuality are extremely limited--but most people are mostly happy, and extreme suffering is very rare--seems at least a little better than non-existence, by the lights of many views about value.

(A lot of what I'm saying here is based on the assumption that, according to very scope-sensitive views of value: on a scale where -100 is "worst possible future" and 0 is "non-existence" and 100 is "best possible future," a technologically unsophisticated future would be approximately 0, because humanity would miss out on the vast majority of time and space in which we could create (dis)value. Which is why, for a technologically unsophisticated future to be better than the average technologically sophisticated future, the latter has to be net negative.)

Oh I agree, I feel like superintelligence cannot be trusted, at least the kind that's capable of global power-grabs. [...] I think it's largely because humans don't want consistent things, and cannot possibly want consistent things, short of neurosurgery.

I'm not sure if you mean that humans' preferences (a) are consistent at any given time but inconsistent over time, or (b) are inconsistent even if we hold time constant. I'd have different responses to the two.

Re: (a), I think this would require some way to aggregate different individuals' preferences (applied to the same individual at different times)--admittedly seems tricky but not hopeless?

Re: (b), I agree that alignment to inconsistent preferences is impossible. (I also doubt humans can be aligned to other humans' inconsistent preferences--if someone prefers apples to pears and they also prefer pears to apples (as an example of an inconsistent preference), I can't try to do what they want me to do when they ask for a fruit, since there isn't a consistent thing that they want me to do.) Still--I don't know, I don't feel that my preferences are that incoherent, and I think I'd be pretty happy with an AI that just tries to do what I want it to do to whatever extent I have consistent wants.

Comment by Mauricio on My Overview of the AI Alignment Landscape: A Bird’s Eye View · 2021-12-26T18:20:33.284Z · EA · GW

You're right, this seems like mostly semantics. I'd guess it's most clear/useful to use "alignment" a little more narrowly--reserving it for concepts that actually involve aligning things (i.e. roughly consistently with non-AI-specific uses of the word "alignment"). But the Critch(/Dafoe?) take you bring up seems like a good argument for why AI-influenced coordination failures fall under that.

Comment by Mauricio on Is EA compatible with technopessimism? · 2021-12-26T18:06:28.509Z · EA · GW

This shouldn't be too hard if the default case from tech progress is extinction / totalitarianism.

Maybe, although I suspect this assumption makes it significantly harder to argue that a technologically sophisticated future is net negative in expectation (since, at least by ethical views that seem especially common in this community, extinction leads to approximately net zero (not net negative) futures, and it seems plausible to me that a totalitarian future--with all the terrible loss of potential that would involve--would still be better than non-existence, i.e. not net negative).

I don't know what a proof of "solving alignment" being impossible looks like

Just to clarify, I wouldn't demand that--I'd be looking for at least an intuitive argument that solving alignment is intractable. I agree that's still hard.

I still haven't understood what it's like inside the mind of someone who believes alignment is possible

As a tangent (since I want to focus on tractability rather than possibility, although impossibility would be more than enough to show intractability): the main reason I think that alignment (using roughly this definition of alignment) is possible is: humans can be aligned to other humans; sometimes we act in good faith to try to satisfy another's preferences. So at least some general intelligences can be aligned. And I don't see what could be so special about humans that would make this property unique to us.

Returning from the tangent, I'm also optimistic about tractability because:

  • People haven't been trying for that long, and the field is still very small
  • At least some prominent, relatively new research directions (e.g. [1], [2], [3]) seem promising

Some intuition: [...]

Yup this seems plausible, you get the bonus points :)

Comment by Mauricio on Is EA compatible with technopessimism? · 2021-12-26T17:42:15.010Z · EA · GW

I can understand why the bar is very different for organisations to react to information - some need to react very fast (startups), some need to be very cautious (judiciaries). But does that also apply to individual's personal opinions?

I was thinking that individuals have different background beliefs (and different attentional/incentive environments), and these things create differences in what it will take to change people's minds. E.g. some people in the community are much closer to agreement with techno-pessimists than others--they'll likely have a lower burden of proof.

note most systemic change is not constrained by capital alone

Agreed, I had a broad definition of "resources" in mind (including e.g. labor/careers, connections), but I could have been more clear.

Comment by Mauricio on Is EA compatible with technopessimism? · 2021-12-25T19:28:00.512Z · EA · GW

Other thoughts:

  • The bar also seems very different depending on precisely what organizations/individuals within the community you're trying to convince, and what you're trying to convince them of (e.g. the global health side of the community seems to be unresponsive to careful, intuitive arguments that are unaccompanied by RCTs, while the AI people are very interested in such arguments).
  • I think most of the difficulty comes from the generically very high burden of proof for arguing that any given cause area / intervention is the most effective use of resources, not from techno-pessimism-specific disagreements (although those don't help).
  • Re: "If you feel this is a bad framework, please let me know."--yup, it seems like this framework overlooks/obscures many other potential reactions to (1).
Comment by Mauricio on Is EA compatible with technopessimism? · 2021-12-25T19:27:34.457Z · EA · GW

This isn't directly answering your question, but one thing I can easily imagine happening is: some techno-pessimists spend lots of effort pulling together evidence and arguments for one premise of techno-pessimism, rush (or leave unmade) arguments for other crucial premises, and then find it frustrating that this doesn't change many people's minds. My guess is there's, say, 5+ substantive disagreements that are all upstream of typical disagreement on techno-pessimism, so I expect the bigger challenge for techno-pessimists in convincing this community will be, not necessarily having watertight arguments for any one background premise, but doing a thorough enough job at addressing the wide range of background disagreements.

I don't know about the community as a whole, so speaking just for myself, I think I'd require at least intuitive arguments (no RCTs needed) for each of the following premises to consider working on this--I'll leave it to you to judge whether that's a reasonable burden of proof:

  • That a technologically sophisticated future would be bad overall in expectation (i.e. (1)), even under ethical views that are most common in this community
  • That permanently, globally halting technological progress is tractable (in a way that isn't highly objectionable on other grounds)
  • That sufficiently improving the trajectory of AI is relatively intractable (or that AI in particular shouldn't be a dominant consideration)
  • Bonus points: directly addressing heuristics that make techno-pessimism very unintuitive (especially the heuristic that "ban all new technologies" is so un-nuanced/general that it's very suspect)
Comment by Mauricio on Do you have an example impact calculation for a high-impact career? · 2021-12-20T00:58:26.360Z · EA · GW

Hm, this seems tricky for an intro pitch, if we're talking about careers in areas this community considers especially high-impact:

  • In global development, non-earning-to-give careers in the community seem to be rare and pretty different from one another, so estimating the (near-term) impact of a typical such career seems tough.
  • In existential risk reduction, it seems like quantifying the value of anything quickly gets into huge numbers, lots of guesswork, and weird questions, while in factory farming, quantifying impacts quickly gets into tricky inter-species comparisons and focusing on non-human animals. For an intro pitch, my intuition would be to lead with better-grounded and more intuitive content.

On another note: on the off chance you haven't seen them, you might find the short introductory presentations here useful.

Comment by Mauricio on High School Seniors React to 80k Advice · 2021-12-17T01:23:10.809Z · EA · GW

Thanks for sharing this!

I can imagine people coming away from this with the impression that impact-oriented career communications like those of 80K should change their framings to better pre-empt these reactions, e.g. by more strongly emphasizing that taking on big problems is not for everyone (not because the author explicitly drew this conclusion, but because it seems like a natural one). It's pretty non-obvious to me that this is a right takeaway. Arguably, a majority of the impact of places like 80K comes from supporting people who are very dedicated to impact. Catering to audiences with lukewarm interest in impact will have some benefits, but I worry these might come at the cost of e.g. 80K failing to do very well at motivating and guiding people who are most excited to prioritize impartial positive impact. At least personally, I found it very motivating to come across a site that works with assumptions like "of course we're happy to take on big problems/responsibilities--how could we look away and do nothing, when these problems are out there?"

A bit more about where my intuitions are coming from:

  • I suspect there's lots of motivated reasoning behind objections 1-5 (since 1 & 4 are strawmen, 2 is name-calling, and 5 is irrelevant) . Addressing weak objections that come from motivated reasoning seems like a doomed time sink--if people are looking for reasons to believe something, they'll find them.
  • I've seen groups that focus on supporting people in getting into high-impact careers spend lots of time trying to engage people who aren't that interested. It doesn't seem to have paid off much--people who aren't all that interested seem to drift away often, and not do very impact-targeted work even when they do remain somewhat engaged.
  • "if you are able to be exceptional at an important "normal" job or be high-impact within that career, that can be good too"--some version of this seems plausible, but as stated this feels close to watering down career advice in a way that seems very risky (by de-emphasizing the potentially huge difference between different career paths' impacts).
Comment by Mauricio on My Overview of the AI Alignment Landscape: A Bird’s Eye View · 2021-12-16T03:32:38.381Z · EA · GW

Thanks for writing this! Seems useful.

Questions about the overview of threat models:

  • Why is it useful to emphasize a relatively sharp distinction between "treacherous turn" and "you get what you measure"? Since (a) outer alignment failures could cause treacherous turns, and (b) there's arguably* been no prominent case made for "you get what you measure" scenarios not involving treacherous turns, my best guess is that "you get what you measure" threat models are a subset of "treacherous turn" threat models.
  • Why is it useful to think of AI-influenced coordination failures as a major threat model in the alignment landscape? My intuition would be to think of it as falling under capabilities (since the worry, if I understand it, is that--even if AI systems are aligned with their users--bad things will still happen because coordination is hard?).

*As noted here, WFLLP1 includes, as a key mechanism for bad things getting locked in: "Eventually large-scale attempts to fix the problem are themselves opposed by the collective optimization of millions of optimizers pursuing simple goals." Christiano's more recent writings on outer alignment failures also seem to emphasize deceptive/adversarial dynamics like hacking sensors, which seems pretty treacherous to me. This emphasis seems right; it seems like, by default, any kind of misalignment of sufficiently competent agents (which people are incentivized to eventually design) creates incentives for deception (followed by a treacherous turn), making "you get what you measure" / outer misalignment a subcategory of "treacherous turn."

Comment by Mauricio on Celebrating 2021: What are your favourite wins & good news for EA, the world and yourself? · 2021-12-14T21:54:35.704Z · EA · GW

I'm excited by the recent explosion of activity aimed at supporting people in entering impactful careers, e.g.:

  • The number of people doing such work at universities roughly doubled this past year, the amount of support going to each of these campus organizers also greatly increased, and relevant organizations have big plans to further build relevant, professionalized university centers.
  • A bunch of programs that support undergrads in learning about some pressing problems / getting relevant research experience all started or greatly scaled this past year, e.g.: the Stanford, Cambridge, and Swiss Existential Risk Initatives' summer research internships; global introductory courses on important fields; and a high-impact internship board for undergrads.
  • 3+ programs aimed at supporting impact-driven high schoolers launched.
  • The OpenPhil Tech Policy Fellowship, the ML for Alignment Boot Camp, and the ML Alignment Theory Program--all of which were launched this year--are collectively supporting dozens of people with intensive training/support for entering impactful careers.

Broadly makes me optimistic that the movement will do well at turning its funding overhang into impactful, direct work.

Comment by Mauricio on Aiming for the minimum of self-care is dangerous · 2021-12-13T02:43:01.168Z · EA · GW

Ah, good points!

Comment by Mauricio on Takeaways on US Policy Careers (Part 1): Paths to Impact and Personal Fit · 2021-12-12T20:43:26.567Z · EA · GW

Thanks for the kind words and for sharing your experience!

Comment by Mauricio on Aiming for the minimum of self-care is dangerous · 2021-12-12T08:22:25.189Z · EA · GW

Thanks for this! Broadly agree on the importance of sustainability. I also want to push back on generally framing this movement as just "doing good better." If we agree with Tessa that "some good things you can do matter 100x or 1000x more than others," then I'd worry that just "doing good better" makes it too easy for people to have just 1/100th or 1/1000th of the positive impact they could have (since they can easily be satisfied with doing the less important things better, or with doing the important things just somewhat better). And that seems like a major waste of potential.

(Even in the context of donations, it seems very suggestive that ~85% of this movement's funding comes from two sources that are very big on optimizing. I'd hate to have missed out on them.)

(More broadly, if most impact comes from hitting small targets, then it seems like we'll have a hard time getting this impact without optimizing. So I'm much more optimistic about "optimize with self-care heuristics" than about "satisfice.")

Comment by Mauricio on AI Governance Course - Curriculum and Application · 2021-12-12T05:23:50.244Z · EA · GW

Thanks! I expect we'll get them out by the end of the month (hopefully on the week after the application deadline), although there's enough potential bottlenecks that I can't promise that.

Comment by Mauricio on Biblical advice for people with short AI timelines · 2021-12-07T00:04:13.917Z · EA · GW

Thanks for the detailed response / sharing the resources! I'm familiar with them (I had been wondering if there was a version of (a) that didn't involve the following modification, although it seems like we're on a similar page)

To clarify, what I should have said is that while such an outcome could appear to be an error on the part of the AGI, it would really be a human error

Comment by Mauricio on Biblical advice for people with short AI timelines · 2021-12-06T20:15:04.039Z · EA · GW

Yeah--is your sense that "enslave everyone" (in the context of what humans to humans) feels like an especially good handle on either of those scenarios? (That's all I initially meant to nitpick--not whether such scenarios are plausible.)

Another nitpick: actually, I haven't heard about (a) as described here--anything you'd suggest I look at? (I'm initially skeptical, since having such a mistaken conception for a long time doesn't seem all that superintelligent to me. Is what you had in mind scenarios in which torture is motivated by strategic extortion or maybe sadism, since these don't seem to require a mistaken conception that it's helping?)

Comment by Mauricio on Is the doing the most good really that simple? · 2021-12-06T10:48:49.415Z · EA · GW

Thanks for the question! You might find articles like this one interesting. (That article's a bit outdated but I'd guess still roughly right.)

Comment by Mauricio on Biblical advice for people with short AI timelines · 2021-12-05T23:56:09.766Z · EA · GW

or b) kill/enslave everyone

Tangent: did you mean this literally? I know some folks who are worried about people being killed, but I haven't heard of anyone worrying about human enslavement (and that distinction seems like a point in favor of "people worried about this stuff aren't picking scary-sounding scenarios at random," since automated labor would presumably be way more efficient than human labor in these scenarios).

Comment by Mauricio on Takeaways on US Policy Careers (Part 2): Career Advice · 2021-11-30T20:34:29.834Z · EA · GW

Thanks! I've added a caveat to the post, linking to this comment.

Comment by Mauricio on AI Governance Course - Curriculum and Application · 2021-11-29T22:07:36.182Z · EA · GW

Thank you! That would be great.

(I'm not sure I'd have capacity to manage hosting of the audio files, so no worries if that would be a requirement (although my sense was that it generally isn't, with Nonlinear?))

Comment by Mauricio on How would you define "existential risk?" · 2021-11-29T06:40:41.549Z · EA · GW

Of the definitions proposed in this paper, I like the clarity and generality of the 3rd: Existential risk is risk of an existential catastrophe, and:

An existential catastrophe is an event which causes the loss of a large fraction of expected value.

(I'd count non-events, since having another term for those isn't catching on, and I'd append "for the long-term future.")

Comment by Mauricio on The Explanatory Obstacle of EA · 2021-11-28T23:16:09.679Z · EA · GW

Seems right; maybe this was implied, but I'd add (D) pick a cause & intervention that scores very well under scale/neglectedness/tractability/personal fit considerations

Comment by Mauricio on The Explanatory Obstacle of EA · 2021-11-28T11:01:31.324Z · EA · GW

Agreed! So maybe differences in feasible impact are: career >> (high-skill / well-located) volunteering > donations >> other stuff

Comment by Mauricio on The Explanatory Obstacle of EA · 2021-11-27T20:58:36.372Z · EA · GW

Yup, this also lines up with how (American) undergrads empirically seem to get most enthusiastic about career-centered content (maybe because they're starved for good career guidance/direction).

And a nitpick:

In most cases, individuals can do much more good by changing their career path or donating

I initially nodded along as I read this, but then I realized that intuition came partly from comparing effective donations with ineffective volunteering, which might not be comparing apples to apples. Do effective donations actually beat effective volunteering? I suspect many people can have more impact through highly effective volunteering, e.g.:

  • Volunteering in movement-building/fundraising/recruitment
  • High-skill volunteering for orgs focused on having positive long-term impacts, or potentially for animal advocacy orgs (since these seem especially skill-constrained)
  • Volunteering with a mainstream policy org to later land an impactful job there (although this one's iffy as an example since it's kind of about careers)

(Still agree that emphasizing volunteering wouldn't be very representative of what the movement focuses on.)

Comment by Mauricio on Should I go straight into EA community-building after graduation or do software engineering first? · 2021-11-26T12:07:18.030Z · EA · GW

Maybe you're already thinking this, but specifically for community building / meta-EA career paths, my impression is doing 2 years of meta-EA would be much better (in terms of both career capital and direct impact) than 2 years of software engineering at a startup. Intuitions behind this include:

  • Community building experience seems to be highly valued by meta-EA employers / grantmakers / cofounders, because of its clear relevance
  • Maybe the impacts of community building are cumulative, which makes additional work early on especially valuable
  • I'd guess recent graduates tend to have a much easier time becoming friends with university students than older people (and anecdotally, these friendships seem super important for community builders' impact)
  • Getting more meta-EA experience soon would help prepare you to take on management/mentorship roles or otherwise more ambitious projects in meta-EA sooner. More generally, if impact per year grows as one gets more relevant experience, then starting earlier lets you have more time at the heights of your ability to have impact.

(I guess the exception would be if you want to do meta-EA through software engineering.)

Comment by Mauricio on Is it no longer hard to get a direct work job? · 2021-11-26T04:38:50.410Z · EA · GW

In particular, just in the case of uni EA groups, I imagine that there might be one organizer for every, say, 20 to 50 people (?? I really have no idea about this), which is also a ratio of 2 to 5%.

Anecdotally, my (potentially skewed) personal impression is that [students who are very dedicated, hard-working, decent fits for university organizing, and apply for grants to do university group organizing] have chances > 50% of getting some grant.

(By "very dedicated," here and in the other comment, I mean to point at something like: has a solid understanding of and intense commitment to core ideas and mindsets of the movement, e.g. prioritization with reasoning & evidence--not just excitement about doing good more effectively.)

Comment by Mauricio on Is it no longer hard to get a direct work job? · 2021-11-26T04:28:28.879Z · EA · GW

As a minor point, I'd consider potential publication biases when interpreting articles about how hard it can be to get these jobs. I imagine if someone had an easy time getting one of these jobs, they might be hesitant to write a post about it, to avoid looking self-celebratory or insensitive.

Comment by Mauricio on Is it no longer hard to get a direct work job? · 2021-11-26T04:15:50.115Z · EA · GW

My impression is Michael's update could easily be directionally correct if we refine that estimate.

  • If we count direct work in non-EA orgs (which Michael seemed interested in), this opens many more options; ~34% of survey respondents (11.7 + 8.7 + 5.6 + 4.2 + 3.9) seem to be doing such work, although it's unclear how many of them are working on causes they see as most pressing.
  • The 2020 survey of the community found that ~20% of respondents self-reported "high engagement" with EA. (And that's likely an overestimate due to survey selection effects.) This knocks down our denominator by a factor of ~5, approaching the possibility you raise that maybe "'hard work and dedicated'" does convey 2 to 10% to your audience."

And now some anecdotal evidence:

I don't get the impression that the number of "EA jobs" has literally doubled in the past year

Maybe not doubling across the board, but some areas (especially within meta-EA and longtermism) seem to be growing very quickly:

  • CEA's number of staff nearly doubled this past year
  • The number of CEA focus universities with community building grantees roughly doubled this past year (as a proxy for how the number of grantees has changed), with plans for creating more full-time work opportunities for university organizers
  • Redwood Research started up, announced plans for some exponential growth, and has already hired 10+ people to do AI safety research
  • Rethink Priorities also doubled in staff size this past year

What makes me most optimistic is probably that, of a handful of recent graduates I happen to know who are very dedicated to meta-EA and/or longtermism, a large majority have been able to get either employment or grants to do direct work (typically following ~1-2 years building relevant skills / track records, often while at university). (Caveat: most of them are from "top schools" and/or live near EA hubs, which likely helps.)

Comment by Mauricio on What is most confusing to you about AI stuff? · 2021-11-23T22:47:12.031Z · EA · GW

I've since gotten a bit more context, but I remember feeling super confused about these things when first wondering how much to focus on this stuff:

  1. Before we get to "what's the best argument for this," just what are the arguments for (and against) (strongly) prioritizing AI stuff (of the kind that people in the community are currently working on)?
    1. People keep saying heuristic-y things about self-improving AI and paperclips--just what arguments are they making? (What are the end-to-end / logically thorough / precise arguments here?)
    2. A bunch of people seem to argue for "AI stuff is important" but believe / act as if "AI stuff is overwhelmingly important"--what are arguments for the latter view?
    3. Even if AI is overwhelmingly important, why does this imply we should be focusing on the things the AI safety/governance fields are currently doing?
    4. Some of the arguments for prioritizing AI seem to route through "(emerging) technologies are very important"--what about other emerging technologies?
    5. If there's such a lack of strategic clarity / robustly good things to do in AI governance, why not focus on broadly improving institutions?
    6. Why should we expect advanced AI anytime soon?
  2. What are AI governance people up to? (I.e. what are they working on / what's their theory of change?)
  3. What has the AI safety field accomplished (in terms of research, not just field-building)? (Is there evidence that AI safety is tractable right now?)
  4. A lot of the additional things I found suspect were outside-view-y considerations / "common sense" heuristics--to put it in a very one-sided way, it was something like, "So you're telling me some internet forum is roughly the first and only community to identify the most important problem in history, despite this community's vibes of overconfidence and hero-worship and non-legible qualifications and getting nerd-sniped, and this supposedly critical problem just happens to be some flashy thing that lines up with their academic interests and sounds crazy and isn't a worry for the most clearly relevant experts?")

(If people are curious, the resources I found most helpful on these were: this, this, and this for 1.1, the former things + longtermism arguments + The Precipice on non-AI existential risks for 1.2, 1.1 stuff & stuff in this syllabus for 1.3 and 3, ch. 2 of Superintelligence for 1.4, this for 1.6, the earlier stuff (1.1 and 3) for 4, and various more scattered things for 1.5 and 2.)

Comment by Mauricio on The Case for Reducing EA Jargon & How to Do It · 2021-11-23T04:48:51.506Z · EA · GW

Thanks Akash! This seems clear to me when it comes to communicating with people who are new to the community / relevant jargon. Just to clarify, would you also advocate for reducing jargon among people who are mostly already familiar with it? There, it seems like the costs (to clarity) of using jargon are lower, while the benefits (to efficiency and--as you say--sometimes to precision) are higher.

(I'd guess you're mainly talking about communication with newer people, but parts like "Ask people to call out whenever you’re using jargon" make me unsure.)

(I also suspect a lot of the costs and benefits come from how jargon affects people's sense of being in an in-group.)

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-21T23:14:50.261Z · EA · GW


Yup, to be clear I didn't mean to suggest "more of the same," although you're right that my examples near the end may have been overly anchored to the events fellowships currently have.

a more effective way of learning might be for someone to summarize things / identify key ideas for you

Hm, maybe. One hypothesis is that people tend to understand and remember ideas much better if they engage with them for longer amounts of time. If true, I think this would mean more (good) content is better. This seems likely to me because:

  • It seems much more common for people to have big life/worldview changes from books than from talks or articles.
  • Collections of somewhat detailed readings let people check links, see responses to a wide range of counterarguments, look things up if they're missing context, and more generally get a more thorough version of an argument.
  • A bunch of core motivations for EA and its cause areas are potentially paradigm-shifting, so they seem especially hard for people to quickly slot into their existing worldviews.
  • More time spent on an idea --> more attention spent on it, more chances to make downstream updates
  • Much of the K-college educational system seems built on this assumption (which is definitely not rock-solid evidence, but it's some evidence)
  • The spacing effect is a thing (unless that too has failed to replicate?)

So I'm skeptical that people can "really get" ideas like "we're always in triage," or "maybe animals matter," or "maybe we should think a lot about the future" from just brief summaries. (Brief summaries accompanied by things that motivate people to look into things more deeply on their own seem great, if we can pull that off.)

So I'm still hesitant about replacing content with projects. Still excited about:

  • Content + additional ways to dive in
  • Or replacing some content with other activities that encourage deep engagement (e.g. certain retreats) if we can figure out good follow-up

(I'm also not sure about the self-directed fellowship format--we could mitigate the relevant downsides by adding accountability measures, but doesn't that largely bring it back to being a not-so-self-directed fellowship? We could also do more individualized accountability like 1-on-1's, but that's significantly more costly.)

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-21T07:22:59.336Z · EA · GW

Tangent/caveat to my point about practice: Actually, it seems like in the examples I mentioned, practicing on easier versions of a problem first is often very helpful for being able to do good practice on equivalents of the real thing (e.g. musical scale drills, sport drills, this). I wonder what this means for EA groups.

(On the other hand, I'm not sure this is a very useful set of analogies--maybe the more important thing for people who are just getting into EA is for them to get interested in core EA mindsets/practices, rather than skilled in them, which the "practice" examples emphasize. And making someone do scale/sports drills probably isn't the best way to get them interested in something.)

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-21T06:45:34.984Z · EA · GW

Good points! Agree that reaching out beyond overrepresented EA demographics is important--I'm also optimistic that this can be done without turning off people who really jive with EA mindsets. (I wish I could offer more than anecdotes, but I think over half of the members of my local group who are just getting involved and seem most enthusiastic about EA stuff are women or POC.)

I'm not convinced that weird people know how to do good better than anybody else

I also wouldn't make that claim about "weird people" in general. Still, I think it's pretty straightforward that people who are unusual along certain traits know how to do good better than others, e.g. people who are unusually concerned with doing good well will probably do good better than people who don't care that much.

I don't want to miss out on these people, even if it's a more efficient allocation of time/resources to only focus on people we expect will become highly engaged.

Man, I don't know, I really buy that we're always in triage, and that unfortunately choosing a less altruistically efficient allocation of resources just amounts to letting more bad things happen. I agree it's a shame if some well-off people don't get the nice personal enrichment of an EA fellowship--but it seems so much worse if, like, more kids die because we couldn't face hard decisions and focus our resources on what would help the most.

Edit: on rereading I realize I may have interpreted your comment too literally--sorry if I misunderstood. Maybe your point about efficient allocation was that some forms of meta-EA might naively look like efficient allocation of resources without being all that efficient (because of e.g. missing out on benefits of diversity), so less naive efficiency-seeking may be warranted? I'm sympathetic to that.

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-21T00:18:00.544Z · EA · GW

Thanks for the thoughtful response! I think you're right that EA projects being legibly good to people unsympathetic with the community is tough.

It is practice at what this process looks like, it is a way to improve our community in a small but meaningful way

I like the first part; I'm still a bit nervous about the second part? Like, isn't one of the core insights of EA that "we can and should do much better than 'small but meaningful'"?

And I guess even with the first part (local projects as practice), advice I've heard about practice in many other contexts (e.g. practicing skills for school, or musical instruments, or sports, or teaching computers to solve problems by trial and error) is that practice is most useful when it's as close as possible to the real thing. So maybe we can give group members even better practice by encouraging them to practice unbounded prioritization/projects?

I think it makes for crappy advertising because most people don't think about the world in Expected Value

There's a tricky question here about who the target audience of our advertising is. I think you're right that working on mainstream/visible problems is good for appealing to the average college student. But, anecdotally, it seems like a big chunk (most?) of the value EA groups can provide comes from:

  • Taking people who are already into weird EA stuff and connecting them with one another
  • And taking people who are unusually open/receptive to weird EA stuff and connecting them with the more experienced EAs

And there seems to be a tradeoff where branding/style that strongly appeals to the average student might be a turnoff for the above audiences. The above audiences are of course much smaller in number, but I suspect they make up for it by being much more likely to--given the right environment--get very into this stuff and have tons of impact. Personally, I think there's a good chance I wouldn't have gotten very involved with my local group (which I'm guessing would have significantly decreased my future impact, although I wouldn't have known it) if it hadn't been clear to me that they were serious about this stuff.

I agree that a dollar would go much further in terms of extreme poverty than college students, but I'm less sure about an hour of time

That's fair. I guess we could say one could always spend that hour making extra money to give away, although that's kind of a copout (and doesn't address the optics issue).

(As a side note which isn't very decision-relevant / probably preaching to the choir, it really annoys me that some people think the anti-colonialist move is to let poor foreign kids die of malaria.)

Lastly, two other tactics for advertising/optics that I'm optimistic about:

  • With things like AI safety, I think you're right that most of the actual good done is just in expectation and won't be clear for a while. But I'm not sure it's only good in expectation--I'm optimistic that there's lots of potential for longtermist work to have good, higher-probability spillover effects in the near term. For example, even if work on AI interpretability doesn't help avoid AI deception, it may be more clearly a step toward mitigating algorithmic bias. Or even if work on truthful AI also doesn't help avoid AI deception, maybe it can help mitigate the misuse of AI to create misinformation. I imagine there's similar nice spillovers in biosecurity. Emphasizing such benefits / potential applications might be enough to appeal to risk-averse, scope-insensitive audiences.
  • I'm also optimistic that these mainstream audiences would see tangible "intermediate steps" toward impact as progress, even if they hadn't clearly paid out yet. E.g. I suspect "we ran a class on alternative protein, and it got 100+ students, and this is important for addressing sustainability and zoonotic disease and animal abuse" will sound like concrete, tangible impact to the not-so-analytical audiences we're talking about, even though its impact remains to be seen.
Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-20T07:28:39.979Z · EA · GW

Thanks for this! Tangent:

students are really excited about actually doing stuff. And this can be difficult to reconcile with EA. This semester, we decided to do Effectively Altruistic projects limited into the scope of our school (e.g., what can we do to improve student wellness the most? Decrease the school's carbon footprint? etc.).

Hm, I'm kind of nervous about the norms an EA group might set by limiting its projects' ambitions to its local community. Like, we know a dollar or an hour of work can do way more good if it's aimed at helping people in extreme poverty than US college students... what group norms might we be setting if our projects' scope overlooks this?

At the same time, I think you're spot on in seeing that many students want to do projects, and I really appreciate your work toward offering something to these students. As a tweak on the approach you discuss, what are your intuitions about having group members do projects with global scope? I know there's a bunch of EA undergrads who are working on projects like doing research on EA causes, or running classes on AI safety or alternative proteins, or compiling relevant internship opportunities, or running training programs that help prepare people to tackle global issues, or running global EA outreach programs. This makes me optimistic that global-scope projects:

  • Are feasible (since they're being done)
  • Are enough to excite the students who want to get to doing stuff
  • And have a decent amount of direct impact, while reinforcing core EA mindsets
Comment by Mauricio on EA Communication Project Ideas · 2021-11-20T02:40:28.328Z · EA · GW

Thanks for this!

Make a written intro similar to Ajeya's talk

This script and these slides are heavily inspired by (and are several years more recent than) her talk--might be useful for someone who wants to do this.

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-20T02:34:32.936Z · EA · GW

Ah got it - thanks!

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-19T23:44:54.565Z · EA · GW


Sorry, I'm a bit confused about how this relates to my response. It sounds like this is an argument for changing the distribution of content within the current fellowship structure, while my response was meant to be about which changes to the fellowship structure we should make. (Maybe this is meant to address my question about "what [content] can be cut?" to implement an activities-based fellowship? But that doesn't seem like what you have in mind either, since unlike in the activities-based fellowship you seem to be suggesting that we keep the total amount of readings roughly constant.) So I'll interpret your comment as an independent case for changing the fellowship content, holding structure constant (rather than as a case for some of the alternative structures proposed in the original post)--let me know if I've misunderstood!

I'd mostly be on board with shifting to more materials that convey core principles/mindsets, if we had promising guesses about how to implement this. My main hesitation: (1) I don't yet know of additional content that would do this well, and--in the absence of that opportunity cost--(2) the object-level content seems pretty good.

Do you have specific ideas for epistemics/mindset content in mind? I share your interest in adding more such content, but specifically in epistemics I've had trouble finding satisfactory content. These are the challenges I've come across:

  • Academic sources that cover epistemics tend to be super long and dry for our target audience
  • LessWrong Sequences content is spread out among a bunch of small posts that very much build on one another, so one-off readings will often transmit little knowledge
  • Clearer Thinking doesn't seem to have that much epistemics-focused content, and its most relevant content is often relatively niche and long
  • ACX/SSC is very long/tangential, is more controversial, and doesn't have that much epistemics-focused content
  • HPMOR / other online fiction is hard to heavily emphasize if we're trying to signal professionalism/legitimacy
  • There's good one-off content on a few things, e.g. Bayes' Theorem and cognitive biases, but I'm skeptical that these are valuable enough readings to be worth the opportunity cost (skeptical about reading about Bayes' Theorem because people already roughly apply it intuitively/roughly, while applying it explicitly/precisely is usually intractable; skeptical about cognitive biases readings because the literature suggests we can't do that much about them).

With mindset/motivational content, we've added all the best stuff I'm aware of--curious what else we can add!

I also think the object-level content about cause areas is fairly valuable:

  • Main EA cause areas are (by design) very unconventional/neglected. So I worry that people might never come across strong arguments for each--or bother to engage with them--if these aren't put in front of them. Or they might want to engage with the strong versions of these arguments but not know where to find them (sure, they're somewhere in the EA-sphere, but how will new people know where?)
  • I'm a bit skeptical of the very sharp distinction between mindsets and cause areas--cause areas provide (a) examples of mindsets/principles (e.g. looking for large-scale problems) being applied, and (b) opportunities to apply mindsets/principles (e.g. "Here are two compelling causes--which should we prioritize? What does this mean for your career?")

(I also agree the exercises aren't great, although my sense was that most fellows and facilitators mostly ignore them, so for us they don't currently seem to be a big part of the fellowship.)

Comment by Mauricio on We need alternatives to Intro EA Fellowships · 2021-11-19T05:03:08.339Z · EA · GW

Thanks! I'm sympathetic to the broad idea here, but the pitfalls you point out seem pretty significant (maybe less so for the 3-week version, but that one also seems most similar to the current structure).

My main hesitation with activity-based fellowships is that intro fellowships are already pretty light on content (as you point out, they could fit in a busy weekend), so I suspect that cutting content even more would mean leaving even more massive gaps in participants' knowledge of EA. (Right now, content is roughly an intro to core EA mindsets and an intro to the main cause areas--what can be cut?) Then, I'd worry that by default we'd get a bunch of people doing vaguely EA-related projects, mostly dropping off after a bit (since doing independent projects is hard for many people), and not having read much EA-specific content. E.g. maybe someone will look into how to efficiently prevent floods in wealthy countries and just read about that for a few weeks.

That said, it seems like you're right that we probably could and should do much better than the status quo. To vaguely gesture toward potential ideas:

  • Most of my worries about alternatives 1-3 come from the intuition that students will get busy with and prioritize classes, so maybe there's room for more of these accelerated programs to take place over times when students don't have classes? (Although then it's also harder to do in-person interaction.) Or maybe setting it up as a (paid?) larger time commitment (maybe framed as an "internship" or something?) would make students more willing to make and stick to that commitment?
  • Maybe there's things we can add to the current intro fellowship, to keep the current baselines of accountability and content but add more optional ways for people to quickly dive in? (E.g. more social events, more opportunities to start organizing, more workshops, etc.)

I'd also be very curious to hear more details about your own (or others') experience with the fellowship! E.g. which of the limitations of the fellowship felt most salient for you?

Comment by Mauricio on Penn EA Residency Takeaways · 2021-11-18T06:32:40.890Z · EA · GW

Thanks for the detailed reflection! Nitpick, for the sake of readers having more complete info about the track record of residencies: I think it was Georgetown that got 1 Stanford EA organizer (who prefers to not be named here) for a couple of days. My understanding is this also wasn't enough time for them to do that much. (1 FTE also feels high, since there weren't enough ongoing group activities / emails in the mailing list / availability from local organizers for the organizer to spend more than several hours per day helping with Georgetown EA.)

The other data points of residencies I'm aware of are:

  • 2 organizers went to Princeton for about a week (more info in this comment)
  • Several organizers went to Boston, focusing on MIT for times ranging from several weeks to a few months
  • One organizer has been at Berkeley for a few months
Comment by Mauricio on Takeaways on US Policy Careers (Part 2): Career Advice · 2021-11-11T21:55:57.598Z · EA · GW

Thanks for flagging!

Things may have changed since then. Also, at the time this was more true at Brookings

As you suggest, this other post cites a 2017 statistic which suggests this is still the case about Brookings and is becoming less true about think tanks in general (although the statistic is about "scholars" rather than "senior scholars"):

Among a representative group of think tanks founded before 1960, for instance, 53% of scholars hold PhDs. Among a similarly representative group of think tanks founded between 1960 and 1980, 23% of scholars have such advanced degrees. And among those founded after 1980, only 13% of scholars are as highly educated.

Comment by Mauricio on Takeaways on US Policy Careers (Part 2): Career Advice · 2021-11-11T21:45:40.964Z · EA · GW

Thanks for catching that! I think it should be fixed now.

Comment by Mauricio on Many Undergrads Should Take Light Courseloads · 2021-11-08T03:09:17.698Z · EA · GW

I'd be curious to hear from someone who knows more about graduate programs (in general or specific ones) to what extent this advice generalizes to those contexts.